The hardest part of creating a ci has nothing to do with compressing them. The hardest part is to make a self-contained application under /opt. Because openoffice and firefox are pretty much self contained and are under /opt they were good test cases. I have since made python2.3.4 which works under /opt so to make it a ci is easy. Also, the ci has a limitiation of being read only. That is why there is an oouser.tar.gz. Those parts that require updates can be "linked" into a writeable directory. I see user Chris V made a firefox user file. Try to avoid making .dsl as they cause additional demands on the system. Sometimes, you will be required to make a shell wrapper to start the app. As being self contained implies that the app's libraries are also stored locally with the app. This usually implies requiring a LD_LIBRARY_PATH, thus the shell wrapper. Anyway, make the app. Test it as a .tar.gz. Once the app is working then to make a ci do the following.
1. The app is installed into /opt in the your_app directory
1. become root
2. cd /opt
3. mkisofs -R -hide-rr your_app | create_compressed_fs - 65536 > /mnt/somedrive/your_app.ci
Note: there is currently a limit of 4 cloop devices available. So, I don't see a need for many .ci files unless the app is very large and is very popular.
Follow-up there are now 8 devices for uci extensions.
Follow-up for UCI files
UCI files are ci files that store the user.tar.gz (writeable areas) inside the mounted ci with the specific name of user.tar.gz. Then when the uci is mounted the system looks insdide the mount for a user.tar.gz and then performs a mydsl-load on that tar.gz. This makes for an easy single file download and load.
To change/update a UCI file do the following: (example here is firefox.uci)
1. mount the uci as normal
2. mkdir work
3. cd work
4. cp -a /opt/firefox/.
5. cd firefox
6. mkdir user
7. cd user
8. tar -zxvf ../user.tar.gz
9. tar -ztf ../user.tar.gz > list
10. -- now edit the actual files as needed --
11. -- be sure to edit/update the "list"
12. tar -T list --numeric-owner --no-recursion -czvf ../user.tar.gz
13. cd ..
14. rm -rf user
15. cd ..
16. mkisofs -R -hide-rr-moved -cache-inodes -pad firefox/|create_compressed_fs - 65536 > ../firefox.uci
17. cd ..
20. rm -rf work
(Thanks to Robert Shingledecker)
INTRODUCTION
Before getting started, lets consider some reasons why we might choose to go with a *.tar.gz (or 'green') extension rather than the standard *.dsl way. First, there is the issue of memory usage. Installation of a *.dsl extension causes the execution of the script 'mkwriteable', which copies a larger part of the filesystem into the ramdisk to allow writing to these files. This of course uses more RAM. Opening up the additional files also opens up the system to potential damage. When a program or library can be overwritten, it can possibly cause problems. A *.tar.gz extension does not execute mkwriteable, and so base system libs and programs cannot be modified. These extensions write only to /opt, /var, and /home.
On the other hand, there are reasons for not choosing this method. Many programs are built in such a way that they have to modify certain files or write to certain directories which are not open to *.tar.gz. Or perhaps they don't need to be installed into these directories, but require some changes to these files in order to run properly. There is also the fact that a lot of people don't know how or just don't want to have to compile source, or maybe they don't have the tools to do so. It can be easier to gather the files installed though a package system and put them in the same places in a myDSL package.
Building a green extension can sometimes be as simple as packing up a pre-compiled binary with the appropriate ownership. More often, though, it's a bit more difficult. Since pre-compiled programs usually look for their support files in the directories to which they were installed (/usr/share, /usr/lib, etc.), it's sometimes better just to start from the source and compile it specifically to run in /opt. This procedure also allows you to make DSL-friendly changes to the program which might not be so simple after the application has been compiled.
Here is one method...Building From Source. This document assumes a program which can be installed through the use of a typical "configure, make, make install" method. There are other ways to install programs from source, and it would take a book to explain them all. At this time the majority of Linux applications use this method. Another assumption is that you have access to a computer system which has a development environment suitable for compiling Linux applications.
As an example, we'll be building a green extension from the ImageMagick source.
--------------------------------------------------------------------------------
READ THE DOCUMENTATION
First step is to do a little reading.
Grab the source archive of your program (imagemagick.org), extract the archive, and make sure to read the README file and any files dealing with license and copyright. Linux programs are not all open source, and doesn't always mean you can redistribute the package. GPL programs are a pretty safe bet that you can modify and redistribute.
Next to read is the INSTALL file, which sometimes contains just generic text, but often gives you some useful information about the specifics of the installation procedure, and finally, just before getting to work, run the configure script (if one exists) with the '--help' option. This will list all of the options you can send to configure so you might not have to go back and modify the Makefiles by hand. You may want to redirect the output of this command to a file, which you can have in sight while testing various configure options: "./configure --help > conf-help.txt". As with anything else, the more you compile, the more confortable you will become with it, and the more you will understand what you can do to build a better extension. One configure option which will almost always be used in a green extension is '--prefix=', which sets the install location of the application.
--------------------------------------------------------------------------------
CONFIGURE
Having done your research, feel free now to execute configure. In the directory containing this script, do "./configure --prefix=/opt/imagemagick" and it should begin checking your system for necessary components. If it spits up an error about not having a compiler, you don't have the needed development environment with which to compile applications. Unfortunately this how-to is not equipped to explain all the necessary steps to putting together a workable development system.
Assuming you have at least a basic development system set up, the script will likely give you feedback on your system, and eventually either end successfully or cough up an error telling you what you are missing. If you have successfully run configure, the script will have created the makefiles which make uses to compile the program. If you get errors
concerning a component which you believe is not vital to the program, you might try disabling that component and run configure again. The '--help' option of configure should tell you which features can be disabled this way (e.g. ./configure --prefix=/opt/imagemagick --disable-x). Otherwise you may need to install some support files. Again, this is beyond the scope of this document, and you really should consider learning a bit about compiling software in Linux.
--------------------------------------------------------------------------------
COMPILE
So let's just say configure went smoothly. You now can do 'make', and if that succeeds you can do 'make install' as root, installing the app into /opt/imagemagick. If you happen to have successfully compiled this program within a DSL system, you can probably launch the program without trouble. However, if the application was built on another system, you may have some serious testing ahead of you. If this is the case, copy the /opt/imagemagick directory into DSL and first see if DSL has the support files needed to run the application. You can do this by typing "ldd /opt/imagemagick/bin/display". This will list the libs required by the program, and tell you whether those libs are found in DSL. If you notice some "not found" messages concerning libraries which were installed into the program's lib directory, you may need to tell ldd that they exist:
LD_LIBRARY_PATH=/opt/imagemagick/lib ldd /opt/imagemagick/bin/display
The ldd program should now at least find any libs which were installed with the program. If you see additional "not found" messages, you'll now need to find those libs and copy them over to DSL. The most convenient place to put them would be in the same lib directory with the program's own libs. If the program does not have any libs, you can create an /opt/imagemagick/lib directory and place the additional libraries in there. At runtime you will most likely need to set the LD_LIBRARY_PATH to tell the program where to find the libs. Fortunately Imagemagick builds and links its own libs, so any extra libs can be dropped into its lib directory without needing to change the LD_LIBRARY_PATH. In the case of ImageMagick there are a couple of libs required which are not part of the base DSL, so they will need to be copied from your development system into /opt/imagemagick/lib.
It is possible that your program requires a huge number of libraries, or very large libraries. In this case you may want to consider whether it might be a good idea to try to whittle off some fat. Think about what is necessary in a "Damn Small" system and what can be considered extra bulk. If some of these libraries are not terribly important and can be disabled through the configure script, it may not be a bad idea to go back and recompile the application without those libs.
--------------------------------------------------------------------------------
TEST THE APPLICATION
When you have all of the necessary libs copied into a directory in DSL where they can be found, you're ready to test the program itself. This testing should be done in an unmodified DSL system, such as one booted from liveCD without any restoration or extra software installed. From a terminal window (so you can see the output of the program), type the command to launch the program, including the path since /opt/imagemagick/bin is not in dsl's $PATH. Any error messages? Hopefully not. If so, try the LD_LIBRARY_PATH command mentioned in the last section to see what is missing in DSL. If you have all the necessary libs in a place where the program can find them, chances are your program will load. If not, you may have problems beyond the scope of this document, and may want to consider looking for help at damnsmalllinux.org or #damnsmalllinux irc channel on freenode.
If the application loads up fine, things are looking good. Please do not assume that this means everything is ready to go, however. There is always the possibility that a program will eventually call another program which is not required merely to start your program. It's always a good idea to thoroughly test your application's features and functions before assuming that it is complete. Open some files, save some files, edit files within your program, if that's what the program does. Whatever the program's use is, it should be tested with as many of its functions as possible.
In the spirit of keeping it small, you might also consider stripping your binary file(s), if you have the strip program handy. This will remove comments and debugging objects from the program, reducing its size. After doing this you should test more, just in case something went wrong.
--------------------------------------------------------------------------------
BUILD THE FILE STRUCTURE
Now, does your program work? Good. Now we can package it up as a DSL
extension.
First we decide whether or not to include an icon. If the program is a non-interactive console application there is not much reason for an icon. Most of the time, however, it will be an interactive program which may be launched from the desktop. Ultimately it comes down to whether or not you want to bother adding the icon, and whether you have a suitable image for it. Let's assume that you want an icon, just for the sake of this tutorial. The icon file must be in a format readable by xtdesk (png, gif, jpg, and xpm can be used...probably some other formats as well), and is most conveniently placed in the /home/dsl/.xtdesktop directory. The only other file necessary is the *.lnk file which Xtdesk uses to place and name the icon on the desktop.
The lnk file is a simple text file with a lnk extension (something.lnk).
The format of the file is as follows:
table Icon
Type: Program
Caption: ImageMagick (the name which will appear next to the icon)
Command: /opt/imagemagick/bin/display (command to launch the program, in this case it is the imagemagick gui)
Icon: /home/dsl/.xtdesktop/imagemagick.xpm (path to the icon file)
X: 420 (X position)
Y: 384 (Y position)
end
There are other options available for the file, but this is suitable for a program launcher. If you needed to point the application to some libraries at runtime then you can add that path to the 'Command' line:
LD_LIBRARY_PATH=/opt/iamgemagick/lib /opt/imagemagick/bin/display
This sets the LD_LIBRARY_PATH variable specifically for this one application. Another option is creating a wrapper, as mentioned before. The wrapper simply contains a command or two to set up the environment, after which it launches the real program. Remember, ImageMagick has it's own linked libraries, so this should not be necessary this time.
Next we need to copy all the installed files into a directory structure with which to create a tarball. Since the application was installed entirely within a single directory(+ $HOME files where applicable) there's no need to hunt for the files.
Become root and create a directory somewhere out of the way, in $HOME for example.
Let's assume the work directory will be named '/home/dsl/work'.
sudo su
mkdir -p /home/dsl/work/opt # the opt part is just to save a step in the future.
cd /home/dsl/work/
IMPORTANT!!: Pay close attention to the paths you use. Everything done from this point is done from within /home/dsl/work/. A command done to './opt' or 'opt' is NOT the same as a command done to '/opt'. '/opt' is a system directory, and 'opt' or './opt' is the opt directory within /home/dsl/work.
If you have files going into /home/dsl you'll need to make this dir as well:
mkdir -p ./home/dsl/.xtdesktop # xtdesktop is of course for your icon, if any.
Now copy the installed files.
cp -Pr /opt/imagemagick ./opt/
cp -Pr /home/dsl/whatever ./home/dsl/ #only if there are other files needed in /home/dsl
If you want to have a menu item it will be placed in var/tmp/mydsl.menu/. The decision to create a menu item is much the same as the decision to use an icon. If the application is non-interactive and generally requires various commandline options, a menu item probably won't be useful.
Again, let's assume a menu item will be used:
mkdir -p var/tmp/mydsl.menu
touch var/tmp/mydsl.menu/imagemagick
var/tmp/mydsl.menu/imagemagick is the name of your menu item, used by mydsl-load. While at the time of this writing the menu item in a tar.gz extension does not need to have the same name as the package, it is recommended simply to be consistent with uci extensions (and because things may change in the future which require it).
The contents of the menu item are identical to the lines within the fluxbox
menu file:
[exec] (display name) {command}
If you need to specify a library path it can be done in the same way as in the icon file. There may be trouble with the menu item being used along with other DSL extensions, where two menu items end up on the same line in the menu and only the first one is displayed. This can be avoided by adding a line break after the menu line in var/tmp/mydsl.menu/imagemagick
Assuming all files are copied into their proper directory structure, it's now time to check ownerships of the files. Since root created a home/dsl dir, we know that at least this will need to be chowned to dsl. A quick way to do this is to simply give root ownership to everything and then change the ownership of the few dsl-owned files afterward.
/opt and everything within it is owned by root.root (0.0)
/home is owned by root.root (0.0)
/home/dsl and everything in it is owned by dsl.staff (1001.50)
/var and everything in it, except for the menu file itself, is root.root (0.0)
This should give proper ownership of all files:
chown -R 0.0 ./{opt/,var/}
chown -R 1001.50 ./home/dsl/
chown 1001.50 ./var/tmp/mydsl.menu/something
We use numeric ownership in the event that the system on which the extension is being built does not have user 'dsl' and group 'staff'.
--------------------------------------------------------------------------------
CREATE FILE LIST
Next we need to create a file list of everything which will go into the package.
find . > files.txt
This uses the find command to create a file list of everything in the current directory and below. Now we need to make some edits in order to remove lines which are either unnecessary, or could possibly cause trouble with ownerships.
Open files.txt in a text editor and start removing some selective lines. Depending on how many files are included, this may be a tedious process.
The file will look something like this (numbers are added here for reference):
1 .
2 ./files.txt
3 ./home
4 ./home/dsl
5 ./home/dsl/.xtdesktop
6 ./home/dsl/.xtdesktop/imagemagick.xpm
7 ./home/dsl/.xtdesktop/imagemagick.lnk
8 ./var
9 ./var/tmp
10 ./var/tmp/mydsl.menu
11 ./var/tmp/mydsl.menu/imagemagick
12 ./opt
13 ./opt/imagemagick
14 ./opt/imagemagick/bin
15 ./opt/imagemagick/bin/display
16 ./opt/imagemagick/man
17 ./opt/imagemagick/man/man1
18 ./opt/imagemagick/man/man1/display.1
1 You absolutely should remove this. Installation will be performed
from the root directory, which already exists. You do not want to
overwrite this directory, particularly if you made a mistake with
permissions. Delete this line.
2 This is the file list...not wanted inside the archive. Delete.
3-5 Same as 1 - any directory which already exists is not needed.
6&7 These ones you want to keep.
8&9 Same as 1
10 This will be created automatically with 11, so it's not necessary
11 Keep this one.
12-14 Delete.
15 Keep.
16-18 Delete. Man pages can be found in hundreds of places online.
Keeping your extension "damn small" is the name of the game. The same
can be applied to other help documents, includes, and anything else which
isn't vital to running the program. Testing the app without certain parts
available will help determine what is not terribly important.
So what you would end up with in this case is 4 lines, 6,7,11, and 15. Of course this is only a simple example, and is not complete even for the imagemagick package, but the method can be applied to larger lists.
--------------------------------------------------------------------------------
CREATE THE ARCHIVE
tar cvf imagemagick.tar --no-recursion --numeric-owner -T files.txt
gzip -9 imagemagick.tar
Now you're ready to test the final product. Hopefully it installs properly with the MyDSL system, including any menu items and icons you may have used. If it fails to run properly, or if other programs stop working after the installation, it's likely that there was a problem with ownerships of the files in the archive. You'll need to go back a few steps and carefully check your files.
If this is an extension for the official DSL repository, be sure to create an info file (check some of the others for reference) and md5sum:
md5sum imagemagick.tar.gz > imagemagick.tar.gz.md5.txt
Submitted by Mikshaw