HOWTO for creating .uci extensions?Forum: Other Help Topics Topic: HOWTO for creating .uci extensions? started by: whiteweasel Posted by whiteweasel on May 04 2006,00:58
I've read the HOWTO for creating .dsl extensions from Debian packages and it is easy enough. I'll be submitting a couple in the near future. Is there any documentation for creating .uci extensions? .dsl extensions from things not available as Debian packages? If so a link would be really, really helpful.Thanks. Posted by mikshaw on May 04 2006,01:22
There is some stuff in the wiki< http://damnsmalllinux.org/wiki/index.php/Category:MyDSL > Ideally I suppose there could be more info, but writing docs is no fun =o) Posted by pr0f3550r on May 04 2006,17:00
So:- .dsl is when you build from debs and .uci when you build from sources? - can you build .uci from .debs? - If so, why not obsolete .dsl's and promote only .uci's? - If so, why the .uci extensions intead of .tar.gz? - Don't you think .uci resemble, in some misterious ways, the Slackware package? Posted by cmanb on May 04 2006,18:27
Other people know a lot more about this than I do, so correct me if I'm wrong, but ... - If so, why not obsolete .dsl's and promote only .uci's? UCIs are very carefully contained under /opt by being mounted as a device. They are usually not "installed" at all. They are mostly Read-Only. DSLs install, I believe, system wide. Or at least wider than a UCI will. They are also "installed", in the traditional sense that files are written to the system. (Is it in loading a DSL that mkwritable is run?) - If so, why the .uci extensions intead of .tar.gz? For the same reasons listed above: UCIs are easly loaded (mounted), contained under /opt, can be unmounted, and are Read-Only. tar.gzs are none of these things. - .dsl is when you build from debs and .uci when you build from sources? Although deb2dsl makes the process simple, I don't think you have to have a deb package to create a dsl package. Could be wrong. And everything else, I don't know! Posted by mikshaw on May 05 2006,03:28
I think you're pretty much spot on. Building from source is not required of any extension type, and debs are not required to build *.dsl. The deb2dsl script uses deb because debian packages are precompiled to install into /usr, and installing a mydsl extension into /usr requires a *.dsl. However, with some tweaking you can usually make a debian package install into /opt...it will need some directory shuffling, and might need to include a wrapper script to set up the environment before the application runs. It's easier to build a *.dsl extension simply because the bin and lib directories used are in standard locations, and using a debian package the application is already built to run in those standard places. Installing an app into /opt/something/bin means that the program will need to be added to your $PATH or called with a full path, libs will usually need to be forced with the $LD_LIBRARY_PATH variable, and programs compiled to run elsewhere might be expecting their support files to be found in /usr. When compiling you can make it install where you want it, and often you can do it without needing a wrapper. Compiling usually also allows you to limit what is built into the program, so you may be able to prevent the need to include libs that are not already available in DSL.You can also take applications from another distro and see if they will run in DSL. Some will run fine, and other may need some help. Posted by pr0f3550r on May 05 2006,12:04
I'm confused. I haven't found any other reference on how to create a .dsl package other than from debs.Shouldn't there be consistency on how to create an extension? So, how do you create a .dsl from source and how would you create an .uci from debian packages? A Linux distro is all about package management, otherwise people would be happy with a shell and a compiler, like in 1992. Projecting this in a 5 year time seems too confusing to me. Posted by clacker on May 05 2006,16:41
There are three type of extensions: *.dsl, *.tar.gz, and *.uci (there was a *.ci but they aren't really around anymore). The difficult part about creating extensions is finding the files that go into them. That's why cbagger's < deb2dsl script > is so popular: it finds the files you need on it's own. That said, here are the mechanics of creating extensions, followed by some tip and tricks on making menus and icons, and finding files.The easiest extension to create is the *.dsl extension. Its files can go in any directory. The disadvantages are they load the files into memory (bad for low memory systems) and they can overwrite files that already exist (someone could overwrite an important file like /etc/mtab). Once you have a list of files (mine is called files) you create a *.dsl package using the following command line: tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files The second type of package is a *.tar.gz. It is created almost identically to a *.dsl file except the files it contains must all be located in the /opt, /home, /etc, /var, and /tmp directories. Again with a list of files called "files" you create the package using this command line: tar -czvf mypackage.tar.gz --numeric-owner --no-recursion -T files The third type of package is a *.uci. It also has the restriction that files must be in the /opt/mypackage (replace mypackage with a name you choose), /home, /etc, /var, and /tmp directories, but it also requires that the program can run correctly if /opt/mypackage is read-only. Many programs don't write their executable files so you could be OK, but you need to check it out. It looks harder to make, but it's fairly easy if you build a working *.tar.gz extension version first. I don't know about you, but I never get these right on the first shot. Something always needs tweaking and *.dsl and *.tar.gz extensions are far easier to tweak. If you do need to write to the /opt directory you'll still have a working tar.gz extension. The extension name, the directory under the opt directory, the menu file name, and the icon name must all be same. In my case this is "mypackage." The same way dsl and tar.gz files are a gzipped tar bundle, a uci file is a compressed file system created using mkisofs and create_compressed_fs. It is similar to the /KNOPPIX/KNOPPIX file which is created in a similar way. The compressed system is mounted to a directory create in /opt, and because the compressed file system is read only, you can't write to that directory. The uci's root is your tar.gz package's /opt/mypackage/ directory. Inside that compressed file system is anything you had originally in that directory, plus a user.tar.gz file which contains everything that needs to be in the /home or /tmp directories (it's your original mypackage.tar.gz without the opt/ directory). It sounds a lot more complicated than it really is. Lets say you have an extension named mypackage.tar.gz and it works the way you want it to. The following lines will convert your tar.gz extension into a uci extension: mkdir work sudo tar -xzvf mypackage.tar.gz --same-ower -C work tar -tzf mypackage.tar.gz | grep -v "^opt" | tee work/user.files cd work sudo tar -czvf opt/mypackage/user.tar.gz --numeric-owner --no-recursion -T user.files cd opt mkisofs -R -hide-rr-moved -cache-inodes -pad mypackage/ | \ create_compressed_fs - 65536 > ../../mypackage.uci Here is a first cut at a script that converts a *.tar.gz package into a *.uci package. It does absolutely no checking to see that you have kept the same names between the package, the /opt/package, the tmp/mydsl.menu/package, and the home/dsl/.xtdesktop/package.lnk files.
I've been told that all files in /opt need to be owned by root and the group needs to be root as well. I can neither confirm nor deny this. I'm just passing it on, but it would be nice to know which way it is and then I'll go back and edit this... ---------------------------------------------------- Creating menu entries isn't hard, but I've found it's easier to do after you've already made a *.dsl or *.tar.gz extension. What you need to do is create a file in a tmp/mydsl.menu directory that has the same name as you package has, capitalization and spelling count. So if my package was called mypackage.dsl, my menu would need to be called mypackage. The menu file is just a text file so create and edit it using whatever you want: beaver, vi, nano, whatever. The entries are fairly straightforward. For a single menu entry the form is: [exec] (The string the menu displays) { command the menu runs } to create submenus the form is: [submenu] (Submenu name) [exec] (item1 string) { command for item 1 } [exec] (item1 string) { command for item 1 } [end] Or go crazy and have submenus in submenus if you need to: [submenu] (Submain0 menu name) [submenu] (Submain1 menu name) [submenu] (Submain2 menu name) [exec] (item1 string) { command for item 1 } [end] [end] [end] a common way to run cli programs is in an xterminal. To bring one up and run your application in it the form is: [exec](my menu name){aterm +tr -T "Window Title" -e /opt/mypackage/bin/myprogram} So to add a menu to your new package, you world create 2 directories, create the menu item, run chmod 644 on the new menu item (it will not work if you leave out that chmod step), and add it to your extension. The only hard part is that you can't add files to compressed tars, so you'll need to use gzip to uncompress and recompress the tar. To do this the file name must end in tar.gz, so you'll have to rename your dsl package (not needed for tar.gz packages). Like so: mkdir tmp mkdir tmp/mydsl.menu beaver tmp/mydsl.menu/mypackage # add your menu entries to your menu file chmod 644 tmp/mydsl.menu/mypackage mv mypackage.dsl mypackage.tar.gz # if it's a dsl package gunzip mypackage.tar.gz tar -rzvf mypackage.tar --numeric-owner --no-recursion tmp/mydsl.menu/mypackage gzip mypackage.tar Some people, myself included, find it easier to unpack their extension to a directory, create the tmp/mydsl.menu/mypackage menu item there, add that file to the files list, and then repack. It takes more room but it's easier: mkdir work cp mypackage.dsl work cd work sudo tar -xzvf mypackage.dsl --same-owner mkdir tmp mkdir tmp/mydsl.menu beaver tmp/mydsl.menu/mypackage # add the menu items, save and close editor chmod 644 tmp/mydsl.menu/mypackage tar -tzf mypackage.dsl > files # add tmp/mydsl.menu/mypackage to the files list using beaver or whatever tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files ---------------------------------------------------- Making icons requires you to create a mypackage.lnk file and an image file for the icon in a home/dsl/.xtdesktop directory. The icon and image name must be the same as the package name. the format for the lnk file is:
MenuCommand1: is kind of neat. You add items here and if the user right clicks on your icon, a menu pops up and they can choose to run this command. Separate your text from the command with a ':' See the /home/dsl/.xtdesktop.Beaver.lnk for an example. Right click on the Beaver icon and you get the option of starting beaver as root. The icons are normally 48x48 pixels but they can be bigger or smaller, and can be one of the following types: gif, png, jpg, ppm, or xpm. You can create the icon in XPaint if you want to, but if you want an alpha channel (the see through parts of the icon) you'll need a stronger program. Following a similar procedure to the above menu steps, you could do: mkdir work cp mypackage.dsl work cd work sudo tar -xzvf mypackage.dsl --same-owner mkdir home mkdir home/dsl mkdir home/dsl/.xtdesktop beaver home/dsl/.xtdesktop/mypackage.lnk # edit with beaver or whatever to change names and positions # create or copy an icon into home/dsl.xtdestop # change read-write-execute permissions to -rw-r--r-- chmod 644 home/dsl/.xtdesktop/mypackage.* tar -tzf mypackage.dsl > files # add /home/dsl/.xtdesktop/mypackage.lnk and mypackage.png to the files list tar -czvf mypackage --numeric-owner --no-recursion -T files ---------------------------------------------------- Finding the files for the file list, as I said in the beginning, is the hardest part. A lot of the time I'll use the find command to make it easy on myself. I'll load the gnu-utils.dsl package first so I can use the better find command than the one that comes with the busybox stuff in dsl. Since the normal order of operations in compiling is: ./configure make sudo make install I'll create a temp file just before the make install step and the search for files created after the install step, which tells me what was installed. ./configure make touch mymarker sudo make install sudo find / -not -type 'd' -cnewer mymarker | grep -v "\/proc\/" | tee files Remove the leading /ramdisk from each line of the file (use beaver's replace or whatever). The first time you create the archive, you'll need the leading / to tell tar to get the files from the root directory. Then use the tar -czvf mypackage.dsl --numeric-owner --no-recursion -T files command to create a dsl package like you did above then untar the package to a work directory using tar -xzvf mypackage.dsl --same-owner -C work You can then use the tar -tzf mypackage.dsl option to get the list of files back. You don't want a forward slash "/" at the start of the lines in the files list once you start tarring and untarring from the work directory. You also want to be careful that you dont include any directories that already exist. While it doesn't delete the existing on, it can change the ownership/permissions and that can foul things up. So look at the files and check for things like bin/ or etc/ all by themselves with nothing after them. You need to be careful to preserve the owners and groups of the files in your archives. If you ever unpack and then repack your extension, say when you're adding menus and icons, make sure to use the --same-owner switch in the tar command to preserve owners and groups. I have read < elsewhere in a useful post by ke4nt1 > that the owner.group for files in your extension directories should be: /etc = root.root /usr = root.root /home/* = dsl.staff /tmp/* = dsl.staff /opt = root.root (usually) Another great piece of advice from that post was to look at the files in your extension using tar -tzf mypackage.dsl > files and comparing them to files already included in dsl. Are you overwriting any existing files? Do you really need to? ---------------------------------------------------- If programs won't run because they're missing libraries, you can use the ldd command to list what libraries the program wants to load and then add those libraries to your package. You need to be careful because sometime ldd gives you a library that is really a link to another library. You end up adding the library but you only added a link that points to something you didn't put into the package. You need both the links and the real libraries in your package. deb2dsl is a great script, but it includes every library that the apt-get said you needed. Sometimes you only need parts of what you actually downloaded. ldd can tell you that if you look at all of the executables in your package. ---------------------------------------------------- I would never put the following files in extensions. They are always getting caught in the find command but I know they will overwrite other existing files and will be bad. /etc/mtab /etc/fstab /etc/ld.so.cache (you can save /etc/ld.so.conf but make sure your app runs sudo ldconfig) /lib/modules/2.4.26/modules.dep /lib/modules/2.4.26/modules.generic_string /lib/modules/2.4.26/modules.ieee1394map /lib/modules/2.4.26/modules.isapnpmap /lib/modules/2.4.26/modules.parportmap /lib/modules/2.4.26/modules.pcimap /lib/modules/2.4.26/modules.pnpbiosmap /lib/modules/2.4.26/modules.usbmap (your program can run depmod and these will be set correctly if it's needed) ---------------------------------------------------- The last piece of advice I can offer is in the creation of tar.gz and uci packages. It is sometimes possible to compile your program like this: ./configure --prefix=/opt/mypackage make make install This makes all of the executable and library files created by the program go into the /opt/mypackage directory and the executables will know to look there for what they need. ---------------------------------------------------- EDIT: hit return and posted before I was done. Added that /etc can be in tar.gz and uci extensions per Mikshaw's post. Added the part about removing /ramdisk from the start of lines in the file list Added info about icon sizes Added chmod 644 for icons, owner.group info, and warning about overwriting existing files Posted by clacker on May 05 2006,21:05
Yeh, like that wasn't enough. Here is a revised script to go from a *.tar.gz to a *.uci file, hopefully with some useful error messages:
EDIT: added quotes around echo strings per Mikshaw's suggestion added the ability to use the /etc directory, again thanks Mikshaw Posted by mikshaw on May 05 2006,21:43
a couple of minor things about the script....1) Breaking up an echo command for a multi-line echo into more than one command makes reading the script a little confusing (only a little). I suggest one of these options... Use quotes for each echo line, to make what is happening more apparent, visually.
Or use a single echo that spans multiple lines. This one requires quotes, but should pass the linebreaks without trouble
2) What about /etc? A *.tar.gz extension can install files to /etc without the need to run mkwriteable, even though space is apparently limited. Posted by clacker on May 05 2006,22:08
Thanks Mikshaw. I had no idea tar.gz and uci files could contain files in the /etc directory, but I just tried it out and you're right. I thought it might have been because *.tar.gz extensions were supposed to be safer that the regular dsl ones, but apparently there isn't a check for that. I'll modify my posts above to reflect the ability to include those directories in tar.gz and uci extensions. Any other advice on the post or the script is welcome. I did my best but I'm sure there are omissions and errors.
Posted by spock on May 05 2006,22:26
Thanks to unionfs I found it very easy to get acroread 7 in DSL :- in a chroot apt-get acroread (after pointing to the Marillat repositories in /etc/apt/sources.list) - deb2dsl - unpack the dsl extension in /opt/acroread - mkisofs -R -hide-rr acroread | create_compressed_fs - 65536 > /cdrom/optional/acroread.uci This extension is then loaded in the sytem with the unionctl command. I tried it with a fat PDF 1.6 document I have never seen it so fast ! I think this should be a main option for creating extensions in the future because it's so easy and quick, and it's very ram friendly. Posted by whiteweasel on May 09 2006,03:57
The building of .uci extensions is clear to me now. With 2.4 supporting 64 cloops I suppose these will become more popular. My first test will probably be to create a flock .uci. Thanks bunches for the link! The docs are spread out, as you know, and probably could be better organized. Posted by whiteweasel on May 09 2006,04:04
clacker, thanks. I've got it (I think!) and can go from here. Your explanation (which is truly excellent, BTW) should really be copied into the wiki.
Posted by mikshaw on May 09 2006,05:08
I'd say tar.gz and uci are "safer" than dsl, simply because they don't access most of the base system. However, these safer packages can still easily cause the complete destruction of your OS if created by a malicious fiend and not checked by the user. For example, bootlocal.sh could be replaced by something that causes lots of damage next time you boot. Or more immediately, .bashrc could be replaced by a different script that would run the next time you opened bash. Poof, there goes all your data. |