Roll your Own Extensions

Forum: Other Help Topics
Topic: Roll your Own Extensions
started by: Fordi

Posted by Fordi on Sep. 13 2004,23:44

Current DSL extensions come in three flavors:

"Safe" .tar.gz files; they are just tar.gzs that are relative to root, and write to home and opt only. "tar -xvzf file.tar.gz -C /" will extract 'em for you.

"potentially unsafe" .dsl files; also .tar.gz files, but they're given the .dsl extension so that DSL knows it's going to try to write to the base CD system - it needs to prepare for that. I won't tell you how to extract these - find out on your own, or let DSL do it, you can damage your system here.

"Unified Compresed Image", or .uci files. These are large, static extensions. They're referred to as "unified" because of the old way of doing things. Previously (in versions 7.x), the OS would mount a .ci image and extract its .tar.gz separately. The new .uci format is essentially the same, except the .tar.gz is found inside the compressed image under the name /user.tar.gz
These don't really get "extracted", but mounted. This saves a lot on your RAM.

Code Sample

losetup /dev/cloop[1-8] file.uci
mkdir /opt/file
mount /dev/cloop[1-8] /opt/file
cd /opt/file
tar -zvxf user.tar.gz -C /

* Notes:
cloop1 - cloop8 are available by default in DSL 8.0. Whichever you pick, pick for both. Also, I would suggest picking high when you're doing test-mounts; I don't know how DSL does accounting on uci's
file is the name of the .uci extension in question, without the .uci at the end. for example, openoffice.uci mounts itself in /opt/openoffice

How to make your own?

Well, say you want to package Apache2 for your own use later. This is a large program, and you may not have a lot of ram available, so we're going to go for the .uci option (essentially 'cos the other forms of extension creation are pieces of piss, once you've got all the files and libs not naitively supported by DSL in one place)

Download apache2 from one of their mirrors.

extract it under your home directory (if you're in DSL, it's /home/dsl)

Code Sample

tar -xzvf httpd-2.0.50.tar.gz

enter that folder and check out your configuration options

Code Sample

cd httpd-2.0.50/
./configure --help

you should see a list of options, one of which is "--prefix". use that to your advantage and proceed with normal build

Code Sample

./configure --prefix=/opt/apache2 ...[other options]...
make
make install

Note: no header snafu. Sometimes, you'll have a library, but not the header to go with it. This is especially pertinant in DSL, as much has been removed to make room for functionality. If compilation stalls out on a function or other, go apt-get install or upgrade whatever libs it spits at you.

As for the other options portion - you may not want to compile everything in; this method basically compiles apache as a monolith (2.2M with no configuration). I normally just compile in the stripped-down basic modules that I have in my Win32 version (that's what I used first) and live with a statically linked apache.

Additionally, if you're going to distribute this extension to anybody, you have to select your architechture as i386 using environment variables. See the man pages for gcc, page 445.

***eats chips and waits for compile***

Ok, now you've got the whole server in /opt/apache2

First: we have a conf file where our uci's going to be mounted, meaning it'll be non-writable. time to change that.

Code Sample

mv /opt/apache2/conf/httpd.conf /home/dsl/.httpd
scite /opt/apache2/conf/httpd.conf

Now, enter this line into httpd.conf:

Code Sample

Include "/home/dsl/.httpd"

Now wipe out the original htdocs folder

Code Sample

/bin/rm -r /opt/apache2/htdocs

And replace all instances of "/opt/apache2/htdocs" within httpd.conf with "/home/dsl/htdocs" or wherever else you'd like it to come from (my personal favorite is to have it linked there, but have that as a symlink to my USB pen drive)

Finally, the fun part, making an image. I'm not going to do this the classical mkisofs way, as I like to have a bit of control later on for minor tweaks (iso is not writable, but making an ext2 image does have two disadvantages: more commands, and slightly lesser compression)

Code Sample

root@box:~# du -sh /opt/apache2
26M /opt/apache2
root@box:~# dd if=/dev/zero of=apache2.img bs=1M count=26
26+0 records in
26+0 records out
27262976 bytes transferred in 0.244728 seconds (111401094 bytes/sec)
root@box:~# mke2fs apache2.img
apache2.img is not a block special device.
Proceed anyway? (y,n) Y
...
This filesystem will blah blah every 29 mounts or blah blah blah
root@box:~# mkdir apache2
root@box:~# mount apache2.img apache2 -o loop
root@box:~# cd apache2
root@box:~/apache2# cp -a /opt/apache2/* .
root@box:~/apache2# tar -cvzf user.tar.gz -C / /home/dsl/.httpd
root@box:~# cd ..
root@box:~# umount apache2
root@box:~# create_compressed_fs -b apache2.img 65536 > apache2.uci
.......
[10] Block# ........ size 65536 -> ....... [compression ratio ...%, overall ...%]
..........

You now have what (should be) a working apache uci. Keep in mind we didn't go over configuring apache further (way out of the scope of this document), so you'll probably have to make changes to .httpd once you install everything.

Posted by guest on Sep. 14 2004,07:18

Quote

I don't know how DSL does accounting on uci's

look at /usr/sbin/mountci

Posted by guest on Sep. 14 2004,12:16

if i try your method:
create_compressed_fs -b remaster.img 65536 > remaster.ci

then when i do a myDSL-load i get wrong:
fs type, bad option ,..
and .ci doesnt load

but when i do:
mount remaster.img /mnt/test -o loop
mkisofs -R /mnt/test | create_compressed_fs -b - 65536 > remaster.ci

all goes well

i think the newer cloop driver check for iso?

Posted by Fordi on Sep. 14 2004,15:44

Quote (Guest @ Sep. 14 2004,08:16)

Yeah. I looked at the mountci script. It specifies -t iso9660 (unneeded, really; linux will automatically figure out iso9660 and ext2, as well as a couple other fs types). In other words, I am mistaken. uci files must be iso's passed through create_compressed_fs.

That's not to say compressed ext2 filesystems don't work, just that dsl won't handle them proper.

Funny thing is, it's a matter of removing 11 bytes from a single shell script (a modified mountci without the -t argument appears to work fine for compressed ext2)

Posted by mikshaw on Jan. 03 2005,15:51

I suppose it's time I learned more about uci, with the Xfree86 and nvidia extensions providing a way to add some heavier applications such as 3D games.

The explanation given by Fordi looks thorough, but the thread in general just leaves me confused.

1) How is /home/dsl/.httpd installed by the end-user? Manually?
2) What is create_compressed_fs? Is this a DSL- or cloop-specific application, and how can it be installed/used on another distro?
3) So what is the proper method of creating a UCI? It seems like there is some contradiction here.

Thanks for your time.

Posted by Pellaon on Feb. 07 2005,17:32

I can't compile apache 2. I have downloaded the gcc dsl. It is correctly detected but i get errers and a shutdown while building...

If someone could compile apache 2 for the DSL it would be great.

Posted by pellaon on Feb. 23 2005,13:44

Ok, i'm successfully made an Apache Dsl with PHP5... Still having problems with Mysql...