March extensions
Forum: The Testing Area
Topic: March extensions
started by: roberts
Posted by roberts on Mar. 06 2008,20:49
Thanks to Jason for an update to:xine.uci.
Now posted.
Posted by roberts on Mar. 10 2008,23:17
Thanks goes to Paul for:owfs.uci
to newby for:
bdfedit1.3.tar.gz
and to Juanito for:
xorg72-dev.tar.gz
Posted by roberts on Mar. 22 2008,04:22
Now posted!Thanks to Paul for an update to:
owfs-2.7p4.uci
Thanks to Uncle for:
openvpn.dsl
Thanks to Bram for:
wireshark.uci
Thanks to newby for:
jhfCAPTCHA.310107-1.tar.gz
Please read the info file for more information.
Posted by roberts on Mar. 25 2008,18:30
Thanks to Juanito for an update to:hplip.uci
Much smaller - read the info file for usage.
Posted by stupid_idiot on Mar. 26 2008,13:41
Hi all:Have been very busy these few weeks on various personal things.
I am trying to finish the DSL chroot (both the archive file and the wiki document) within the next week.
Posted by ^thehatsrule^ on Mar. 26 2008,18:12
All of extensions from "newby" are invalid and needs to be pulled. I'd suggest that further extensions from this user do not be posted for now - evidently, this user did not heed to any advice given in the user's (older) forum thread. See jhfCAPTCHA.310107-1.tar.gz and bdfedit1.3.tar.gz.This brings me to ask (roberts): are there any checks automatically done on new submitted extensions (such as the declobber script)? If so, perhaps an addition that can check for directory structures (such as being in /opt for targz's), etc. could be used as well. Or maybe something a submitter has to run and verify before sending it in?
I know that myDSL is a community process that can take a lot of time, but these files can be considered harmful to users' systems. And even though they are in testing, these kind of mistakes should always be avoided so that they don't get online. Side note: the only reason why I looked into these extensions was because I remembered the other thread.
Posted by roberts on Mar. 26 2008,21:21
Extensions are a community process and as such are to be self policed. I am only the conduit to get the extensions posted.I don't have time and/or equipment necessary to test and/or moderate submitted extensions. I have asked John to assist in this task but to no avail.
When Kent was active in DSL, he would thoroughly review each extension. We currently don't have such luxury.
Currently there are so many extensions in testing that need to be categorized or
Open to suggestions for handling submitted extensions. Should there be some automated test procedure?
^thehatsrule^, thanks for bringing to my attention.
Posted by lucky13 on Mar. 26 2008,23:56
With those I checked from newby, it was easy enough to just test the archive and see that they weren't self-contained (iirc, they installed across /). I think this particular issue is isolated. If it were more prevalent and others were submitting things that were similarly misconfigured for MyDSL, then I could see doing more before posting to testing. The policing/testing from the community should be adequate if there are other issues that crop up like bugs, bad behavior, etc.Maybe one way around it is to require additional steps for initial submissions from new contributors to make sure they have the concepts straight before posting to testing. Or have initial submitters post to their own or third-party sites and then get community approval for adding to a contributors list. There has to be some way that won't require Robert to micromanage this.
Posted by WDef on Mar. 27 2008,12:04
| Quote |
| (such as the declobber script)? If so, perhaps an addition that can check for directory structures (such as being in /opt for targz's) |
It would be easy enough for me to add tests for some more basic mistakes to extend the declobber script, such as location of installed files not matching extension type. Eg also maybe to find and/or fix things like "Status: anchor" lines in xtdesktop icon .lnk files, and maybe non-dsl4+ compatible menus etc. I need to revisit that script anyway and add getopts as Robert suggested. Anyone have other bug hunting suggestions I could try to implement?
I see where you are coming from kicking ideas around like prior community approval for extensions posted externally but I have a few doubts about that one. Last time I looked, all puppy packages were on disparate sites and the community was supposed to provide feedback in their forum, but it just doesn't work very well. They end up with multiple versions of the same software in various places, some hard to find, some in various states of repair, and some plain broken. However they have no central repo at all, which is a key difference with what you're suggesting.
Posted by mikshaw on Mar. 27 2008,16:04
I assumed that's what the "testing" category was for. All new extensions go there.I would hope they stay there forever until there is positive feedback concerning their useability (rather than just being integrated after not receiving any negative feedback), or get removed when found to be faulty or replaced with a better version.
Posted by ^thehatsrule^ on Mar. 27 2008,16:41
There has been a few external personal repos used already, and they were fine due to the fact the submitters posted links here on the forum and were eventually posted on the official site. My guess is that this was also used for convenience, as uploading attachments on email can have problems.It sounds like having prior community approval might be complicated... and time consuming, but I can see how it could work.
On the subject of automatic testing, I was thinking of a quick 'surface' scan like script, similar to what WDef has in mind. I guess those extensions could always wait in testing/ but certain things would be found much faster if something could always check for those things. I was wondering if something could be done on the repos' server, but I suppose that probably wouldn't be allowed.
As for thorough inspections, they can be left to the community. I may have time for that sometime soon, but I still need to see how extensions work for 4.x :P
EDIT: removed comment about mydsl.es.org due to misunderstanding
Posted by roberts on Mar. 28 2008,23:42
I think WDef's suggestion for enhanced declobbler script as an initial test of appropriateness is a very good idea and would be most welcomed. I would post it so that extension builders could download and self-test as well as I could run before posting.
Posted by newby on Mar. 29 2008,19:12
| Quote (roberts @ Mar. 28 2008,18:42) |
| I think WDef's suggestion for enhanced declobbler script as an initial test of appropriateness is a very good idea and would be most welcomed. I would post it so that extension builders could download and self-test as well as I could run before posting. |
Yes, that's what I've been wishing I had. (Apologies for any problems.)
I'm not up to the level of the ten other guys who are active on this board, but I'm trying to climb the learning curve.
I found the line of bash code and see what people mean.
Do I understand correctly that packages can be forced into /opt via a script?
If so, can anyone point me toward an easy example so I can fix the problems.
Posted by curaga on Mar. 29 2008,20:18
./configure --prefix=/opt/packageworks every time..
Posted by Jason W on Mar. 29 2008,20:50
Newby-Do you have a copy of the DSL book? It covers all the details about building the different extension types. It is a good idea for anyone to get it and read it before submitting any extensions, especially .tar.gz and .uci ones. Also, become familiar with building from source. Reading through Linux From Scratch would help a lot. I bought that book too and read it on vacation when I needed a Linux fix.
This is not the right section, and this is also covered in other places, so I will be brief. When I install into /opt, "./configure --prefix=/opt/package && make && make install" is the first step. I copy the resulting /opt/package/lib/pkgconfig/package.pc file (if there is one) to /usr/lib/pkgconfig and 'export PKG_CONFIG_PATH=/usr/lib/pkgconfig' if I have multiple libraries or apps that will depend on each other in the extension. Without pkgconfig files, ./configure options may be used to tell an app where the libraries are. There are other steps like exporting library paths and such that can be needed, but this is a basic starting point. The LFS process involves installing things in non-standard places and is a good resource on compiling and package building.
Practice building and installing stuff and getting familiar with the details of making an extension of what you built. Build a few extensions, use them a while, and when you know you have a keeper submit it.
EDIT: I see Curaga beat me to it with /opt/package while I was writing.
Posted by lucky13 on Mar. 29 2008,21:53
| Quote |
| If so, can anyone point me toward an easy example so I can fix the problems. |
Read the steps on the wiki. There's a page for each kind of MyDSL extension.
< http://damnsmalllinux.org/wiki/index.php/Creating_MyDSL_Extensions >
I again want to pick the nit I started about the php script. That's not the kind of thing that even requires compilation (move it to /opt/phpcaptchascript/nameofphpcaptchascript), and is something anyone using php can already find and stick in ~/ or wherever it's needed or desired for testing or use. Is making it into a MyDSL tar.gz really necessary or even desirable?
Posted by newby on Mar. 30 2008,03:59
| Quote (lucky13 @ Mar. 29 2008,16:53) |
| Read the steps on the wiki. There's a page for each kind of MyDSL extension. < http://damnsmalllinux.org/wiki/index.php/Creating_MyDSL_Extensions > |
Been there, done that. That's where I found:
| Quote |
| There is also the fact that a lot of people don't know how or just don't want to have to compile source,or maybe they don't have the tools to do so. |
Since I am one of those to which the above applies, I was trying to find some simple progs that the following would apply to:
| Quote |
| Building a green extension can sometimes be as simple as packing up a pre-compiled binary with the appropriate ownership. |
Apparantly, I made a mistake. OK, get over it. I seem to recall a rather famous person who suggested, "Let he who is without sin cast the first stone."
Nit picking is never helpful, nor is name calling, uttering slurs et cetera and so on. Posting tools that make the task more accessible to everyone _is_ helpful. Answering the question asked is also helpful.
So, once again, I recall reading about forcing a program to run in /opt when it wants to run somewhere else. I think it involved writing a "wrapper script" to package up with the application. If I had an example of one, I'd probably be able to figure out how to modify it for any app I upload.
===> Can anyone point me to an example? <===
Posted by Juanito on Mar. 30 2008,05:35
This is kind of off-topic for the extensions thread, but you could perhaps try this as an example of how to build and test an extension:1. Create your workspace:
| Code Sample |
| $ sudo mkdir /usr/src $ sudo chown dsl /usr/src $ chgrp staff /usr/src |
2. Download a source tarball and unpack it to /usr/src - lets say grub-0.97.tar.gz for the sake of an example.
3. Prepare things to compile:
| Code Sample |
| $ mydsl-load /path-to-file/compile-3.3.5.uci $ export CPPFLAGS=-I/opt/compile-3.3.5/include |
4. Compile:
| Code Sample |
| $ cd /usr/src/grub-0.97 $ ./configure --prefix=/opt/grub-0.97 $ make [open a terminal window as root] # cd /usr/src/grub-0.97 # make install |
5. Navigate to /opt/grub-0.97 and delete stuff that is not needed - info files, man files, doc files, etc. Then strip the binaries, libs, etc:
| Code Sample |
| $ sudo strip --strip-unneeded /opt/grub-0.97/bin/* $ sudo strip --strip-unneeded /opt/grub-0.97/sbin/* $ sudo strip -g /opt/grub-0.97/lib/* [not required in this example] |
6. Create links for the binaries to /opt/bin and save them for the extension to use:
| Code Sample |
| $ ls /opt/grub-0.97/bin/* > /path-to-file/files_user $ ls /opt/grub-0.97/sbin/* >> /path-to-file/files_user $ beaver /path-to-file/files_user [edit to read /opt/bin rather than /opt/grub-0.97/bin, etc and save] $ sudo ln -s /opt/grub-0.97/bin/* /opt/bin $ sudo ln -s /opt/grub-0.97/sbin/* /opt/bin $ sudo tar -T /path-to-file/files_user --no-recursion -zcvf /opt/grub-0.97/user.tar.gz |
7. Make the extension:
| Code Sample |
| $ cd /opt $ sudo mkisofs -R -hide-rr-moved grub-0.97/ | create_compressed_fs - 65536 > /tmp/grub-0.97.uci |
8. Tidy up and load the extension:
| Code Sample |
| $ sudo rm -r /opt/grub-0.97 $ mydsl-load /tmp/grub-0.97.uci |
9. Test the extension by making a boot floppy:
| Code Sample |
| $ fdformat /dev/fd0H1440 $ mke2fs /dev/fd0 $ sudo mount -t ext2 /dev/fd0 /mnt/floppy $ grub-install --root-directory=/mnt/floppy fd0 $ sudo umount /mnt/floppy |
Apologies for sticking this in this thread, hope it helps.
Posted by mikshaw on Mar. 30 2008,10:56
| Quote |
| Apologies for sticking this in this thread, hope it helps. |
| Quote | ||
I was trying to find some simple progs that the following would apply to:
|
| Quote |
| I recall reading about forcing a program to run in /opt when it wants to run somewhere else. I think it involved writing a "wrapper script" to package up with the application. If I had an example of one, I'd probably be able to figure out how to modify it for any app I upload. |
< http://linux.die.net/man/7/hier >
< http://www.pathname.com/fhs/ >
< http://www.tldp.org/LDP/Bash-Beginners-Guide/html/index.html >
< http://www.tldp.org/LDP/abs/html/index.html >
Posted by lucky13 on Mar. 30 2008,12:21
| Quote |
| , I was trying to find some simple progs that the following would apply to: Quote Building a green extension can sometimes be as simple as packing up a pre-compiled binary with the appropriate ownership. |
My nitpicking point is, your captcha script wasn't a pre-compiled binary. It's a php script. Scripts are much more portable than binaries -- you call scripts with a binary or another script from wherever you want or need it. Binaries generally aren't portable because of the various and sundry linking between them and libraries, etc.
| Quote |
| Nit picking is never helpful, nor is name calling, uttering slurs et cetera and so on. |
I was trying to be constructive with my nitpicking -- such as asking if scripts are really appropriate for MyDSL since users probably don't want to search through /opt for every possible thing that could be submitted as a tar.gz. BTW, who called anyone names or uttered slurs?
You don't need a wrapper script to kludge a php script into /opt. All you'd need to do for that is put the script where you want it in /opt (e.g., /opt/phpcaptcha) and then make a tarball of it. When you enter the command, you'll get output like the lines beneath it in this example (first I made the directory and for this I just touched "something.php" in it):
| Code Sample |
| % tar cvfz ./captchascript.tar.gz /opt/captchascript/ tar: Removing leading `/' from member names /opt/captchascript/ /opt/captchascript/something.php |
Voila, I have a file called captchascript.tar.gz in my pwd (present working directory) that conforms to MyDSL, but could include a menu entry (another reason I dislike the idea of scripts like this is invoking it from a menu -- why?). I could mydsl-load that and it would install it in /opt just as it was. That's all there is to it -- no wrappers, just a tarball with a file in a directory in /opt.
The standard examples of binaries that work well for relocation to /opt are Open Office and mozilla products like Firefox, Seamonkey, etc. These are most often distributed as binary packages that install everything they need in one portable, nested directory (e.g., /usr/local/openoffice-1.2.5 is the default for that particular version but a change of its install script can put it anywhere else including /opt). If it's all already contained in one nested directory like that, it's probably a very good candidate for tar.gz/uci extension.
Many other binaries won't work for tar.gz/uci without being recompiled because they're internally linked with paths across standard directories (e.g,, a DEB for xmms will install xmms in /usr/bin and its libs in /usr/lib/xmms and not /opt/xmms/lib/xmms, so that binary requires its libs to be in a specific place which makes relocation ridiculous). If it's not all nested in the same directory, you'll need to compile with prefix=/opt/application-name. Also, some things that you try to compile as tar.gz/uci won't like being in /opt for some reason. That's when you'd try a wrapper to kludge it together.
If you want to try making some tar.gzs and ucis, Mozilla just made some major security updates to their products this week. I was going to submit gtk1 versions later today (after my Longhorns beat Memphis) because it's been a while since the versions in MyDSL have been updated. I'll let you do it if you want -- these are pretty easy because all their stuff goes into one nested directory. It's not very difficult at all. You can untar in /opt and they'll be set up right there -- no need to relocate them, pretty straightforward. If you do make any changes, it might be to rename (mv) from firefox/seamonkey to firefox-2.0.0.13/seamonkey-1.1.9 (respectively) in case users want to use multiple versions (if UCI, though, users can unmount one version and then load another and use the same mountpoint). Use the steps 6+ Juanito posted to add a user.tar.gz for links and menu entries and then make UCI extensions for each. Let me know if you're going to mess with these.
The "Bon Echo" gtk1 version is here:
< http://www.lamarelle.org/firefox....tar.bz2 >
And the update for Seamonkey gtk1 version is here:
< http://releases.mozilla.org/pub.....tar.gz >
I don't think anyone here is out to give you a hard time. We just want to insure that MyDSL extensions work and don't cause any trouble. I think we all appreciate your desire to contribute to MyDSL, which is why (speaking for myself) I tried to help with the first round you submitted.
EDIT: Here are the relevant pages for the links above. I haven't read the build info for each so I don't know if there are any issues for DSL (I've been using both in Vector without issue since they were released this past week but that's with all up-to-date libs):
< http://www.seamonkey-project.org/releases/#1.1.9 >
< http://www.lamarelle.org/mo-zi-lla/mozilla.php#fx >
Posted by roberts on Mar. 30 2008,14:47
Thanks to mikshaw for:rlwrap.uci
and an update to:
lighttpd.uci
Read the info files. rlwrap a readline wrapper looks very useful for command line users, e.g., murgaLua, sqlite, tcl.
[EDIT] Have been playing with rlwrap and sqlite - now sql is very nice at the command line.
Posted by WDef on April 01 2008,10:16
| Quote |
| I think WDef's suggestion for enhanced declobbler script as an initial test of appropriateness is a very good idea and would be most welcomed. |
OK Robert I'll get on it shortly.
Posted by andrewb on April 01 2008,23:44
Seeing as this thread has become a general discussion on problems / scripts for loading extensions.....Perhaps this is already done, but would it be possible to check whether a system is running with unionfs before executing the mkwritable script when a .dsl extension is selected for loading. So far as I understand having unionfs running would mean that mkwritable would not be required? This would make the unc & dsl extensions operate in the same manner when unionfs was running & (probably) remove the need for the unc extension type (could unc extensions not be made to load on legacy, non-unionfs, systems if the mkwritable script was executed?)
Posted by roberts on April 02 2008,00:35
Not sure I am following what was stated but:Unionfs is only available upon boot time.
Once unionfs is in place, mkwriteable does an immediate exit and never runs even when called via a .dsl load..
Unionfs and mkwriteable are not equals, to do so would generate too many symlinks and therefore inode usage that many computers would report "out of memory". Unionfs is an overlay so much less inode usage. Before there was unionfs several attempts where made to increase the directories in mkwriteable only to find many machines exhausted their inodes.
Posted by andrewb on April 02 2008,11:21
OK,that clears up what I suspected that loading a dsl extension on a sustem with unionfs doesn't result in mkwritable being executed. Very useful for low memory systems.
Posted by stupid_idiot on April 04 2008,12:33
Hi all:The "build a DSL compilation environment from scratch" guide is going smoothly and should be ready in a few days - if nothing goes wrong!
Posted by WDef on April 04 2008,21:21
Ping: RobertI've emailed testdisk.uci to you twice but it hasn't appeared as yet - perhaps search your email for it?
Posted by roberts on April 04 2008,22:42
| Quote (WDef @ April 04 2008,13:21) |
| Ping: Robert I've emailed testdisk.uci to you twice but it hasn't appeared as yet - perhaps search your email for it? |
Nothing in extensions mailbox. I sent a test message to be sure it is working and it is.
Posted by mikshaw on April 05 2008,02:31
might be useful to include "mydsl" in the subject if you haven't. Robert mentioned that a while ago, and I've been doing it since, in order to prevent it being tagged as spam.
Posted by WDef on April 05 2008,07:24
I'll give that a try.Just sent to extensions@damnsmalllinux.org with subject MyDSL testdisk.uci
Posted by roberts on April 05 2008,18:13
Nothing received and I even checked the spam folder.I received several from Juanito and will now post them.
Posted by WDef on April 05 2008,20:59
That's odd. Something with yahoo and attachments perhaps? There's nothing bouncing back, sent three times, and just forwarded again for good measure. Other emails from the same address are being received, and it's the same account I've sent other extensions to you on.
Posted by roberts on April 06 2008,02:01
Perhaps a change in Yahoo mail? Other users have complained about Yahoo mail. Try packaging all three files into a .tar.bz2 and see if it gets to me.