How can we improve our .dsl extension files?
Forum: myDSL Extensions (deprecated)
Topic: How can we improve our .dsl extension files?
started by: ke4nt1
Posted by ke4nt1 on June 10 2004,03:13
I have been paying close attention to many threads and posts about ".dsl" extensions, ".ci" files, and ".tar.gz" files for DSL ...I have looked inside most all of the ".dsl" and ".tar.gz" files submitted so far ...
My hope is that this thread will answer several questions, and offer some further input from users and creators alike about
what would be the "proper" or "most efficient" way to make ".dsl" and ".tar.gz" files for all systems and users of DSL ...
I personally have had help and assistance from everyone that I have requested help from, and many thanks to all.
During DSL version 0.7.0 release was my first attempt at writing any type of file for a linux system, ever...
and I didn't have any idea about what I was doing, short of emulating someone else's method or style ...
At this point, while I'm familiar with many of the terms and methods of using extensions to add to the DSL base,
I'm still unsure of WHY I should do, or not do, certain things, when I am working on a new addition to DSL.
I will list several of them here, and await suggestions from others..
1. When should I choose to use a .tar.gz, instead of a .dsl ?
2. What advantage does a .tar.gz offer over a .dsl, and WHY?
3. Should we always try to remove all unnecessary files from
our .dsl and .tar.gz builds, like doc, AUTHORS, man files,
README's, to keep the .dsl size to a minimum?
(helps save RAM ?) or should SOME be left intact?
4. Does leaving out these files harm the AUTHOR of the
program, or break the GPL for distributing a program
without giving the true AUTHOR recognition for his work?
5. Are there any directories that a .dsl file should try to avoid
writing to, or directories that are preferred to be used?
6. Same question as above, but with .tar.gz files?
7. What about including Icons for the desktop in our .dsl's?
Is that preferred or just a bonus?
8. The ".ci" files are amazing at what they do, but what
determines if an extension build should ever be one?
( whether it is shared with DSL, or used at home only? )
Clear answers to these questions will surely help a number of
us who are aiming to contribute to DSL, and help encourage
others to participate with the incredible group of users here.
73
ke4nt
Posted by roberts on June 10 2004,04:52
I will try to answer your questions. Recall that myDSL was made to give liveCD users an easy way to add applications. It was designed to end the debate of how big should DSL be. If you want to 64MB system, a 210MB, or any size that you want. With myDSL you can have DSL "your way" Also, it is easy to grab a new version of DSL and your custom apps will carry forward. No need to re-install and re-setup icons, and menus. The 0.7. release was made for liveCD. Now 0.7.1 also allows hard drive installed users to have access to the extensions..
1. When should I choose to use a .tar.gz, instead of a .dsl ?
When the files can load into an existing un-modified liveCD environment, i.e., the default writeable areas of /opt /home/damnsmall Many program load into these areas. It has very little impact to the system. They have actually been around for a long time. It was made with the intension of having "shared" apps. This benefited hard drive. Instead of each user have a copy of firefox it was shared among the users in a multi-user system. Now .tar.gz is ideal for factoring your backup.tar.gz for a more personal myDSL system. See below for more explanation. This idea came to me by reading Rapidweather's posts of how he factored out his firefox from his backup.tar.gz so that his backup would fit on a floppy.
2. What advantage does a .tar.gz offer over a .dsl, and WHY?
The concept of a dsl file came to me when I was implementing cbagger01 synaptic script. Actually I had used his script in the past and saw the impact of using i-nodes on low ram machines. A 32 MB test machine would quit because of i-node usage. As I implemented cbagger's script I tried to get the most mileage out of it. I could see the furstration of trying to backup the deb files from apt cache and the fact that there is no icon nor menu item. So why not have these deb files or any collection of files be cleaned up some to be an integrated part of DSL. These files need the modified "writeable" directory structure in oder to work. Same as for Synaptic script.
3. Should we always try to remove all unnecessary files from our .dsl and .tar.gz builds, like doc, AUTHORS, man files, README's, to keep the .dsl size to a minimum? (helps save RAM ?) or should SOME be left intact?
As for the other files they are taking up ram space. Remember DSL supports many low ram machines. Also to be consistenent with the existing DSL files, most of the man pages, docs, and debian menu files have been deleted.
4. Does leaving out these files harm the AUTHOR of the program, or break the GPL for distributing a program without giving the true AUTHOR recognition for his work?
I am very big on giving credit to the author who creates software of any kind. If you see lic files then they should stay. Never remove author's name or comments even from scripts that you may have found useful. These butched scripts often show up on other fourms and could cause confusion.
5. Are there any directories that a .dsl file should try to avoid writing to, or directories that are preferred to be used?
Mostly these are Synaptic installed files. It is the prgram's author that is determing this. Using dpkg -L packname to start your collection of files to make your dsl.
6. Same question as above, but with .tar.gz files?
As stated above the tar.gz can really only be written to /opt and /home/damnsmall. Otherwise you are assuming somebody has already loaded a dsl file and has modified the file system. Their intended use is for factoring out parts of your current backup.tar.gz. For instance I have a myPrinter.tar.gz, myXsetup.tar.gz, myWireless.tar.gz. With creative use of these files you can make a totally hands free boot up cdrom such that everything to ready to use. Plus you backup.tar.gz is now only actual files and not system setting etc. Even low ram users can benefit from these types of files.
7. What about including Icons for the desktop in our .dsl's? Is that preferred or just a bonus?
Some program such as games might have too many icons to be practical. It is a matter of taste. Would you want you desktop to be covered with all the ace-of-penguins game icons?
8. The ".ci" files are amazing at what they do, but what determines if an extension build should ever be one? ( whether it is shared with DSL, or used at home only? )
These type of extensions are mounted at boot time. Therefore long ago DSL made the /opt directory writeable for shared apps in a multi-user hard drive installed environment. Now if the apps is self contained under a mount point then this type would be preferred as they have very little impact. I have run openoffice.ci on a 128MB machine. Many apps may be able to convert to this style. Many times it is more work to make shell wrappers and extend library path search. But the efforts certainly show results.
I purposely started with what I thought was the easiest type for a user to make. Recall the initial web page showed firefox.tar.gz as an example of a user created extension. It was just tarred up after the download script. Then because of cbaggers contribution of the synaptic script, it was a natural extension to have extensions be able to write across the filesystem as Synaptic does. I have been running .ci since April. I held back with their release so as not to confuse the effort.
I must say I am very happy with the users contributions. This is an exciting time for DSL users. Thanks to all who have contributed and I look forward to more to try. They are after all just a collection of files. All the code to make them work is part of the DSL system.
Posted by clivesay on June 12 2004,18:49
I think this has been touched on a little but, is there a possibility of having a mydsl script for the HD install enthusiast?Say you have a HD install of DSL and a cool new extension comes along. You download the extension to your root directory and run a script similar to the mydsl script at boot to install the extension. Then you could delete the .dsl file from the root dir.
Is this feasible? I am clueless when it comes to development so I am just throwing the idea out there.
Posted by ke4nt1 on June 12 2004,19:02
Starting with 0.7.1, you can type in a terminal, from user damnsmall..mydsl-load
type inthe location of your new extension in the box, for ex:
/mnt/hda4/storage/mplayer.dsl
and it will load your new extension, and make it available to run.
You can keep your extensions on another drive/partition/location, and
when 0.7.2 comes along, simply re-install to HD the new DSL, keeping
your backups, .dsl files, and so forth still available to use and restore from...
So Cool
73
ke4nt
Posted by clivesay on June 12 2004,19:33
Thank you. I guess I must have missed that info somewhere on the site (embarrassed).It is pretty cool!
Thanks
Chris
Posted by ke4nt1 on June 12 2004,20:40
Anyone playing with the extensions noticed this happening?When using the mydsl-menu feature, from LiveCD, things are perfect.
Apps load, icons are placed on desktop, choices are added to mydsl-menu.
If, after using a few extensions, browsing a couple of internet sites,
adding bookmarks, changing themes, etc... I decide to make a BACKUP,
then strange things happen the next time I restore from BACKUP...
The icons I previously loaded from mydsl extensions are on the desktop,
but useless, as well as the mydsl menu choices - useless.
When I reselect ( install) them from mydsl-menu, I now have 2 listings
in MyDSL... These were saved in the backup, as well as the things I wanted.
Of course, the only reason I did a backup in the first place was to
save bookmarks, high scores, cache, etc...
Maybe there is a way to "uninstall" the mydsl extensions before making
a backup?
I could choose to backup the files the extensions installed,
then the icons and menus would be useful..
but that would be HUGE !!
Or, make several different filetool.lst files, like filetool.firefox, filetool.desktop.
Then, make choices in the menu under file restoration for each...
Perhaps this is where cbagger01 and roberts idea of a Myfirefox.tar.gz,
MyDesktop.tar.gz, etc would work, rather than letting the BACKUP utility
handle all the chores...
I enjoy the mydsl and mydsl-menu options
I also enjoy the backup & restore features, especialy on bootup
They don't seem to get along with each other , unless I get creative
with editing the filetool.lst each time I backup
( for ex: removing /home/damnsmall from filetool.lst, and substituting only
parts of it, like /home/damnsmall/.phoenix /home/damnsmall/.xgalaga )
Any Thoughts???
73
ke4nt
Posted by roberts on June 12 2004,21:56
I would change my filetool.lst to not backup /home/dansmall/.fluxbox and not backup /home/damnsmall/.xtdesktop That way no conflicts with myDSL extensions.In your filetool.lst add /home/damnsmall/.phoenix that will backup your bookmarks.
It is all possible. It is all in knowning/learning where the file "lives" and what you want to do with it, include or exclude.
Posted by roberts on June 12 2004,21:58
Chris,Starting with 0.7.1 there is also a myDSL button inside emelfm. So using the file manager navigate to the desired extension and click a button.
Posted by clivesay on June 13 2004,01:32
Thank you, robert. I have been installing extensions from my HD install. There are a bunch of apps I would like to see extensions for but instead of listing them I will see if I can build one first! 
Chris
Posted by colinbes on June 17 2004,00:28
Interesting thread.I have couple of applications I would like to add to /usr/sbin using dsl extension. I created a tar.gz file using tar -zcvf (and included paths/usr/sbin) command (I didn't rename it to .dsl). During the boot process I noticed error stating failure to load due to read only file system.
I am using dsl 0.6x and used frugal_install to create bootable CF card.
Earlier on in this thread a question is asked about difference between .dsl and .tar.gz - the answer implies (to me at least) that a tar.gz will only extract to /opt or /home/damnsmall directories. Is this correct and will a .dsl extension solve this problem (I want to extract to /usr/sbin).
Unfortunately I am at home now and can't test this out. I didn't think to try this as I was under the impression that a .dsl file was simply a .tar.gz file renamed (is this correct?).
If a .dsl file will extract to /usr/sbin then could you please explain why this is.
Thanks to all
Posted by colinbes on June 17 2004,00:33
My apologies, only after reading preview and submitting did I realise that I am using ver 0.7 and not 0.6xCheers
Posted by roberts on June 27 2004,00:07
DSL extension authors, I am having some concerns with the lsm files. I personally think it is confusing when the lsm was designed for software authors. Extension authors are merely re-packaging the goods. Not that this is necessarily trivial. It is not. However, the fields in the lsm template are cofusing. As Author, Version, Desc, License, in my opinon should apply to the software. The maintained-by should be used by with a disclaimer that being re-packaged by. Also, I think it is critical that we have change log fields for the repackaging. The user of the extension not only needs to know what version of the sotware this is but also what changes have been made to the re-packaging. Icon added, directory now writeable, fix permission bug, etc. So, are we limited to only these fields in the template? Is there some internet wide indexing engine that will limit us. If you look at some of the lsm files that I recently posted you will see how I tried to curve-fit what I am talking about. I would like to re-arrange the fields so that the original author and related fields are grouped together followed by maintained-by and add change log fields. What say you? Extension authors and those who enjoy using the extensions please post.
Posted by cbagger01 on June 27 2004,00:42
I think that regardless of the original use of the *.lsm files, here at DSL we are using them as plain text files that describe the contents of the DSL package.Since we are not massaging the *.lsm through some kind of package indexing system or database program, the format and structure of the file is no longer critical.
For example, if I wanted to type in a "README.txt" type freeform explanation below the standard *.lsm fields, it would help the user who clicks on the files and reads them.
The ultimate purpose of the myDSL *.lsm files is to make things easier for the end user who is clicking on them while viewing them from their web browser.
I guess that my pet peeve is *.lsm files that don't actually TELL me anything that will help me decide whether to download and install or not.
For example, hypothetically there might be a package called "xlemmings.dsl" and the *.lsm file says something like "contains the xlemmings program".
Thanks for all the help. I could have figured out that part without an *.lsm file.
What is xlemmings? Is it a Game? Word Processor? Programming Language?
If it is a Game, what kind of game is it? A card game? A strategy game? An arcade game?
These burning questions are just begging to be answered. And they could probably be answered in 2 or 3 sentences, tops.
Just my $0.02
Posted by ke4nt1 on June 27 2004,02:05
Well, having not ever used an .lsm file before the creation of "mydsl', I was building my .lsm files, based on the template John provided us with.Following that template, my belief was that the .lsm files were present to show people when updates were made to our .dsl files, by the version and
the date posted within the .lsm, and to verify the sizes of the md5's and .dsl's
When I make a change to a .dsl , which is common for me to do. ( I always find mistakes AFTER i post it up
)I change the version, date, and the byte counts to reflect the change.
My thought was that the template provided to us was somewhat strict in the pattern or order of information listed.
I was not aware we were free to add text and extra information about the program.
I would enjoy that freedom, posting both some details about the program, and about changes I made, as compared to the original tar or deb.
John and Robert need to set a standard for this, and then us contributors can follow your plan or template for info placed into the .lsm
I can only imagine what a new user or contributor must think about maintaining and updating .dsl's, md5's, and lsm's , having only used linux a few times, and looking to get a favorite item to work with DSL
I do agree with roberts item about listing the version of the SOFTWARE, and perhaps another line listing the version of the .dsl file.
I already list the author of the SOFTWARE, but my descript could improve.
ke4nt
Posted by roberts on June 27 2004,04:13
When I started to write the lsm, I kept thinking is this for the SOFTWARE or the EXTENSION.This is an example of what I curved-fitted into the existing template
Begin4
Title: PHP
Version: 4.3.7
Entered-date: 06/22/04
Description: The PHP general scripting language repackaged for DSL
Keywords:
Author: www.php.net
Maintained-by: DSL packaging by robert[at]damnsmalllinux[dot]org
Primary-site:
Alternate-site:
Original-site: www.php.net
Platforms:
Copying-policy: php license
End
This is what I am proposing...
Title: PHP
Description: The PHP general scripting language
Version: 4.3.7
Author: www.php.net
Original-site: www.php.net
Copying-policy: php license
==============
Extension-by: robert[at]damnsmalllinux[dot]org
Change-Log: 06/22/04 - Initial posting with entry for Money Web Server
I dropped keyword as there is no index/search on these files.
I dropped Primary-site and Alternate-site as it is the repository and the mirrors
I dopped Platforms. Its Linux repacked for DSL
I don't show the byte counts. Isn't that the purpose the md5sum?
I modified Maintained-by to Extension-by
I modified Entered-date: to Change-Log
The first six lines give full credit to the original Author/Company and apply to the SOFTWARE only
Followed by the extension contact info.
The Change-Log can then be added to with the format date and change/corrections
The Change-Log entry can keep growing as we fix, change, or add features. That way the "lsm" can reflect the changes we make to the re-packing of the extension.
I think it is important to have a standard. Let me know if you think I am missing something.
Posted by ke4nt1 on June 27 2004,14:20
So far, it's a good layout, roberts ...The items dropped will make lsm construction much easier for
many new builders ...
I like cbagger01's idea of more description space ...
The changelog space is very usable ...
Beats re-writing the whole damn thing again ...
( especially recalc on the byte counts )
What about a package version number ?
( or will date in changelog be the identifier )
After looking on the web at different sites using .lsm's,
perhaps we should name then .log , and not .lsm,
since we are using them as a continuous history and log file,
instead of indexing and software authoring.
Next, I suggest finding a webspace for uploading .dsl's for testing,
since many users do not have public webspace available to share their files..
Some space where users can go to upload a new build,
or download a submission BEFORE it is placed into the repository,
during its initial creation/testing phase...
This would also keep John and roberts from having to deal with all this
extension traffic during building and testing of both new and upgraded .dsl's
The repository > = stable tested extensions for current version of DSL
The webspace > = unstable testing builds and new additions to DSL for review
A place where we can exchange builds, and upload improvements
to other builders .dsl's before a user submits them to the DSL repository ...
( the forum works to some degree, just no file exchange without using email )
73
ke4nt
Posted by John on June 28 2004,01:42
We can use ibiblio's system to get the extensions into the repository, but that will mean sticking to the way ibiblio likes to use .lsm files and working with their anonymous FTP system.Check out my post at:
< http://damnsmalllinux.org/cgi-bin....#entry3 >
Unfortunately Robert, it means we are going to have to use a less than perfectly logical format for .lsm files because we need to make sure that the right hooks are used to make the ibiblio robot put the files in the right place.
Posted by roberts on June 28 2004,01:57
No problemo here. If we can use robots to help automate. Then that's the way to go.
Posted by ke4nt1 on June 28 2004,03:37
Based on this current thread, and the other one John referred to here, < http://damnsmalllinux.org/cgi-bin....#entry3 >we seem to be going back and forth with how to manage extension submissions, and formatting of .lsm files that are useful and meaningful
to DSL users, easy for builders to construct, and unique in being able to log history data about the extensions growth and improvements.
I like the idea of taking some weight off of John and roberts load. Having to manage this .dsl extension load is another 'job' for them to do.
We're all better off having them continue to make progress on DSL.
I'm guessing John and roberts can upload to ibiblio, without the strict lsm policy, and can place the uploads wherever they want ...
(seeing as our current lsm files have made it into the repository without this strict policy..)
What we need is a 'testing ground' , that users can upload and download either thru login or anonymously, (prefer logins), that updates quickly.
That has the flexibility to use some or all of the suggestions made recently regarding lsm files, extension updates and improvements, and other
scripts and addons that benefit DSL and its users.
When a contrib is past a point of 'testing' , then it can be submitted into the repository as a 'stable' extension by either John or roberts.
We have other sites besides ibiblio that users can download the latest .iso, boot img's , tar.gz, etc... Why can't a similar site be setup for DSL testing?
It's my feeling that the ibiblio solution will cause John and roberts as much effort in requests for bad and refused submissions, as well as the confusion
roberts pointed out about filling the /damnsmall main dir with untested dsl's.
In the spirit of DSL having a reputation for being well tested, and continuing to be in or near the 'top ten' distros,
having several 'untested' extensions in the main ftp area is not in the best interest of DSL, and still would have John and roberts managing and moving them frequently...
Certainly, we don't want a reviewer stopping by for a taste of DSL, only to add to his review that many applications were broken or not working..
And what a disaster if someone were to upload damaging, intentionally harmful, or simply destructive files, and these get used by DSL members..
I strongly feel this process should have some distance from the main DSL presence.
If cost is an issue, for webspace or location, certainly there are some alternatives to be discussed here. I would be receptive to increasing my
donations to DSL to make this happen. Other users will have to speak up for themselves, but webspace and bandwidth are cheap these days.
Certainly, one of us could branch off on our own, and easily host a DSL extension website, and manage it on our own, but that would be taking it
out of the 'home', so to speak... I'd rather see this done under the direction of John and roberts, as this is their 'baby'.. My wish is simply to contribute..
In short.
1.) Ibiblio is the slow, messy, but cost effective way to manage this issue.
2.) roberts has some good ideas about unique uses for DSL lsm files. I'd like to see that these suggestions have a place for experimentation.
3.) Extension building and development would progress much faster and have more involvement if the procedure and location were more user friendly.
My thoughts, and of course, my support.
ke4nt
Posted by ke4nt1 on June 29 2004,06:51
Feedback... I'm looking for some feedback to my last post here,extension users and builders....speak up... this impacts YOU..
Thinking caps,
Caffeine overload,
New keyboard,
Whatever it takes...
Get some ideas rolling in here... now's the TIME.
ke4nt
Posted by nucpc on June 29 2004,11:40
Not overly sure which one of the two current thread this should best be postedin..... but I'll put it here (as response has been requested).
On the untested .dsl front I agree with many of the views expressed so far,
particularly those concerning broken .dsl's appearing in the repository, which
should be avoided at all cost. I do understand that I come from the luxurious
position of having some available personal webspace though.....
I attempt to do what I can for other users and I would hope that people who make
.dsl's can manage, by cooperating with others, to post, test and finalize a
package before it is included in the repository. Pre-filtering via the forum I think
is the only way forward. So advertise you .dsl, if folk are interested (which is an
important requirement) let that be known and anybody who can, try and help out.
On the other subject under discussion, .lsm's, I'm very very uncomfortable. I feel
that at best they're a necessary evil but we're still verging close to `claiming' work
that isn't ours. All I do is repackage other people's hard work and any effort involved
in that is pretty trivial by comparison. So a minimum use of these things would
please me.
All the best.
Posted by roberts on June 29 2004,16:50
| Quote |
| On the other subject under discussion, .lsm's, I'm very very uncomfortable. I feel that at best they're a necessary evil but we're still verging close to `claiming' work that isn't ours. All I do is repackage other people's hard work and any effort involved in that is pretty trivial by comparison. So a minimum use of these things would please me. |
I couldn't agree more with you nucpc!
That is why I have been relucant to partake in posting them. That is why I posted "my curve fitting" use of them.
I still have major issues. The fields author, version, title, desc, license, I believe belong to the original software author/vendor.
I am even uncomfortable with the maintained-by field. I don't maintain openoffice, php, pyhton, etc., etc.
If these are going through ibiblio.org are they then subject to the search engine at ibiblio.org. < http://www.ibiblio.org/linsearch. > Talk about "stepping on toes!" That would mean anyone would possible find these. ibiblio hosts many many projects. Even more of a reason that I don't want to seemingly take credit where it does not belong!
And looking in the repository, how can I tell the difference between ayttm.dsl and ayttm2.dsl, as there is no field which indicated what has been changed, added, deleted!
Posted by cbagger01 on June 29 2004,17:33
All of your points are valid concerns.Unfortunately, the "mirror" system does not seem to be taylored for our needs with regards to testing packages.
It seems like we have a few options:
(1) Try uploading files to ibiblio and deal with the screwed up lsm / upload issues.
(2) Create a short script to semi-automate the lsm creation process. If people use the script to generate their lsm files, the odds of a screwed up lsm file are much lower. If we really wanted to make it foolproof, the script would even do the ftp upload for you and send the files to the correct location.
(3) Someone with a few hundred megs of web space graciously donates it for use as a testbed for DSL extensions. This person also figures out a way to allow users to upload their files in a managable way.
Option (1) sounds like a disaster waiting to happen. Others or myself can help out with option (2) although it would be a few days before I can get around to it. Also it will require a test cycle before it is published for general use. Option (3) would be the best solution, but it requires a volunteer to make such a committment come true.
Posted by nucpc on June 29 2004,18:34
I can't promise anything but I will try to have a look at something along the lines of`Option 3'........it'll take a couple of days for the softening up process on my System's
Manager to take effect but I'll get back to you as soon as I can.
All the best.
Posted by John on June 30 2004,00:23
Alright, no lsm files! We'll come up with another standard.We are also working on a system which will have more flexibility than using the Ibiblio anonymous ftp, stay tuned.