Not Your Father's Operating System
A Developer's Perspective of DSL
What follows is the history of the major design decisions that make DSL what it is today.
As a commercial Linux appliance developer (see: www.shingledecker.org), I had made my own desktop appliances based on Red Hat 6.2 using Icewm. I would occasionally checkout Distrowatch to see if others had similar ideas.
Most everyone knows that operating systems are installed onto hard drives. I had grown tired of installing OS onto hard drives. I had just completed work on several server appliances that do not install to hard drive. Instead they run from a combination of multiple ramdisks and cdrom.
Most liveCDs are designed as demos and have a lot of large applications. This makes them feature rich, but slow to run from the CD, especially on older hardware, such as I have.
I had seen DemoLinux, Thinknic, and Knoppix. So, I was checking the smaller versions. Mostly I was disappointed until I found John Andrew's DSL. This was September 2003. Like many of you, I made my own remaster with additions that I wanted.
Soon I began to post in the forums. I made suggestions for additions. John incorporated many of my early suggestions. Then I began to write to John with my ideas for more structural changes. I won't bore you with all the details and the mods along the way, except to mention the milestones that make DSL unique.
First of all, it seems everybody's mindset is to install to hard drive. That is the way its always been done. That is the way Microsoft does it, the way Unix does it, VMS does it, and so on. It is the way your Father did it. Yet, I find it terribly boring. I like original creative ideas that push the envelope. Writing those non install server appliances had only whetted my appetite.
My interest was to do the same for the desktop. After all, that would be the easiest way to promote Linux, as a robust no install desktop.
Having see the liveCD distros were too slow, for my needs, DSL, was the first small distro that impressed me. It had a solid foundation, thanks to its Knoppix base. I did notice that DSL had a hard drive install script, but so what. I was more interested to see how far I could push the envelope with this small liveCD distro called DSL.
Backup/Restore
The first major milestone that I contributed was an extremely flexible backup/restore script. What good was a liveCD without the ability to save your configs. But wait, there is more than just configs. Instead of trying to dictate or control the user. I wanted to design a method that was wide open. Hence the birth of the filetool.lst. The user has complete control by specifying the full path of files or directories to backup. Then with the creation of the boot time option of restore the system became much more usable. Later I added the autorestore feature.
Writable /opt
DSL like other Knoppix based distros installed most apps into /home, DSL also had the install to hard drive script. Many requests were for multiuser hard drive installations. In trying to keep the liveCD and the installed system consistent. While at the same time keep system requirement to a minimum, (ram and inode usage). I wanted a method to share apps and not have multiple copies in each /home directory. I wanted the least impact to liveCD users. I felt that the best implementation was for a writable /opt. There both liveCD and hard drive installs have the same structure. That structure supports the concept of shared apps and the /opt files could easily use the backup/restore for liveCD users. Everyone benefits with the least system impacts. It is always a design goal to try to keep the liveCD on par with the structure and capabilities of the hard drive installed users.
Birth of Frugal
Some had machines that could not boot from cdrom. Some would boot from floppy. Soon, some would copy over the KNOPPIX/ folder to their hard drive and use the boot floppy. This became known as the poorman's install. At first I did not have much interest in the poorman''s install. Only because a boot floppy seemed like a step backwards. But the challenge of installing and using a compact flash changed everything for me. John, had sent me a link to a website for just such a script. I took a look at Terry Gray's script for "Building a Silent PC". I modified and extended this script making it suitable for DSL. I called it frugal. No more boot floppy! I added an inline lilo and the boot time options for DSL. Now you could not only have no floppy, but a "hands free" bootup. The script prompts for keyboard, toram, daemons and such. The script can be used for compact flash or hard disk, an "enhanced" poorman's. I emailed Terry Gray, thanked him, and sent a copy of my enhanced script and a link to our site. Now DSL is way more than a liveCD. Now it is an easy to install, easy to upgrade, easy to customize boot, all on a small footprint. This seemed to really push the envelope. Note to upgrade, just copy over the single file KNOPPIX/KNOPPIX and your done! Now this added to the other milestones make the standard hard drive installations not so attractive anymore.
Hard Drive Installs Last Advantage
Some of you may remember, my "advantage" posts in the forums. As a developer, I don't disclose what I am working on nor what I am thinking. Until it is done and pr oven, or you could end up looking foolish. But I do occasionally post teaser posts. When I am confident, I will let out some information. Also, I don't want to be copied, especially before I can release it. Anyway back to the topic. The case was still true for users wanting to install software, hard drive installs was the way to go. Debian Apt is hard to compete with. It would not be possible to offer everything for the liveCD user. Also, at the same time, I saw many who copied DSL only to offer a few more MB, a few more apps. They lacked original ideas. Some would wholesale copy every new feature I created for DSL. This was the most frustrating time for me. I would not have minded if they would have given credit. Or if they would have, like I did, email the original author and share their mods or copied version. But this did not happen. My thoughts turned to wanting to offer something really big. Something that would compete with the slightly larger copy cats. I did not want to be just bigger, I did not want to abandon our users on smaller hardware, or the 50MB limit. But I also wanted to give "advantage" to the liveCD users that it seemed only the hard drive installed users had. My solution to eliminate the hard drive installed advantage of being able to add software and at the same time to allow users to easily create any sized iso they wanted. This would allow DSL to compete with the slightly larger distros. DSL would become variable sized.
The Birth of myDSL
With the accumulated experience of the previous milestones I set out to offer such capabilities. I had to keep in mind the less capable hardware that DSL had built its reputation on. Having seen John's break though techniques of download scripts (then it was called firebird). Then with my writable /opt for shared apps. I was able to use openoffice as a proof of concept. I could keep openoffice as a simple gzipped tarball and drop it into /opt. Then create boot time options to load it back into opt. Thereby not having to include it in the backup/restore filetool.lst. I then added another oo-user.tar.gz for the icon and menu item. This was going to work! I tried some other apps, Some I would need a small wrapper script to force the library and bin paths under /opt. Still I needed more. I had seen cbagger's opera script that made more of the file system writable. I emailed cbagger and asked his permission to use some of his script. Opera was never included in DSL because it is ad sponsored. I broke off the parts that I wanted and modified them to what I had in my mind. Still giving credit to cbagger for the mkwritable script. Now having this, I could offer a tiered approach to boot time loaded nodules. At about the same time I became fascinated with mountable applications. I called them .ci for compressed images. I quickly changed openoffice to a openoffice.ci. This would really help in reducing systems resources. Later I combined the ci and it associated user file into the uci. I put together six example modules ( today they are called extensions) and made a small simple script to quickly and easily create a custom mydsl.iso and burn it to cdrom. All within one script. This release had to have much testing, with the already frugal, backup/restore other milestones to ensure a solid reliable system. Looks like your Father's way of installing is fading away. DSL emerges as an any size "distro" that you want; myDSL.
Dynamic Loading
After the release of myDSL extensions and mkmydsl custom cdrom maker, the most requested feature was to load the extensions without requiring the making of a cdrom. John and I were a little hesitant at first. We were concerned because of bandwidth. If users became lazy, they would just download the extensions every time. So I made a boot time option to specify a persistent store for mydsl. This was to be specified by mydsl=hdxx. I added the code for dynamic loading. John created a GUI front end to my mydsl-load programs. Now DSL even offers more, What was that "other" way to install, and why now would I want to do it?
Pendrives
With the increasing popularity of pendrives, I wanted to make the install process easier. Even though DSL was the first to be on a pendrive, the process was a manual one. We would point the interested user to the knoppix.net website for the instructions. I decided to make a variation of the frugal script to copy over the compressed file and supporting software to ake "Install to Pendrive". The frustration with pendrives reminds me of the early days of cdrom drives. There is no standard. The most popular is the 32 sectors per track USB-ZIP boot BIOS option. So the script supports that specification. But do you see a pattern here? Compressed installs. Be it liveCD, poorman's, frugal both compact flash and hard disk, and now pendrives. Your father's way of installing an operating system can't do that.
Qemu
Part of the frustration of booting from USB pendrives is solved by moving the booting to a host OS and running a virtual machine. Thanks in part to Metropipe for bringing Qemu to our attention. DSL again quickly embraces new technology. The frugal install, the "Install to Pendrive" are merged and molded into a new 5-way boot script for pendrives. This gives DSL the most flexibility on pendrives than any other distro. But once again, notice there is no "standard" hard drive install used here. Seems like Father's way of installing does not support the new technology. See what I mean about pushing the envelope.
More Data Stores
DSL's implementation of persistency will be to offer the default writable /home and /opt to external media. We have seen that compressed installs offer a non-corruptible, near bulletproof, small footprint, easy upgradable installation. And with the existing user specified persistent store of mydsl and the immensely flexible backup/restore we have a truly robust system. But, what, what's that I hear, "At what point does the backup/restore become too time consuming?" My solution to this. More data stores. Using the same principle of the mydsl user specified persistent data store to both home and opt. Adding more robust icon and menu management. This will allow "frugal" compressed installs even further advantages. You still have all the benefits and gain again. Can your Father's way of installing to hard disk offer all of these advantages? Can you upgrade so easily, can you reboot to a pristine state, yet still not lose your settings, your extensions, your flexibility, your pendrives, your compact flash, your qemu, all the new technology.
Seems to me installing to hard drive is becoming obsolete.