Wiki on DSL?

[News]

News

[Stats]

Stats

[DataPacRat]

I'm in the early stages of a project proposal, in which various sorts of data are to be copied onto archival-quality DVDs/Blu-Rays and shelved indefinitely, for (much) later future reference. I'm considering throwing DSL onto at least one disc, and possibly even on every disc - if I can figure out how to do so properly, and have it work.

One of the more popular proposals to try this general idea out is to make a local copy of a reasonably current copy of the English Wikipedia, along with the software to read it. Compressed copies of Wikipedia can be found at TPB, running around 5-10 gigabytes, which should fit on a Blu-Ray disc; what I'm hoping to find is whatever software would be necessary to add to DSL to read any of those archives.

Does anyone reading this know offhand if DSL extensions for this already exist? If not, how hard would it be to arrange?

My first instinct is to try the Debian package of Kiwix ( http://packages.debian.org/sid/kiwix ), and one of the pre-indexed ZIM dumps ( http://www.kiwix.org/wiki/Template:ZIMdumps ) - but as I still have to read man pages for even the simplest bash commands, I'm not very confident that I'd be able to get an experimental Debian apt-get install package to work on DSL without breaking something...

[CNK]

First of all, I'd think twice about using even archival disk media. It usually uses a metallic layer that can't corrode as in normal discs, however I don't believe it does much to prevent the deterioration of the organic layer which the laser burns to store the data. There's lots of controversy about this though. By the way, disc technology really hasn't changed since the CD days (hence the following links), the lasers have just got better.

http://www.myce.com/news/cd-recordable-discs-unreadable-in-less-than-two-years-6450/
http://www.cd-info.com/archiving/longevity-report/index.html
http://www.cdmediaworld.com/hardware/cdrom/cd_quality.shtml

I suppose the discs you got say they are guaranteed for 100/200 years. When I read that though, I just think of the VHS tapes I have with "Lifetime Guarantee" written on their jackets. Can you imagine trying to get compensation after you lost a precious home movie on VHS tape that was recorded twenty plus years ago? Heck the company might not even exist.

Of course, then you have to wonder what media is suitable. Mechanical HDDs are probably in the same category as Floppy disks in term on non-suitability due to their mechanical complexity. I don't know the facts, but it seems like far too long relying on the charge storage of Flash Media. Tape drives? Well the majority of VHS tapes lasted all right (though some of those guarantees probably deserve to be claimed) and they often use similar technology, but I wouldn't recommend it. I read an article about Bell Labs using some "holographic" technique to use lasers for burning data into crystals, fun for a read anyway.

However I think the best bet is with Millenniata's "M-Discs" (http://www.milenniata.com/). As much as the advertising screams to me those old "last until we're out of business" guarantees, the principals seem very sound. With the high powered laser diodes of today (trust me, they are), there's no real need for special organic dyes when a good laser can burn into the solid surface of the disc. You will need to buy a compatible DVD drive though, and their rather expensive discs.


Anyway, onto topic, it looks like Kiwix was only released last year, so the chances of finding it in the repository are pretty much zero (especially seeing as I did a search). You can forget about using the Debian package at that link, DSL was compatible with Debian 3.0 "Woody" which is now long obsolete. The Debian "Sid" package you linked to is for a version very different from DSL and just won't work.

As a result you will have to compile the software yourself, if it is at all compatible. Indeed with such new software, it could well be incompatible with the 2.4 kernel version used in DSL. Still, well worth a try if you're up to it. There are some packages in the DSL repository for setting up a compiling environment in DSL. Basically you'll need "gnu-utils", "gcc1-with-libs", "compile-3.3.5" and "gcc-2.95". Though "compile-3.3.5" is getting a bit old now so you may end up having to compile some new programs to use instead of the old versions it includes. Then you need to download the source of Kiwix, look at the "README" and "INSTALL" files from the tar.gz archive and work out the dependencies it needs (many won't be in DSL by default, or will be too old). You now need to find and download these and try to install them (look at the instructions in the INSTALL file, or just do the normal "./configure --help | less", choose arguments to append, if any, "./configure <args>", "make", "make install"). While doing these, you will find the "configure" script will probably want yet more dependencies before it can install some dependencies. Yes dependencies need dependencies (especially in DSL), this can go on for a while. Then you may well at some point find a dependency that just doesn't like kernel 2.4, at which point you can either find a way around it, move to DSL-N with the 2.6 kernel, or give up.

Well that's got to have put you off. I'm not saying that it's bound to be that much work, but I know it can be (anyone else tried to compile "glom" database software for DSL?). Go on, give it a go, look what you'll get out of it.

[DataPacRat]

However I think the best bet is with Millenniata's "M-Discs" (http://www.milenniata.com/). As much as the advertising screams to me those old "last until we're out of business" guarantees, the principals seem very sound. With the high powered laser diodes of today (trust me, they are), there's no real need for special organic dyes when a good laser can burn into the solid surface of the disc. You will need to buy a compatible DVD drive though, and their rather expensive discs.

As it happens, M-Disc DVDs actually are the archival discs I have in mind.

(The company put out a press release last month that in the second quarter of this year, they'll be releasing Blu-Ray M-Discs, which they claim will even be writable on regular drives. Given the seven-fold increase in capacity, I'll likely be waiting for that product to store the main archives on.)

Anyway, onto topic, it looks like Kiwix was only released last year, so the chances of finding it in the repository are pretty much zero (especially seeing as I did a search). You can forget about using the Debian package at that link, DSL was compatible with Debian 3.0 "Woody" which is now long obsolete. The Debian "Sid" package you linked to is for a version very different from DSL and just won't work.

As a result you will have to compile the software yourself, if it is at all compatible. Indeed with such new software, it could well be incompatible with the 2.4 kernel version used in DSL. Still, well worth a try if you're up to it. There are some packages in the DSL repository for setting up a compiling environment in DSL. Basically you'll need "gnu-utils", "gcc1-with-libs", "compile-3.3.5" and "gcc-2.95". Though "compile-3.3.5" is getting a bit old now so you may end up having to compile some new programs to use instead of the old versions it includes. Then you need to download the source of Kiwix, look at the "README" and "INSTALL" files from the tar.gz archive and work out the dependencies it needs (many won't be in DSL by default, or will be too old). You now need to find and download these and try to install them (look at the instructions in the INSTALL file, or just do the normal "./configure --help | less", choose arguments to append, if any, "./configure <args>", "make", "make install"). While doing these, you will find the "configure" script will probably want yet more dependencies before it can install some dependencies. Yes dependencies need dependencies (especially in DSL), this can go on for a while. Then you may well at some point find a dependency that just doesn't like kernel 2.4, at which point you can either find a way around it, move to DSL-N with the 2.6 kernel, or give up.

Well that's got to have put you off. I'm not saying that it's bound to be that much work, but I know it can be (anyone else tried to compile "glom" database software for DSL?). Go on, give it a go, look what you'll get out of it.

As a compromise to get myself more used to Linux, I've put Puppy Linux on a USB drive and have started using it as my main OS. It's not quite as small as DSL, but comes a touch closer to what I'm familiar with, while still giving me a bash shell, and all that other Unixy goodness. I'm quite happy to learn how to compile - for one, I'm rather fond of the screensaver I've been using for years ( http://www.fourmilab.ch/skyscrsv/ ), and if I can work out both how to compile it (from the source at http://www.fourmilab.ch/skyscrsv/download/3.1b/skyscrsc.zip ) and get it arranged so it can be a Linux screensaver instead of just a standalone executable, I'll be even happier. I figure if-and-when I can figure out, or find, how to manage that, I'll be much closer to working out how to recompile Kiwix for DSL.

[CNK]

As it happens, M-Disc DVDs actually are the archival discs I have in mind.

(The company put out a press release last month that in the second quarter of this year, they'll be releasing Blu-Ray M-Discs, which they claim will even be writable on regular drives. Given the seven-fold increase in capacity, I'll likely be waiting for that product to store the main archives on.)

Ha, so I'm preaching to the converted then. Interesting that they are going for Blu-Ray disks compatible with standard recorders, I thought that Blu-Ray laser power was getting pretty darn strong, but I didn't realise it had reached this point yet. I heard that earlier drives have lower powered lasers than the later ones though, so it will be interesting to see how that works out.

I'm quite happy to learn how to compile - for one, I'm rather fond of the screensaver I've been using for years ( http://www.fourmilab.ch/skyscrsv/ ), and if I can work out both how to compile it (from the source at http://www.fourmilab.ch/skyscrsv/download/3.1b/skyscrsc.zip ) and get it arranged so it can be a Linux screensaver instead of just a standalone executable, I'll be even happier. I figure if-and-when I can figure out, or find, how to manage that, I'll be much closer to working out how to recompile Kiwix for DSL.

OK, I'm afraid you're a bit confused over the nature of compiling Linux software from source. That screensaver is made for Windows, and the source code simply contains the code used to program it for Windows so that curious Visual C programmers can satisfy themselves. However packages like the linux source for Kiwix are (a) designed to run on Linux and (b) include documentation, a configuration script and a roughly standardised layout that means that any old PC user who never programed a computer to say "Hello, World" can still get the software installed.

Basically, the lesson to learn is that "source code" does not mean "Linux". That screensaver in particular would rely on many features particular to Windows in order to function, in fact I expect a complete re-write would be required for it to work in Linux. Often Linux compatible source code is in a gzip compressed tarball, ie. with a ".tar.gz" extension. In rare cases (generally for old and obscure software), it might be Zip compressed, but usually the ".tar.gz" extension is a good rule of thumb. Err, except for the ".tar.gz" extensions in the MyDSL repository, they are pre-compiled but compressed in the same way.

Still, I agree with starting small. It's much more rewarding to get a small program with few or no dependencies to run than to begin with a complex dependency chain. Looks like you'll have to find a new Planetarium screensaver for Linux though.

[ms_tux]

 
I'm also confronted with looking into long term archives

Just like VHS tape, that lifetime is "LA-Machine"...
The technology changes, and so the media becomes outdated,
and working machines difficult to find. For example, how many
of Edisons Cyclinders and hand cranked players are still available?

The same problem exists with a lot of what we see right now.
DSL, will become outdated simply by kernel updates. And computers
are becoming so small, that even mini DVDs are big bulky things
that the next generation will laugh at.

Flash drives will eventually become more reliable, but I doubt that
technology will have a huge bulky mini USB connector as the
newest gizmo comes out. At the moment small is in.

Watching the latest emulators gives a hint of what media to use.

Hope that's helpful!