a USB HD install

Forum: HD Install
Topic: a USB HD install
started by: winjimmy

Posted by winjimmy on Jan. 30 2006,18:05

I've been trying to do a DSL hd-install to a USB drive. Yes, a flash drive. No, I don't want the USB-pendrive or USB-zipdrive install. I don't want the CD image or compressed file to boot from my USB drive. I want a for-real ext2/3 partition on my USB drive, with a for-real DSL file structure and hierarchy. And yes, I read your (or someone else's) dire warnings about doing this over here < http://damnsmalllinux.org/cgi-bin....;hl=usb > . The gist of the warning is

Quote

Don't do an hd-install of DSL on to a USB flash drive or thumbdrive.

Flash memory only allows a limited number of write commands before the memory will fail (10,000 writes or so).

If you install DSL to the USB drive and perform disk-intensive operations like SWAP file/ SWAP partition, or when you create LOG files or database files, many writes are peformed against the same area of your flash drive and it will fail soon.

Know what I say to this warning? Big hairy deal. Big, fat, hairy, friggin' deal. I will have paid about $25 for this flash drive after rebates. If this drive only lasts me 6 months to 1 yr., but allows me to tote around my work and use a real Linux filesystem and directory structure during that time, I don't care. It will have been $25 well spent, if it allows me to do that. You know, I got ready access to an ftp server and various web storage schemes. I can back up all crucial data there, and if the $25 drive gives out, I'll just chuck it in the trash and get another. But, since my project should be finished within 6 months, another one prolly won't even be necessary.

So, now that you've got the point--the point being I want a real hd install on a USB flash drive and I dont care if it means the drive won't last until the year 2010--let's talk about how I can do that. I know ways of doing this--old-fashioned ways that involve installing to a real HD, then moving all the files over to the USB drive. But that's kinda convoluted. If worse comes to worst, that's what I will do, though. Cuz the install-to-hd script built into DSL ain't workin'. I can create ext2/3 filesystem, but copying over the files fails. Why? Is there some protection built into DSL that won't allow it to complete a HD install to a USB drive? Is there some bug (version 2.1b, btw) interfering? This is the first matter I gotta address. I'd rather not use the old-fashioned method, but would like for DSL's install-to-hd script to do this. Input on getting the routine to work on my USB flash drive will be appreciated.

Second matter is about booting. I plan on creating a GRUB boot floppy and using it to boot from the flash drive. I have some experience with creating this floppy and booting a system with it--though that project involved a regular IDE drive. But for now I anticipate I'm not going to have major problems with the floppy boot scheme.

So, hows about some help with getting the instal-to-hd script to work on my flash drive, eh? You know, save me havin to install to HD, reboot, copy everything over and so forth. It'd be much appreciated.

Thanks, James

Posted by cbagger01 on Jan. 30 2006,18:22

So you can you just use the regular HD install script but choose /dev/sdaX for your partitions instead of /dev/hdaX? (where "X" is the partition number)

If your BIOS supports USB HDD booting, then no grub floppy will be needed. Otherwise, you will need some way to boot the USB device like GRUB or similar.

Posted by winjimmy on Jan. 30 2006,18:45

Quote (cbagger01 @ Jan. 30 2006,13:22)

So you can you just use the regular HD install script but choose /dev/sdaX for your partitions instead of /dev/hdaX? (where "X" is the partition number)

I guess it wasn't totally clear from my post that the hd-install script is failing? Read it over again and see. I get through the step of creating the ext2/3 partition fine. But it won't copy over any files. It just sits there forever, acting like it's going through some step. But no files ever appear on the USB drive. Do you have a USB drive you'd careto test this on? I know it's not working for me (again, version 2.1b): I tried it 2 or 3 times and it always fails. I deduce that either the install script is buggy or some sort of protection has been written into the script that won't let it copy to a USB drive (e.g., to sdX). So, given this failure, how might I make it work? Use another DSL version or something?

Quote

If your BIOS supports USB HDD booting, then no grub floppy will be needed. Otherwise, you will need some way to boot the USB device like GRUB or similar.

I think it does. But it's not my computer, so I don't want to go fiddling in the BIOS and would rather use a floppy. If I can make this work, I'll write up a how to, since there must be others who would want to do something like this too.

James

Posted by winjimmy on Jan. 30 2006,23:00

I got the dsl-hdinstall to work for my USB flash drive. It must'a been somethin about the computer I was trying, cuz it worked first time on this other machine I have at home. Now I'm on to the boot problems. The directions I read for booting DSL with GRUB that made me think I could get this working with a GRUB floppy presume a machine that can boot from USB drives. Because the BIOS allows this, it can apparently represent the USB flash drive to GRUB as a hard drive. On the computer where I've gotten the dsl-hdinstall script to work, however, there is not BIOS option for booting from USB, so GRUB can't see the USB drive and thus boot it. Maybe I'll try it on the real target computer, where booting from USB is possible (I think). I decided to try DSL's bootfloppy-usb on the non-USB-booting machine just to see what would happen. Like I expected, it kinda goes up in flames cuz it can't find the KNOPPIX image. But oddlly, what it does do is boot a an ext2 Linux partition I have on the HD of this machine, but using the DSL kernel! So it does boot to a working system--just not the DSL one I want. It seems like it's pretty close to working though. Some minor tweaks to key files might make it finally boot the flash drive--or maybe unhooking the HD so it has to find the USB flash would do it. We'll see. More experimentation needed, it seems.

James

Posted by cyberoidx on Jan. 31 2006,04:41

what if you suddenly lose your dataa?

Posted by winjimmy on Feb. 01 2006,19:19

Quote (cyberoidx @ Jan. 30 2006,23:41)

what if you suddenly lose your dataa?

Perhaps a moment of satori would just as suddenly ensue . . . ?

Anyways, I've finally got this thing booting from a boot floppy: SUCCESS! (though with some limitations). My solution was as follows: make a bootable FreeDOS boot disk with only minimal files on it for booting the system; put a copy of loadlin on it; put the DSL kernel on it (linux24); put a specially-crafted initrd (that loads modules for, then initializes the USB drive) on the diskette. Put the USB drive into the right USB slot (seems dependent to some extent on the slot it was in when you did the HD install to it), boot from the diskette, then issue

Code Sample

loadlin linux24 initrd=name-of-ur_initrd root=/dev/sdXX

(XX gets replaced by the letter and partition number of your flash drive: mine was a1 [sda1]). Then, you're off to the races. Your drive should boot--mine does. Copying over the kernel and finding a copy of loadlin was small potatoes. The specially-crafted initrd was the more painful part. For the initrd, I appropriated one from a project called "Runt" that aims to run Slack from a USB flash drive. The site is at < http://runt.mybox.org > . He's got a boot floppy there that'll actually boot DSL installed on a flash drive as-is: problem is it uses the wrong kernel for DSL. So you won't be able to do much on the system. To do anything, you'll have to extract his initrd from the floppy image, unzip it, mount and edit it, then re-gzip it, finally copying it over to your boot floppy. I found helpful directions for editing the initrd at < http://simonf.com/usb/#older > . It has you mount the unzipped initrd as a loop filesystem. You use that same process to get the gzipped initrd out of the floppy image. If there's any real interest in doing this, I can provide more detailed directions. But I'm guessing most folks are happy enough with the live-CD-from-flash-drive method of running DSL from flash. It does, apparently, make your flash drive last alot longer to do it this way.

Caveats: booting DSL this way will undo alot of the plug-and-play happiness DSL brings. When you boot from floppy like I've described, it first of all mounts the disk read-only (ro). You can't do much with the system without making it read-write (rw). That's fairly easy to do, and the kernel even spits out a final message telling you how. On top of that, none of DSL/Knoppix's hardware detection wizardry works: all that resides in initrd's big brother--minirt.gz that regular DSL/Knoppix run on boot. What this means is that you'll have to load modules by hand to get all your hardware enabled and functioning (for exampe, your NIC). So, at present, it's not for the faint of heart or extremely inexperienced. But it works for me, and gives me what I want. Depending on response, I may post further to this thread. For those with a bit of experience and/or tenacity, you should be able to figure out all this on your own from what I've said here and the links I've provided.

Be aware that USB is picky. It may not be possible to do this on some computers, or it may take a great deal of experimentation and trouble-shooting to figure out how. YMMV.

James

Posted by winjimmy on Feb. 02 2006,04:23

Quote (Guest @ Feb. 01 2006,14:19)

Caveats: booting DSL this way will undo alot of the plug-and-play happiness DSL brings. When you boot from floppy like I've described, it first of all mounts the disk read-only (ro). You can't do much with the system without making it read-write (rw). That's fairly easy to do, and the kernel even spits out a final message telling you how. On top of that, none of DSL/Knoppix's hardware detection wizardry works: all that resides in initrd's big brother--minirt.gz that regular DSL/Knoppix run on boot. What this means is that you'll have to load modules by hand to get all your hardware enabled and functioning (for exampe, your NIC). So, at present, it's not for the faint of heart or extremely inexperienced. But it works for me, and gives me what I want. Depending on response, I may post further to this thread. For those with a bit of experience and/or tenacity, you should be able to figure out all this on your own from what I've said here and the links I've provided.

I was wrong about this. The read-only mounting problem was because I had the wrong fstab entry for the root filesystem. I set the thing up on another machine where the flash drive was sdb1, and an fstab entry was created stating that the root filesystem was there. Moving it to another machine made sda1 the location of the flash drive, thus the fstab entry was wrong and e2fsck got confused by it and would only mount the partition read-only. Once I changed the root filesystem entry to /dev/sda1, e2fsck was able to run on the drive and it got mounted read-write, like it was supposed to. What's more, the hardware detection/module-loading/network settings stuff all ran fine after this. So the plug-and-play happiness that is so characteristic of DSL/Knoppix all works fine, even on this flash drive install! I did have some hardware troubles with the flash drive though. I decided to go out and get a regular USB HD enclosure and stick one of the gazillion drives I have laying around into it. I'll see if this actually works.

James

Posted by winjimmy on Feb. 03 2006,18:01

The external HD enclosure with a real HD in it did the trick. I've now got a portable DSL on HD I can take around to any computer that can boot from floppy and use DSL on it. Bit of a different booting/disk usage scheme than the live-CD-from-flash method, but could be prefereable in some cases. Final advice: if you wanna try something like I did, be aware that a 512MB flash drive is not large enough. A fresh DSL install takes up only about 1/3 of such a drive, but once you start running the system, and especially if you wanna add any progs, it ballons way out of scope for the 512MB. My problem was that the drive was running out of space, so I was getting error messages and the system basically ground to a halt (hard reset needed). If you wanna use a real flash drive instead of HD in an external USB enclosure, I'd recommend nothing less than 1GB.

Another potential problem is the way the drive can get assigned a different device file (e.g., /dev/sdb1 rather than /dev/sda1) on different machines. If that happens you'll get a kernel panic cuz the root filesystem can't be found (fstab entry for / is wrong). Or it will mount the drive read-only. It can take some experimentation to find out which is the right USB plug for the device name associated with the root filesystem in your /etc/fstab. Another reason why doing dsl-hdinstall to USB devices is discouraged?

James

Posted by cbagger01 on Feb. 04 2006,05:38

The main reason is that most USB devices are flash pendrive types and frugal is the preferred method to protect against premature device failure due to too many writes. Also, frugal is more space efficient, is more easily upgraded, and is harder to "break" because the base filesystem is read-only.

However, the truth is that for linux kernel versions 2.4.x, USB storage devices (including your external hard drive) are enumerated as if they were SCSI hard drives, which is why the "sdx" naming convention is used. Since it is possible to do a traditional hd install to a SCSI hard drive, when you get down to business, there is no good reason why you cannot accomplish what you have done.

The biggest hurdle (besides the whole "The drive is sometimes detected as "sdb" thing) is the difficulty of using traditional bootloaders ON THE SAME HARD DRIVE, to boot up the operating system. If the target PC does not support USB hard drive booting, you get trapped unless you come up with a creative workaround.

FYI, if you want to help cut back on the "sdb" situations, please try to plug your drive into the REAR ports of the PC, preferably port 1 or 2.

Usually, the rear ports are ranked 1st in the sequence of the detection process, so your hard drive is more likely to be found as "sda" instead of "sdb".

You could also try an approach similar to KNOPPIX/DSL and create a miniroot initial ramdisk. Instead of booting directly to your "root" filesystem, you would first boot to the miniroot filesystem, then use the linuxrc to autodetect your hd installation device name (sda,sdb,sdc, etc...) and then chroot over to this filesystem and run the "real" init process.

In theory it is possible. It just takes some playing around to get it up and running.