Storing files on USB


Forum: USB booting
Topic: Storing files on USB
started by: takowl

Posted by takowl on April 09 2008,13:26
It strikes me that using DSL, and indeed certain other distros, on a USB stick is a bit...strained, since they're designed for the rather limited environment of a CD. Specifically, what I'd like to do is to store the files natively, on the USB stick, rather than using the backup/restore features. Key advantages to this:
1. No need to play with backing things up and restoring them--just boot off the stick, and your files are there.
2. When in a different OS, the files can be accessed simply by mounting the memory stick and selecting a folder.

To this end, I would like to change the /home symlink (which is by default to /ramdisk/home) to point to a location on the USB stick, mounted as /cdrom, it seems (so /home -> /cdrom/cargo), and to make the relevant directory writeable as a normal user. I have verified that it is possible to write to files in this location (by becoming superuser). But it does not seem to be possible to give the directory to "dsl" using chown.

I have found how to uncompress the KNOPPIX image, but I cannot see where the symlinks are set up. How might I go about changing the symlink, and giving the default user write permissions?

Thanks in advance.

Posted by Juanito on April 09 2008,13:44
It sounds like you're looking for the "persistant home" feature of dsl - the details are in the wiki.

Alternately, you could leave your configuration files (torsmorc, .gimp-1.2/, etc) in the backup and save other files (docs, photos, mp3, etc)  directly to a folder/partition on the usb stick - something like /mnt/sda1/docs

Posted by ^thehatsrule^ on April 09 2008,17:31
Quote
Specifically, what I'd like to do is to store the files natively, on the USB stick, rather than using the backup/restore features.
However, that's the recommended option to reduce io operations on the flash device, since it would wear it out much faster afaik. (just a tip - but they are getting cheaper)

Posted by takowl on April 09 2008,18:04
Thanks, I think persistent home was what I was looking for.

Experimentation has suggested that the USB disk is a different device depending on what computer it's booted on (and different again if booted on qemu). Will the method on the wiki work for a location on /cdrom? And will that automatically set the permissions appropriately (I will experiment with this later, I'm just putting the questions here as well).

@thehatsrule: I understand, but flash drives are designed to have files written to them, so I think I'll live. In any case, storing it as a backup could, I think, potentially cause more writes (as it's compressed, so changing one file could potentially change the whole backup file (as I understand it)).

Posted by ^thehatsrule^ on April 09 2008,18:10
Quote (takowl @ April 09 2008,14:04)
@thehatsrule: I understand, but flash drives are designed to have files written to them, so I think I'll live. In any case, storing it as a backup could, I think, potentially cause more writes (as it's compressed, so changing one file could potentially change the whole backup file (as I understand it)).
I believe it is the many operations done that wears it out, rather than doing one batch operation at the end.  Most applications have their own profile directory in ~, such web browsers by default, etc. That being said, if you were planning on having a huge ~ for whatever reason, I guess it would take too much time.

Posted by lucky13 on April 09 2008,18:57
Quote
It strikes me that using DSL, and indeed certain other distros, on a USB stick is a bit...strained, since they're designed for the rather limited environment of a CD.

Yes, DSL is a live CD. Yes, DSL is designed to operate in a nomadic fashion. That isn't a limitation, for many users it's a feature.

A persistent home gives you the benefits of the nomadic/CD-based system with your own files located on one partition.

As far as being able to mount your device in another operating system, you would require filesystem support on the other OS. DSL's USB-HDD script sets the persistent home partition in FAT, iirc (been a while since I've booted from USB).

Posted by takowl on April 09 2008,21:47
Quote
Yes, DSL is a live CD. Yes, DSL is designed to operate in a nomadic fashion. That isn't a limitation, for many users it's a feature.


Sorry, I wasn't trying to knock it--just that using a writeable USB stick is clearly going to be different from using a live WORM disk like a CD.

Quote
As far as being able to mount your device in another operating system, you would require filesystem support on the other OS. DSL's USB-HDD script sets the persistent home partition in FAT, iirc (been a while since I've booted from USB).


Sorry? I can mount the USB stick in other operating systems. It's FAT formatted, so I just plug it in and carry on. Unless you mean the KNOPPIX image, but I don't need to access that.

I've had a quick play around, and it seems that persistent home does not do what I'm after. You can only pass it a device name, which has two problems:
1) The device can change, depending on how it's booted.
2) It's already mounted the device, so it tries to remount it and then fails.
Also, as was pointed out, various programs are going to try to write files to ~/.*, which is going to cause unnecessary strain on the USB stick. So, to revise slightly what I'd like to do:

I'd like to symlink from ~/something to a folder such as /cdrom/cargo, and make that folder writeable to the normal user. I'd like this to all happen automatically. So:
-Where can I put a script to run at boot and set up the symlink? (I'm guessing this is quite simple, I just don't know the answer...)
-How can I set a subfolder of /cdrom to be writeable to user dsl? As I remember, chown and chmod said I didn't have permission, even when run as superuser.

Thanks all!

Posted by roberts on April 10 2008,03:56
It not solely WORM. Stating such shows that you do not undertand  DSL.
If you use the standard provided usb-hdd install script you have two partitons. The first, larger one, vfat, is owned by user dsl, the second one is an ext2 owned by root.

If you know *nix permisssions then you know you can write to which you have permission. We do not run everything as root here.

Writing constantly on a pendrive is never a good idea.

The first partition that I mentioned is used both as WORM, only for the large static mydsl extensions in the mydsl directory, and as a floppy to write once upon shutdown. This saves dramatically on writes to the flash device.

However, should you choose to setup your pendrive otherwise then that is your choice and YMWV

Posted by takowl on April 10 2008,15:33
Quote (roberts @ April 09 2008,23:56)
It not solely WORM. Stating such shows that you do not undertand DSL.

No, I got it for the first time a few days ago and have been playing around with it. I don't claim that I understand it. A CD-R, however, is a WORM disk--that much is inarguable.

I didn't use the USB-HDD installation script. I followed the instructions on the wiki for setting up DSL-embedded on a USB flash drive. This leaves me with a single partition (I did try modifying it for two partitions, but when something failed to work, I opted to return to a single partition). Is it possible to have a user-writeable folder on the same partition, or is the only way to achieve this to separate it into two partitions?

I am well aware that we do not run everything as root. I never said I did or intended to.

I would like to write files to a pendrive. This is what they were designed for. I appreciate that perhaps there are ways to enhance their lifetime, but I don't think it will suffer unduly from me writing files to it.

I can't help but feel that the mood has got somewhat confrontational. I'm sorry about this. I understand what I want to do, and I do not think it is an entirely stupid idea. If you are uncertain what I mean, I am happy to try to clarify.

Posted by Juanito on April 10 2008,15:51
Quote
Is it possible to have a user-writeable folder on the same partition, or is the only way to achieve this to separate it into two partitions?

Yes, you can have a user writeable folder on a single partition.

The current "install to usb" menu in dsl makes two partitions as stated previously, but you can manually make a single fat/fat32 partition and boot using syslinux or a single ext2 partition and boot using extlinux and no doubt (I haven't tried) you could also use grub to boot either of these partitions.

See < here >

Posted by takowl on April 10 2008,19:14
Quote (Juanito @ April 10 2008,11:51)
The current "install to usb" menu in dsl makes two partitions as stated previously, but you can manually make a single fat/fat32 partition and boot using syslinux or a single ext2 partition and boot using extlinux and no doubt (I haven't tried) you could also use grub to boot either of these partitions.

Thanks, Juanito.

I'm afraid I didn't quite explain properly--I already have a single (1 GiB, FAT32) partition on my USB stick, with dsl-embedded booting from it. What I would like to do is to give (i.e. chown) a folder within the memory stick (mounted as /cdrom) to user dsl (i.e. the default user in dsl), so that I can write to it without becoming root. When I try this within dsl, it says that I do not have permission, even if I try to do it as root.

Posted by ^thehatsrule^ on April 10 2008,20:52
Quote (takowl @ April 10 2008,15:14)
I'm afraid I didn't quite explain properly--I already have a single (1 GiB, FAT32) partition on my USB stick, with dsl-embedded booting from it. What I would like to do is to give (i.e. chown) a folder within the memory stick (mounted as /cdrom) to user dsl (i.e. the default user in dsl), so that I can write to it without becoming root. When I try this within dsl, it says that I do not have permission, even if I try to do it as root.

That's probably mounted as RO.  Check `mount`

What's your boot parameters?  You could try adding the "frugal" cheatcode or maybe remounting the partition as RW.

Posted by Juanito on April 11 2008,04:10
This allows me to boot dsl from a 1GB usb stick formatted as a single ext2 partition:
Quote
[extlinux.conf]
DEFAULT linux24
APPEND ramdisk_size=100000 init=/etc/init lang=us vga=773 initrd=minirt24.gz nomce noapic quiet frugal toram syslog dma acpid restore=sda1  BOOT_IMAGE=knoppix
TIMEOUT 300


Folders as follows:
Code Sample
$ ls -l /mnt/sdc1
dr-xr-xr-x    2 root     root         4096 Jul 20  2007 KNOPPIX
drwxr-xr-x    3 dsl      staff        4096 Feb 16 03:19 backup
-rwxr-xr-x    1 root     root      1286928 Apr 11 01:24 backup.tar.gz
-rwxr-xr-x    1 root     root         2048 Jul 20  2007 boot.cat
-rwxr-xr-x    1 root     root          123 Jul 20  2007 boot.msg
drwxr-xr-x    4 dsl      staff        4096 Jul 21  2007 build
drwxr-xr-x    6 dsl      staff        4096 Apr 11 01:19 docs
drwxr-xr-x    3 dsl      staff        4096 Sep 28  2007 drivers
-rw-r--r--    1 root     root         2611 Feb 16 03:50 extlinux.conf
-r--r--r--    1 root     root        10072 Jul 20  2007 extlinux.sys
-rwxr-xr-x    1 root     root         1592 Jul 20  2007 f2
-rwxr-xr-x    1 root     root         1595 Jul 20  2007 f3
-rwxr-xr-x    1 root     root          256 Jul 20  2007 german.kbd
-rwxr-xr-x    1 root     root      1005209 Sep 17  2007 linux24
-rwxr-xr-x    1 root     root        33780 Jul 20  2007 logo.16
drwx------    2 root     root        16384 Jul 20  2007 lost+found
-rwxr-xr-x    1 root     root       856748 Feb 16 03:50 minirt24.gz
drwxr-xr-x    3 dsl      staff        4096 Apr  9 18:50 mydsl
drwxr-xr-x   13 dsl      staff        4096 Mar 27 19:27 source
drwxr-xr-x    3 dsl      staff        4096 Apr  5 09:47 tmp

Posted by roberts on April 12 2008,03:23
Not being a windows guy, does fat32 even support multiple permissions? Isn't that why there is/was UMSDOS and UVFAT?

If your requirement dictates a single partiton then I would suggest to use Juanito's method.

Posted by takowl on May 04 2008,23:11
Sorry I've not replied for a while--been busy with one thing and another.

It is possible to write to the mount--I can do so as superuser, and I have verified that the data is still there. I would essentially like to be able to do the same thing without logging in as root.

My preference is to make use of /cdrom, as I know it's consistent (regardless of how dsl is booted or on what system), and it's already formatted correctly for my purposes. If that's impossible, then fair enough...but I can't logically see why it would be impossible.

Posted by ^thehatsrule^ on May 04 2008,23:27
It's due to filesystem limitations as roberts has hinted.
I suppose you could change how it is mounted though...

Powered by Ikonboard 3.1.2a
Ikonboard © 2001 Jarvis Entertainment Group, Inc.