Live / Embeddedness


Forum: DSL Ideas and Suggestions
Topic: Live / Embeddedness
started by: Fordi
Posted by Fordi on Dec. 23 2004,00:22
Ok, I've been experimenting again.

I think it may be possible, with a small patch to linuxrc, to have a live/embedded CD.

QEMU appears to have the ability to have a kernel and initrd start at boot.  I've tested this successfully, and it appears to start, but can't find the CD.  That's fine.  If mounted into QEMU with "-hda path/to/KNOPPIX", it is available.

The only thing missing is a hack to linuxrc to check if /dev/hda *IS* in fact our KNOPPIX image, and mount it.  This should probably be done first ("insmod modules/cloop.o file=/dev/hda" will fail if /dev/hda isn't a cloop image), checked and if working, mounted as /cdrom.

Think about it... pop in a DSL disc and have DSL running inside windows without reboot (and able to access the remote computer..  in fact, using the -append parameter, you could pass the host ip to linuxrc, which could in turn add it to the hosts file as systemhost).  Pop in the SAME disc and reboot, and DSL runs all the faster!

Add to that that the USB and USB-Embedded images could be merged, and you've got a pretty nice idea, don't you think?
Posted by Fordi on Dec. 23 2004,00:24
Why I haven't done this already and am posting with an idea rather than a new copy of linuxrc:

Shell scripting, as I've said before, is not my forte.  linuxrc scares me, to be honest.  I'm getting better, good enough to modify things like mountci, but not THAT much better.
Posted by cbagger01 on Dec. 23 2004,01:34
I agree.

It's a great idea.  I had been thinking about an "embedded" CD-R disk, but I never thought about an embedded/livecd combo.

The "append" idea is also cool.  It allows the user to access the host machine without knowing the raw ip address.  I suppose if you want to take things a step further, you could provide both the ipaddress AND the true hostname of your systemhost computer and the user could talk to it by name.
Posted by Fordi on Dec. 23 2004,03:53
Just a note:
For space's sake, use upx to compress the QEMU binary; it's small, yeah, but smaller is better, and uncompressed data on the base CD == 2x data within KNOPPIX
Posted by Fordi on Dec. 23 2004,04:54
I know I said I wasn't doing this, but I figured I'd go in and trudge about with my so-far-limited shell scripting knowledge, and here's the results.

Inserted at line 528 of linuxrc from dsl 0.9.1:
Code Sample
# last ditch - try to mount KNOPPIX ci file from /dev/hda
if test -n "$FOUND_KNOPPIX"
then
else
echo "${BLUE}Assuming you're running from QEMU${NORMAL}"
echo "${BLUE}Performing last-ditch effort to use ${MAGENTA}/dev/hda${BLUE}...${NORMAL}"
echo "${BLUE}I'm not being clean or silent here, so there'll be an error or two..."
# try to mount /dev/hda as a cloop
$INSMOD modules/cloop.o file=/dev/hda
if mount -t iso9660 /dev/cloop /KNOPPIX
then
FOUND_KNOPPIX="/dev/hda"
echo "${GREEN}Success!${NORMAL}"
else
# try one more time, this time directly trying to mount /dev/hda
if mount -t iso9660 /dev/hda /KNOPPIX
then
FOUND_KNOPPIX="/dev/hda"
echo "${GREEN}Success!${NORMAL}"
else
echo "${RED}Failed..."
fi
fi
fi

This will, after checking EVERYTHING else, try loading from /dev/hda, thus assuming that you're booting from QEMU.
And, yes, I've tested it.  And yes, it does boot.  All three (Embedded under linux, embedded under windows, and naitive as a liveCD) ways.  Go me. ^_^

OH!

the qemu stuff.

starting from ~/dsldisc/ as root (mounted from the test iso), the tree and QEMU commands:

Code Sample
Tree:
autorun.inf - 23
index.html - 391
qemu.bat - 198
qemu.sh - 194
./KNOPPIX
./KNOPPIX/KNOPPIX - 50,158,212 bytes
./boot
./boot/isolinux
./boot/isolinux/
./boot/isolinux/boot.cat - 2,048
./boot/isolinux/boot.msg - 110
./boot/isolinux/f2 - 1,592
./boot/isolinux/f3 - 796
./boot/isolinux/german.kbd - 256
./boot/isolinux/isolinux.bin - 9,404
./boot/isolinux/isolinux.cfg - 1,993
./boot/isolinux/linux24 - 994,807
./boot/isolinux/logo.16 - 36,649
./boot/isolinux/minirt24.gz - 798,277
./qemu
./qemu/qemu - 1,085,736
./qemu/qemu.exe - 237,568
./qemu/SDL.dll - 122,368
./qemu/vgabios-cirrus.bin - 30,868
./qemu/linux_boot.bin - 512
./qemu/bios.bin - 65536


Code Sample
qemu.bat listing:
@echo off
qemu\qemu.exe -L qemu/ -kernel boot/isolinux/linux24 -initrd boot/isolinux/minirt24.gz -hda KNOPPIX/KNOPPIX -append "ramdisk_size=100000 init=/etc/init lang=us apm=power-off vga=788 quiet"


Code Sample
qemu.sh listing:
#!/bin/sh
qemu/qemu -L qemu/ -kernel boot/isolinux/linux24 -initrd boot/isolinux/minirt24.gz -hda KNOPPIX/KNOPPIX -append "ramdisk_size=100000 init=/etc/init lang=us apm=power-off vga=788 quiet"


Code Sample
autorun.inf listing:
[autorun]
open=qemu.bat


Please note that both qemu binaries have been compressed with upx --best
Couldn't tell you why the windows version is smaller, but bonus on that.  Still, the resulting CD is 51.172M in size.  Hope y'all can fix that with a remaster.  I'm not so inclined at the moment, as I've got a bunch of shiny new 3" CDs.

Meanwhile, test it out for yourself.  It'll be fun!
Posted by cbagger01 on Dec. 23 2004,06:06
Excellent Job!

If you can afford the bw, it would be a nice download from your existing myDSL web site.

Maybe if the linux Qemu stuff was removed, it would fit on some 50MB business card CD-R disks.

For me, the Win32 qemu is more useful (and smaller too!) because I am more likely to encounter a Winbox that can't/won't boot livecd or is cumbersome to leave the native OS for a quick task.

If the chosen computer is already running Linux, chances are that the computer is already useful as-is and I would only need DSL as a "personal desktop" and USB drive is more useful than livecd for that purpose.

Just a thought.
Posted by Fordi on Dec. 23 2004,16:18
The DSL 0.9.1 embedded/live combination cd is now available on my site: < http://www.nolifeline.com/dsl-0.9.1-em.iso >

I hunted around and found the smallest builds of qemu.exe and SDL.dll.  They've been compressed with upx --best.  I removed the linux embedded bit and merged qemu.bat into autorun.inf.  In order to save space, I also removed index.html and swapped the old logo.16 in (I'm sorry.  It was pretty, but it took space I needed.)  I also ditched lost+found, as I can't see how it could possibly be used.  I moved boot/isolinux/* to isolinux/* - a paltry 4k savings, but every byte counts at this point (If this is to be used by the DSL team in an offical capacity, I'd like to make it easy as possible to squeeze the remaster down.  68k is a hell of a lot easier than 1.7M).

Code Sample
Tree:
Directory of .
12/23/2004  09:33 AM               122 autorun.inf
12/23/2004  10:22 AM    <DIR>          isolinux
12/23/2004  09:29 AM    <DIR>          KNOPPIX
12/23/2004  09:40 AM    <DIR>          qemu
Directory of ./isolinux
12/14/2004  11:55 PM             2,048 boot.cat
07/25/2004  03:13 AM               110 boot.msg
11/30/2004  12:49 AM             1,592 f2
11/30/2004  12:52 AM               796 f3
07/25/2004  03:13 AM               256 german.kbd
12/23/2004  10:27 AM             9,404 isolinux.bin
12/08/2004  04:27 AM             1,993 isolinux.cfg
08/15/2004  01:31 AM           994,807 linux24
07/25/2004  03:50 AM             7,367 logo.16
12/23/2004  10:02 AM           790,567 minirt24.gz
Directory of ./KNOPPIX
12/14/2004  11:55 PM        50,158,212 KNOPPIX
Directory of ./qemu
07/11/2004  03:20 AM            65,536 bios.bin
11/14/2004  03:55 PM               512 linux_boot.bin
12/04/2004  05:49 PM           237,568 qemu.exe
06/14/2004  03:15 PM           100,864 SDL.dll
07/11/2004  03:20 AM            30,852 vgabios-cirrus.bin
             18 File(s)     52,402,606 bytes

Code Sample
ISO size:
12/23/2004  10:27 AM        52,498,432 dsl-0.9.1-em.iso (50.066M)

Code Sample
listing of autorun.inf:
[autorun]
open=qemu\qemu.exe -kernel boot/isolinux/linux24 -initrd boot/isolinux/minirt24.gz -hda KNOPPIX/KNOPPIX -L qemu

Code Sample
listing of pertinant lines of linuxrc
# last ditch - try to mount KNOPPIX ci file from /dev/hda
if test -n "$FOUND_KNOPPIX"
then
else
echo "${BLUE}Assuming you're running from QEMU${NORMAL}"
echo "${BLUE}Performing last-ditch effort to use ${MAGENTA}/dev/hda${BLUE}...${NORMAL}"
# try to mount /dev/hda as a cloop
echo "0" > /proc/sys/kernel/printk
# hey look, I figured out how to hide errors!
$INSMOD modules/cloop.o file=/dev/hda >/dev/null 2>&1
if mount -t iso9660 /dev/cloop /KNOPPIX >/dev/null 2>&1
then
FOUND_KNOPPIX="/dev/hda"
echo "${GREEN}Success!${NORMAL}"
else
# try one more time, this time directly trying to mount /dev/hda
# this is weird, 'cos it shouldn't work, but usually seems to.
if mount -t iso9660 /dev/hda /KNOPPIX >/dev/null 2>&1
then
FOUND_KNOPPIX="/dev/hda"
echo "${GREEN}Success!${NORMAL}"
else
echo "${RED}Failed..."
fi
fi
fi


All the files necessary to perform the embed mod are avaiable < here >.  All you have to do is extract it to it's own folder, put the isolinux stuff and KNOPPIX in the right place (omitting minirt24.gz, which is already there), and build the disc.
Posted by Fordi on Dec. 23 2004,20:27
Ok, one more time.

embed-mods.tar.gz and dsl-0.9.1-em.iso have been changed.  

The iso now again contains the index.html; I didn't realize that removing it would be GPL violation (removal of the author ident... hehe.. oops).

Both the ISO and the embed-mods now include a custom-built version of qemu.  SDL is now statically linked and also built custom to include only those features required to run in a win32 environment.  SDL, zlib and qemu were rebuilt with no debug data.  All symbols have been stripped from the executable and it's been recompressed with UPX.

All that work saved 29k.  um.  woot.  I'm still about 30k over 50M, but I can honestly say, I've tried.  Without a remaster, I simply can't fit an electric sausage in the iso ^_^.

Meanwhile...
< The ISO >
< The files required to modify an existing DSL ISO to be Live/Embedded >
Posted by Fordi on Dec. 24 2004,01:14
Ok, it took a bit of doing, but I got cbagger01's last suggestion in.
Code Sample
Changes to linuxrc:
------LINE 529, inserting 27 lines:
# last ditch - try to mount KNOPPIX ci file from /dev/hda
if test -n "$FOUND_KNOPPIX"
then
else
QEMU=""
echo
echo "${BLUE}Guess you're running from QEMU${NORMAL}"
echo "${BLUE}Performing last-ditch effort to get KNOPPIX image${NORMAL}"
# try to mount /dev/hda as a cloop
echo "0" > /proc/sys/kernel/printk
$INSMOD modules/cloop.o file=/dev/hda >/dev/null 2>&1
if mount -t iso9660 /dev/cloop /KNOPPIX >/dev/null 2>&1
then
FOUND_KNOPPIX="/dev/hda"
QEMU="TRUE"
echo "${GREEN}Success!${NORMAL}"
else
# try one more time, this time directly trying to mount /dev/hda
if mount -t iso9660 /dev/hda /KNOPPIX >/dev/null 2>&1
then
FOUND_KNOPPIX="/dev/hda"
QEMU="TRUE"
echo "${GREEN}Success!${NORMAL}"
else
echo "${RED}Failed..."
fi
fi
fi
------LINE 673, replacing 8 lines:
rm -rf /etc/ftpusers /etc/passwd /etc/shadow /etc/shadow- /etc/group \
      /etc/ppp /etc/isdn /etc/ssh /etc/ioctl.save \
      /etc/inittab /etc/network /etc/sudoers \
      /etc/init /etc/localtime /etc/dhcpc /etc/pnm2ppa.conf /etc/hosts 2>/dev/null
cp -a /KNOPPIX/etc/ftpusers /KNOPPIX/etc/passwd /KNOPPIX/etc/shadow /KNOPPIX/etc/shadow- /KNOPPIX/etc/group \
     /KNOPPIX/etc/ppp /KNOPPIX/etc/isdn /KNOPPIX/etc/ssh \
     /KNOPPIX/etc/inittab /KNOPPIX/etc/network /KNOPPIX/etc/sudoers \
     /KNOPPIX/sbin/init /KNOPPIX/etc/dhcpc /KNOPPIX/etc/hosts /etc/ 2>/dev/null
------LINE 689, inserting 10 lines
# Hey, we're in QEMU!  Let's get all cozy with the host, shall we?
if test -n "$QEMU"
then
HNAME=$(echo $CMDLINE | /usr/bin/tr ' ' '\n' | /bin/sed -n '/sysname=/s/.*=//p' | /usr/bin/tail -1)
HIP=$(echo $CMDLINE | /usr/bin/tr ' ' '\n' | /bin/sed -n '/sysip=/s/.*=//p' | /usr/bin/tail -1)
echo
echo "${BLUE}QEMU Host is ${MAGENTA}${HNAME}${BLUE} at IP ${MAGENTA}${HIP}${NORMAL}"
echo "${HIP} ${HNAME}" >> /etc/hosts
echo "${HIP} syshost" >>/etc/hosts
fi

Code Sample
Listing of rq.c
#include <winsock.h>
#include <shellapi.h>
int doit();
int doit(char* res)
{
   WSADATA wsaData;
   if (WSAStartup(MAKEWORD(1, 1), &wsaData) != 0) {
       return 255;
   }
int i=0;
   char ac[80];
char ip[16];
   if (gethostname(ac, sizeof(ac)) == SOCKET_ERROR) {
       printf("Error %d when getting local host name\r\n", WSAGetLastError());
       return 1;
   }
   struct hostent *phe = gethostbyname(ac);
   if (phe == 0) {
printf("Yow! Bad host lookup!\r\n");
       return 1;
   }

   struct in_addr addr;
   memcpy(&addr, phe->h_addr_list[i], sizeof(struct in_addr));
sprintf(res, "sysname=%s sysip=%s", ac, inet_ntoa(addr));
   WSACleanup();
   return 0;
}
int main(int argc, char *argv[])
{
char hostip[95];
char* temp;
   int retval = doit(hostip);
temp=malloc(1024);
sprintf(temp, "-kernel isolinux/linux24 -initrd isolinux/minirt24.gz -hda KNOPPIX/KNOPPIX -L qemu -append \"sb=0x220,5,1,5 vga=789 %s\"", &hostip[0]);
ShellExecute(NULL, NULL, "qemu/qemu.exe", temp, NULL, SW_SHOW);
free(temp);
   return retval;
}

compile that thusly (using mingw):
Code Sample
gcc -O2 rq.c -o rq.exe -lshell32 -lwsock32
strip rq.exe
upx -9 rq.exe

It should come out to 3584 bytes.
Code Sample
listing of autorun.inf
[autorun]
open=rq.exe


Ok, now that I've posted all that, what it does:
It allows QEMU to communicate with the host PC via either it's known hostname ("punquin" in my case) or by "syshost".  You see the append line has also changed to include a definition of the emulated SB card.  I haven't tested this (I'm doing my tests from my work (WinXP) computer via rdesktop, and while textmode works fine, using a mouse is a highly wonky).  Still, seeing as the SB card isn't working at all without that line, it can't hurt to throw it in.
Posted by cbagger01 on Dec. 24 2004,04:10
Wow!

I was thinking more along the line of extracting the hostname and ip adress using the batch file, but creating your own C program works too.

As for the 50.066MB ISO image, my guess is that most 50MB business card CD-R disks will work as long as your CD Burning software supports overburning.

I am going to try and use Nero to burn me a copy with my old Imation 50MB CD-Rs.

This will just be -too cool- if everything works out.
Posted by cbagger01 on Dec. 24 2004,06:24
Update.

The live /embedded iso that I downloaded tonight is working for me with the following observations:

This Qemu build must be a little bit out of date.  The arrow keys on the keyboard don't work in an Xterminal window and the network corruption bug still exists in this version.

I suggest either rebuilding with post-0.6.1 latest sources that are newer than 11/21/2004

Or you could just use the exe & dll files from the dsl-embedded-0.9.1 zipfile but this will increase the *.iso size by about 600k

I am guessing that it is possible to overburn a 50MB iso with an extra 660k on a business card CD-R disk.

Also, sound doesn't seem to work, maybe audio was not enabled at the qemu commandline?
Posted by tronik on Dec. 24 2004,06:35
I agree. This is amazing, and just what people in recent days have been ASKING (no wait, BEGGING, for) So...lets hope it continues... I hope after Christmas I get the chance to give it a spin =)
Posted by AwPhuch on Dec. 24 2004,07:06
So let me get this straight this will give you the ability to not only have the embedded version (running windows, with dsl inside), but if reboot it will run as the "LiveCD"?

That would be too cool...talk about instant versatility..it would then double the usefullness of DSL!

You guys absolutely ROCK!

Brian
AwPhuch
Posted by cbagger01 on Dec. 24 2004,17:17
It's even better than that:

Due to the autorun file inside the Live/Embedded CD disk, if you insert your DSL L/E CD in a computer running MSWindows, it will automatically start up DSL from inside a qemu window.

Just add a short readme file to explain the important stuff (CTL-ALT to exit the window, CTL-ALT-F to go fullscreen, press the ALT or TAB key after re-entering the qemu window if the keyboard is "stuck", etc) and anyone with a newer computer can experience DSL out-of-the box.

And yes, if you reboot with the CD inside the drive bay it will bootup into DSL natively like a normal livecd.
Posted by Bryan (Fordi) on Dec. 24 2004,19:21
Oh, I haven't updated the iso from my latest changes, which include:
-The C program to load from autorun
-QEMU 0.6.1 - patched, stripped, statically linked and compressed
-shortened ISO filesystem (everything in isolinux rather than boot/isolinux, qemu moved inside of isolinux to save another 4k)
-consolidated f2 and f3 (saves 2048 bytes)
-index.html is there, but autorun has been modded to include the contents of the index.  Reason: I'm waiting for a response (from John or Robert) as to whether I'm violating GPL by moving the authors' page to a different format.
-reduced minirt24.gz - now only 746k
-linuxrc with network and /dev/hda mounting.  Discovered QEMU has naitive ro cloop support (IE: insmod cloop.o is not necessary in linuxrc, nor is the attempt to mount /dev/hda under cloop.)
-Total iso size=50.006M (ie: <6k over 50M disc.  Should be good enough for overburn.)

Yes!  Squooging is done!

Thing is, I'm not at home, nor do I have better-than-dial-up at the moment (I'm at my g/f's house).  Thus, I shall not be updating the iso until at least monday.
Posted by ypx on Jan. 03 2005,15:36
Wow, what a great idea... it deserve to be added as a choice in your mydsl maker, but i guess that's allready planned :;):
Powered by Ikonboard 3.1.2a
Ikonboard © 2001 Jarvis Entertainment Group, Inc.