Boot sorting the KNOPPIX image

Forum: DSL Ideas and Suggestions
Topic: Boot sorting the KNOPPIX image
started by: Fordi

Posted by Fordi on Mar. 17 2005,07:10

I've been (scatterbrainedly) working on a new linux distro based on DSL and Knoppix ideas, but I've found something that would definately work for DSL.

In my search for ways to optimize performance from a LiveCD, I developed a small mod to QEMU.

Essentially, it logs the sector-level reads from all emulated storage devices.

Code Sample

block.c
//Just after the #includes:
FILE* AccessLog;
//In raw_open:
int64_t size;
char logfile[1024];
sprintf(&logfile[0],"%s.log",filename);
AccessLog=fopen(&logfile[0],"wb");
fd = open(filename, O_RDWR | O_BINARY | O_LARGEFILE);
// raw_read:
int ret;
fprintf(AccessLog,"Disk read at sector %x\n", sector_num/4);
lseek(s->fd, sector_num * 512, SEEK_SET);
//finally, last line of raw_close:
fclose(AccessLog);

Case in point:
extract the DSL "KNOPPIX" from the DSL iso, along with linux24 and minirt24.gz

extract the internal ISO from KNOPPIX using cloop_extract_fs
extract the files within the ISO to a temporary directory.

start qemu with the following command line:
qemu -cdrom knoppix.iso -kernel linux24 -initrd minirt24.gz

Boot fully into X, and close QEMU. You'll have an knoppix.iso.log that contains the sector-level reads of the CD. rename this to access.log.
type "isoinfo -l -i knoppix.iso > isodata.txt"

put the following PHP script in a file and run it:

Code Sample

<?php
function isogettoc($isofile) {
$tocdata=file($isofile);
echo sizeof($tocdata)." entries found...";
$listmat="/Directory listing of ([^\s]*)/i";
$cwd="/";
$ret=Array();
for ($i=0; $i<sizeof($tocdata); $i++) {
$line=$tocdata[$i];
if (strtolower(substr($line,0,9))=="directory") {
$cwd=trim(substr($line,21));
continue;
}
if (strlen($line)>10) {
$size=trim(substr($line,25,15));
$loc=trim(substr($line,54,7));
$name=trim(substr($line,67,12));
if (($name!=".")&&($name!=".."))
$ret[$loc]=$cwd."$name";
}
}
echo "\n";
$ret[0]="!preboot";
return $ret;
}
function getOrder($access, $iso) {
$accessdata=file("access.log");
$id=isogettoc($iso);
$ct=0;
for ($i=0; $i<sizeof($accessdata); $i++) {
$a=$accessdata[$i];
if (strtolower(substr($a,5,4))=="read") {
$sector=hexdec(trim(substr($a,20)))*1;
$fd="";
$s=$sector;
while ($fd=="") {
if (!empty($id[$s]))
$fd=$id[$s];
else
$s--;
}
if ($fd!="") {
if ($fd=="!preboot") continue;
if (empty($GotitArray[$fd])) {
$GotitArray[$fd]=1;
echo str_replace("/", "\\", $fd." ".(9999-$ct)."\n");
$ct++;
}
}
}
}
}
$ret=getOrder("access.log", "isodata.txt");
?>

If you've done everything right, php should start spitting out the optimal order for the files on the CD. Run it again, directing it into a file called "sortie.txt".
rebuild the ISO:
mkisofs -l -r -no-pad -hide-rr-moved -sort sortie.txt -o knoppix_optimized.iso knoppix_contents

and recompress with cloop_create_compressed_fs. Replace the KNOPPIX in your DSL ISO with this file. You now have a mostly optimized boot for DSL.

Mostly. When you go to make your shiny new ISO, you should have the following boot order:

Code Sample

\BOOT\ISOLINUX\BOOT.CAT 9998
\BOOT\ISOLINUX\ISOLINUX.BIN 9997
\BOOT 9996
\BOOT\ISOLINUX 9995
\BOOT\ISOLINUX\ISOLINUX.CFG 9994
\BOOT\ISOLINUX\BOOT.MSG 9993
\BOOT\ISOLINUX\LOGO.16 9992
\BOOT\ISOLINUX\LINUX24 9991
\BOOT\ISOLINUX\MINIRT24.GZ 9990
\KNOPPIX 9989
\KNOPPIX\KNOPPIX 9988

You'll note that boot.cat and isolinux.bin are before their directories. This is because they are directly referred to by the El Torito data. Thus, they don't need to have the folders defined before them.

This order will ensure that any myDSL stuff (in the rot folder) won't slow down access of the Important stuff. That should optimize the pre-kernel stuff so that getting to the linux screen and loading the kernel is nice and speedy.

Posted by lunatec on Mar. 18 2005,19:27

Is there an iso I can download? I love the idea of not needed to reboot.
Paul

Posted by cbagger01 on Mar. 18 2005,23:53

Sombody stop this guy before he figures out a way to launch the nukes from a wristwatch...

This is way outta my league here.

Fordi, it sounds like if you were bored you would probably keep yourself stimulated by rewriting QEMU to include direct access to Win32 hardware devices or something crazy like that.

Nice work, by the way.

Posted by lunatec on Mar. 19 2005,17:11

Slashdot had an srticle early in the week about a knoppix distro you can run on windows < http://www.knoppix.net/forum/viewtopic.php?p=78849 >
It runs slow, but DSL should be much quicker

Posted by gunnix on Mar. 20 2005,09:25

nah, DSL is slow in emulation under windows as well. But what can you expect..?

Posted by protocron on Mar. 27 2005,03:08

Fordi,

Wow. Great idea.

I was thinking further on this. And I am by no means a guru like yourself, but here are some ideas I've had.

1) I think that this would work. cloop uses sequential reads per this article:
< http://www.knoppix.net/forum....tart=40 >
and other stuff I've read.

2) What about using SquashFS? While I am no big supportor of the SquashFS filesystem, it does appear (at least in theory) to have major perfomance advantages.

Posted by protocron on Mar. 27 2005,03:15

Sorry I meant to add more here but hit the wrong button.

3) Kernel and modules, because of the specifc hardware that the qemu envirnment provides, I would think that the kernel and modules could be compiled with only the support for those devices. (I don't know is it?)