Interest in a lower compression iso optimized for USB booting?

[meo]

Hi grindstone! Right now I only have DSL installed in VMware Workstation Pro where it never has been any trouble with the sound. I'm actually listening on a playlist on youtube as I write this. I'm reluctant to install Qemu on this machine again. Made a fresh install of Windows 11 today to get rid of some garbage that had been collected. So now I think at least twice before installing anything directly in windows. I only get 1280x768 in resolution on the installed DSL. I opted for the alpha2 and it works fine. Checked the Control Centre in DSL and the highest resolution awailable is 1280x768 so it can't be changed that way either. In some other distros it was just to choose the resolution after installation. I'm using DSL as my preferred work environment since the tools I use works well. Thanks again grindstone!

// meo

[BG405]

It's well worth giving VirtualBox a try. The maximum resolution which works for me is 1920x1440 (maybe not enough allocated video RAM for 2560x1600?) and the sound works fine, at least as far as the sound test; I haven't tried playing other media as of the time I'm posting this.

This is in scaled mode, obviously. VirtualBox version 7.0.14 on a Manjaro host machine.

   

[meo]

I have tried it several times but it doesn't work so well with my wireless keyboard and mouse. Have tried not to leave any stone unturned trying out DSL-2024.

[dbohdan]

John, have you considered using Zstandard instead of lz4? It has pretty high compression ratios combined with much faster decompression than XZ. The Linux kernel has supported Zstandard-compressed SquashFS since version 4.14. The main caveat compared to XZ is that similar compression ratios require more compression time. I have just tried recompressing linuxfs from dsl-2024.rc1.iso. The Zstandard version ended up being 751 MiB. The XZ original is 632 MiB, and the lz4 version from dsl-2024.rc1.lz4.iso is 1.1 GiB.

The command I used to create the Zstandard version was

Code:

mksquashfs squashfs-root/ linuxfs.zst -comp zstd -progress -Xcompression-level 22

[John]

Thanks Dbohdan. I have experimented with Zstandard compression too. It does seem advantageous, but as DSL sits right now it is just slightly too big to get onto a single CD using it. For USBs and VMs I think lz4 makes sense because it is still faster to decompress than Zstandard and in those applications the increased size isn't really important. If I manage to trim down DSL a bit more and get the room inside 700mb it would be nice to offer Zstandard compression instead of xz.

[dbohdan]

You're welcome. Right, it doesn't look possible to replace XZ with Zstandard for the main ISO and stay within the 700 MiB size limit. However, I think it is possible to replace lz4 with Zstandard for the low-compression ISO and preserve the USB/VM boot speed.

I have performed a rough benchmark to see if this is the case. For the benchmark, I used a VM (VirtualBox 7 on a Linux host) instead of a USB flash drive so USB transfer speed didn't cap the decompression speed. I put the ISOs on an NVMe drive and created a variant of the RC1 ISO with Zstandard SquashFS. To measure the approximate boot time, I started my phone's stopwatch when I reset the VM and stopped it when Conky appeared on the screen. When the boot menu came up, I quickly pressed Enter to skip it. Like I said, rough! :-)

Here are the results I got (in seconds):

• dsl-2024.rc1.iso (648M): 36, 37, 36
  
• dsl-2024.rc1.lz4.iso (1116M): 24, 23, 25
  
• dsl-2024.rc1.zst.iso (767M): 25, 25, 24
  

It looks like lz4 and Zstandard are both so fast that other factors determine the boot time. The higher maximum possible decompression speed of lz4 doesn't seem to make a difference in this situation.

[John]

Nice testing, thank you! You are right, there doesn't seem to be much notable difference in the zst and lz4 boot times. Food for thought for sure! If I could trim away more space it would be really great to squeeze a Zstandard iso into 700MB. That would definitely improve the live CD experience.

[BG405]

I have a thought re. the potential improved experience for Live CDs .. would the difference actually be that significant if booted from an old CD-ROM unit?

I'm thinking the read-out time would still be ~2½ minutes even with a fast unit averaging something like 30x read speed over the whole disc .. which I think is rather pushing the limits of the technology.

[John]

Wow, this is a great question. Let's assume it's like an old Core 2 Duo E6420 @ about 2GHz with 2GB RAM, or, something even older like a P4 with 1G of ram. I'd have to think in those cases the overhead of decompressing is the biggest factor for booting from CD than the speed of the CD Rom drive. I really would test this out though.

[dbohdan]

Although I agree benchmarking is the way to go, we can estimate. Suppose it's a 52× CD-ROM drive. Faster drives are, as far as I know, rare. That's 150 KB/s × 52 ≈ 7.6 MB/s. lz4 and Zstandard should not be a problem with this data rate, which leaves XZ. A 7-Zip issue from 2007 has a user reporting LZMA decompression at 9 MB/s in single-thread mode on a 2.4 GHz Pentium 4. On one hand, GCC is better now than it was in 2007 and should generate somewhat more efficient code. LZMA decompressors may be more optimized, too. On the other hand, a machine booting from a CD-ROM is not going to be able to devote all of its resources to decompression. If we assume a 10 MB/s decompression rate, a single core, and half of the CPU time going to decompression, we get 5 MB/s. So I'd guess that the faster Pentium 4 CPUs may bottleneck XZ decompression off a fast CD-ROM, but not too badly. Core 2 Duo CPUs shouldn't bottleneck it at all.