iB::Topic::Extensions can be recompressed with `advdef`

	Damn Small Linux :: Damn Small Linux Board The DSL Forums

» Welcome Guest
[ Log In :: Register ]

Damn Small Linux Board » Damn Small Linux » DSL Ideas and Suggestions » Extensions can be recompressed with `advdef`

		Mini-ITX Boards Sale, Fanless BareBones Mini-ITX, Bootable 1G DSL USBs, 533MHz Fanless PC <-- SALE $200 each!
		Get The Official Damn Small Linux Book. DSL Market , Great VPS hosting provided by Tektonic

Pages: (5) </ [1] 2 3 4 5 >/

[ Track this topic :: Email this topic :: Print this topic ]

Topic: Extensions can be recompressed with `advdef`, Reduce size of '.dsl' & '.tar.gz' files

< Next Oldest | Next Newest >

stupid_idiot Offline

Group: Members
Posts: 344
Joined: Oct. 2006

Posted: July 10 2007,14:56

To all extension makers:
I would like to bring a great program to your attention - it is `advdef` which is part of the AdvanceCOMP project.
The source code can be downloaded here.

`advdef` recompresses zlib-compressed files with a 100%-zlib-compatible 7zip-originated 'Deflate' algorithm. The size of the reduced file is usually between 90-100% of the original. The size reductions are enough to suggest using `advdef` always. All '.dsl' and '.tar.gz' extensions are suitable. '.uci' extensions are not - my layman's explanation is that '.uci' files are iso images with compressed sectors rather than one whole compressed file. This does not make '.uci' inferior - `create_compressed_fs` already uses 7zip's LZMA in addition to 7zip's Deflate for optimal efficiency.

Point of interest:
The latest version of `create_compressed_fs` (2.05) can create slightly smaller '.uci' files than the version in DSL (2.01) because it can choose between different compression levels for each block (zlib levels 1-9 or lzma) based on size - even though lzma is better in most cases, sometimes it loses due to higher overhead compared to zlib. This process is very time-consuming. It is used only if you specify `-L-2`. It is best-used during the final make.
The source code for the latest version can be downloaded
here.
You might want to read the help for `create_compressed_fs`, because the syntax has completely changed between 2.01 and 2.05.

mikshaw

Group: Members
Posts: 4856
Joined: July 2004

Posted: July 10 2007,19:40

Quote

The size of the reduced file is usually between 90-100% of the original

This doesn't sound like much of an advantage, unless there is _no_ extra cost in processing (which also equals time) during decompression. Bzip2 can also create much smaller packages, and is already available in DSL, but is not used because of the dramatic increase in processing needed.

Quote

The latest version of `create_compressed_fs` (2.05) can create slightly smaller '.uci' files than the version in DSL

I recall reading something about the newer version of cloop being made specifically for kernel 2.6. Of course I could be completely wrong about this, and for all I know the cloop module version may have very little influence on the create_compressed_fs version. I do know that at one time I tried to install a recent version of cloop in another 2.4-kernel distro. The cloop module failed, but create_compressed_fs still worked. I thought it was odd.
As with advdef, if the extra time needed extends to decompression in addition to compression, it would probably be faced with some resistance from some people. Otherwise, it sounds like something worth looking into.

--------------
http://www.tldp.org/LDP/intro-linux/html/index.html

WDef

Group: Members
Posts: 798
Joined: Sep. 2005

Posted: July 11 2007,13:30

On a related nore: I tried without success to get anyone here interested in lzop a few years back. Much faster (eg 9X) than gzip or bz2 and only slightly lower compression ratio. A huge improvement for backup times.

^thehatsrule^ Offline

Group: Members
Posts: 3275
Joined: July 2006

Posted: July 11 2007,13:41

I think one of the major points is memory consumption... since gzip really takes up very little. This link may be interesting for this topic: http://tukaani.org/lzma/benchmarks

roberts

Group: Members
Posts: 4983
Joined: Oct. 2003

Posted: July 11 2007,15:01

Reducing size is not of paramount interest when compared to decompression time and memory used. This is especially true given DSL's design philosphy of primarily being a nomadic live CD/frugal, where decompression of MyDSL extensions occurs many times and therefore time and memory required to perform such is paramount.

21 replies since July 10 2007,14:56

< Next Oldest | Next Newest >

[ Track this topic :: Email this topic :: Print this topic ]

Pages: (5) </ [1] 2 3 4 5 >/

Damn Small Linux Board » Damn Small Linux » DSL Ideas and Suggestions » Extensions can be recompressed with `advdef`