Search Members Help

» Welcome Guest
[ Log In :: Register ]

Mini-ITX Boards Sale, Fanless BareBones Mini-ITX, Bootable 1G DSL USBs, 533MHz Fanless PC <-- SALE $200 each!
Get The Official Damn Small Linux Book. DSL Market , Great VPS hosting provided by Tektonic
Pages: (5) </ [1] 2 3 4 5 >/

[ Track this topic :: Email this topic :: Print this topic ]

reply to topic new topic new poll
Topic: Extensions can be recompressed with `advdef`, Reduce size of '.dsl' & '.tar.gz' files< Next Oldest | Next Newest >
stupid_idiot Offline





Group: Members
Posts: 344
Joined: Oct. 2006
Posted: July 10 2007,14:56 QUOTE

To all extension makers:
I would like to bring a great program to your attention - it is `advdef` which is part of the AdvanceCOMP project.
The source code can be downloaded here.

`advdef` recompresses zlib-compressed files with a 100%-zlib-compatible 7zip-originated 'Deflate' algorithm. The size of the reduced file is usually between 90-100% of the original. The size reductions are enough to suggest using `advdef` always. All '.dsl' and '.tar.gz' extensions are suitable. '.uci' extensions are not - my layman's explanation is that '.uci' files are iso images with compressed sectors rather than one whole compressed file. This does not make '.uci' inferior - `create_compressed_fs` already uses 7zip's LZMA in addition to 7zip's Deflate for optimal efficiency.

Point of interest:
The latest version of `create_compressed_fs` (2.05) can create slightly smaller '.uci' files than the version in DSL (2.01) because it can choose between different compression levels for each block (zlib levels 1-9 or lzma) based on size - even though lzma is better in most cases, sometimes it loses due to higher overhead compared to zlib. This process is very time-consuming. It is used only if you specify `-L-2`. It is best-used during the final make.
The source code for the latest version can be downloaded
here.
You might want to read the help for `create_compressed_fs`, because the syntax has completely changed between 2.01 and 2.05.
Back to top
Profile PM 
mikshaw Offline





Group: Members
Posts: 4856
Joined: July 2004
Posted: July 10 2007,19:40 QUOTE

Quote
The size of the reduced file is usually between 90-100% of the original
This doesn't sound like much of an advantage, unless there is _no_ extra cost in processing (which also equals time) during decompression. Bzip2 can also create much smaller packages, and is already available in DSL, but is not used because of the dramatic increase in processing needed.

Quote
The latest version of `create_compressed_fs` (2.05) can create slightly smaller '.uci' files than the version in DSL
I recall reading something about the newer version of cloop being made specifically for kernel 2.6. Of course I could be completely wrong about this, and for all I know the cloop module version may have very little influence on the create_compressed_fs version. I do know that at one time I tried to install a recent version of cloop in another 2.4-kernel distro. The cloop module failed, but create_compressed_fs still worked. I thought it was odd.
As with advdef, if the extra time needed extends to decompression in addition to compression, it would probably be faced with some resistance from some people. Otherwise, it sounds like something worth looking into.


--------------
http://www.tldp.org/LDP/intro-linux/html/index.html
Back to top
Profile PM WEB 
WDef Offline





Group: Members
Posts: 798
Joined: Sep. 2005
Posted: July 11 2007,13:30 QUOTE

On a related nore: I tried without success to get anyone here interested in lzop a few years back.  Much faster (eg 9X) than gzip or bz2 and only slightly lower compression ratio.  A huge improvement for backup times.
Back to top
Profile PM 
^thehatsrule^ Offline





Group: Members
Posts: 3275
Joined: July 2006
Posted: July 11 2007,13:41 QUOTE

I think one of the major points is memory consumption... since gzip really takes up very little.  This link may be interesting for this topic: http://tukaani.org/lzma/benchmarks
Back to top
Profile PM 
roberts Offline





Group: Members
Posts: 4983
Joined: Oct. 2003
Posted: July 11 2007,15:01 QUOTE

Reducing size is not of paramount interest when compared to decompression time and memory used. This is especially true given DSL's design philosphy of primarily being a nomadic live CD/frugal, where decompression of MyDSL extensions occurs many times and therefore time and memory required to perform such is paramount.
Back to top
Profile PM WEB 
21 replies since July 10 2007,14:56 < Next Oldest | Next Newest >

[ Track this topic :: Email this topic :: Print this topic ]

Pages: (5) </ [1] 2 3 4 5 >/
reply to topic new topic new poll
Quick Reply: Extensions can be recompressed with `advdef`

Do you wish to enable your signature for this post?
Do you wish to enable emoticons for this post?
Track this topic
View All Emoticons
View iB Code