High-Speed algorithms added to Pcompress

Pcompress is currently in a first 0.5 Alpha release with a bugfix 0.6 coming up soon.

I have added a couple of very fast compression algorithms into the mix in Pcompress. The first and probably the fastest one is LZFX (http://code.google.com/p/lzfx/). It is based on LZF with some optimizations. The original implementation had fixed size hash on the stack defined at compiled time. I made it dynamic to support 5 compression levels with increasing hash size.

The other one is LZ4 (http://code.google.com/p/lz4/) which is finding mention in many forums. LZ4 also has compile-time defined parameters which I have not yet gone about tweaking. However I have still added 3 compression levels based on which LZ4 routines are being called. LZ4 provides 2 compression variants. A very fast LZ4_compress() and a slower high-compression LZ4_compressHC(). In pcompress at present level one is plain LZ4_compress(). Level 2 is a multi-pass version where LZ4_compress() and LZ4_compressHC() are called in sequence to compress twice. Interestingly enough this yields almost 90% of the compression ratio achieved by LZ4_compressHC() at a fraction of the time taken (see also http://fastcompression.blogspot.in/2012/07/log-file-compression.html). Level 3 is of course LZ4_compressHC() alone.

While reading http://fastcompression.blogspot.in/ I noticed these two posts: http://fastcompression.blogspot.in/2012/05/members-properties.html and http://fastcompression.blogspot.in/2012/05/useful-compressed-streaming-properties.html. Very interesting observations and Pcompress in fact implements majority of those features. One very useful feature, not yet implemented in Pcompress, is the concept of Authorized Members which allows a compressed file chunk to be something other than actual compressed data. This for example can be used to hold things like digital signatures and certificates when implementing security features.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s