Pcompress goes to 0.9.1

Now that encryption and message verification/authentication is done pushed out a couple of releases. Here’s some of the salient features that have come in after the initial implementation of AES encryption:

  1. Hash-based Message Authentication (HMAC)
  2. Optimized SHA256 for AMD platforms
  3. Fix build on Debian platforms (Linux Mint)
  4. Portable printf format string for unsigned int64 value to avoid gcc 4.6 warnings.

The HMAC functionality is available for all chunk digest algorithms namely SKEIN 256/512, SHA 256/512. For CRC64, SHA256 HMAC is used. The HMAC uses the same encryption key. However question now arises which items to verify using the HMAC. Ideally it should be everything in the file to ensure tampering is prevented. Now a Pcompress data file has a file header and every chunk in the file have their chunk headers. Every chunk includes a normal cryptographic disgest which can be SKEIN or SHA-2 to verify data integrity.

At this point applying a HMAC which is the same digest strengthened using an encryption key to the entire chunk data is wasteful. So what I have done is to apply the HMAC to the file header and the chunk headers. So file header is verified and chunk header is verified. The chunk header includes the normal chunk data digest which also gets verified by the HMAC. It is then used to verify integrity of the chunk data. This is good for performance since computing a HMAC is much more expensive than computing a standard cryptographic digest. Apart from these I added a few data validation checks during decompression to ensure that things like chunk sizes mentioned are within bounds.

With respect to the SHA256 optimizations, I found that Intel’s optimized implementations based on SSE 4.2 and AVX1 work fine on AMD Bulldozer and later platforms (Piledriver). So I have enabled those to be used on AMD with the SSE 4.2 and AVX1 feature detection. Thankfully AMD uses the same bits in ECX to indicate SSE4.2 and AVX1 capabilities.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s