2005.09.27 01:05 "Re: [Tiff] Additional Lossless Compression Schemes", by Ron

On Fri, Sep 23, 2005 at 05:11:57PM -0400, Frank Warmerdam wrote:

I was wondering if there is much interest in adding more lossless compression schemes in libtiff.

In addition to Packbits (pretty crappy) we have LZW and the more recent Deflate. My understanding is that Deflate is usually better than the old LZW though not as widely supported. Also, that Deflate is essentially the "zip" compression type.

Since then I have seen "gzip" on Unix, but I am not clear if that is just deflate or not.

For reference:

RFC1951 - zip
RFC1952 - gzip

They use the same compression, but are mostly incompatible in other ways. A tar.gz is more like a .zip (but also not the same).

Then came bzip2 which seem 5-10% better in that gzip on many datasets.

There is a clear time/space tradeoff with this one. It is quite reliably better on size, but your cpu will work noticeably harder for longer both to compress and restore.

So "better" depends on where your bottleneck is, but having the choice of .gz or .bz2 is useful in any large data situation IMO. This would be a good option if LZMA is not clearly better.

Now looking around a bit I see LZMA which I have heard good things about.

   http://www.7-zip.org/sdk.html

Are there any opinions on possibly incorporating LZMA as an additional compression type?

This seems to pop into the limelight every few years, with people saying good things about it. Yet I've still never needed an implementation to unpack something that I downloaded from somewhere. Read into that what you will ;-)

Personally, if libtiff has good generic support for plugging in alternate compression schemes on demand, I don't really mind which are actually implemented at this stage. So long as its easy to add the ones that people really want (ie. enough to submit a patch for cvs now), or the one I really need to get at some wacko non-standard format that one of my clients apps has used... which we may or may not want to ever publish.

I think we probably will want something "better" as the size and number of images that people manipulate grows, so this is good to think about. Otherwise a shop somewhere will surely implement their own brand of better, perhaps without that vital step...

best,

Ron