2005.03.30 00:18 "[Tiff] RFC: fast 'copy free' tiff decoding", by Ron

Hi,

I'm a little uncertain where to start with this one, since its certainly going to be much more intrusive than anything I've proposed to date and I really haven't been here very long to be suggesting major upheavals, so bear with me and feel free to bring as big a cluebat as you need.

It occurs to me that tiff was largely designed to be trivially implemented in hardware. I'd like to more actively exploit that on platforms where we can and I think we can pick up a lot of other niceties (like generic IFD support) very easily along the way.

The basic idea is very simple. First map a tiff into memory, then cast carefully constructed C structures into the right offsets of that memory space to be able to access the data directly.

I have a proof of concept in C++ that simply mmap's the entire file in one go, then walks the offset chain casting structures into place to dump the header and the tags in each IFD. Everything is available, but nothing is actually copied (or potentially even paged in on a modern system) until you access the members of the structures (which correspond to the elements in the now decomposed tiff). No attempt is made to interpret anything beyond the most basic structure so any tiff, baseline or private spec, may be accessed this way. Conformance with those standards and any needed codecs etc. can be cleanly layered on top. It can also be made to degrade nicely if the tiff cannot be mmap'ed, or if only one tile at at time can be etc.

The question now, is where to flesh that out. I'm not really interested in forking a 'competitor' to libtiff (for all the usual long term time budget reasons), but this is a pretty big tangent to throw you people on if you are busy stabilising what you already have. OTOH it seems like a very solid core to build some of the other things we want upon, we get generic tiff for free, and simply need to layer baseline tiff and all the other stuff on top of that.

Is anybody else here interested in following that chain of thought? There are plenty of other real details I can flesh this out with for people who are. Would it be reasonable to run an experimental branch of the libtiff cvs to shake some of these ideas out? I could whip up something more C friendly along these lines to kick start that off if there was sufficient interest.

best,
Ron