2012.11.27 18:09 "Re: [Tiff] help writing thumbnails to TIFF file?", by Daniel McCoy

PS: If we're interested in distant history, I worked with Sam Leffler at Pixar in 1986 and he and I wrote the Pixar Software Standards document together.

Hi Paul,

there was also some dessert-food related work back then as well, if I remember right. Sweet piece of research.

I inherited Sam's desk and Sun 3/50 when he left Pixar for SGI, as well as support of his TIFF library for the Pixar rendering group. Sam and I are the ones who badgered the tiff committee into adding alpha channel support to the spec. Bunch of print people on the committee at the time who wondered why you would ever need more than one bit of alpha. But I digress...

I think Darwyn Peachey did the first hack making partial tiff's readable internally at Pixar.

I re-packed it into something submittable to Sam's library so we didn't have to do proprietary mods to every release. That became the CheckpointDirectory function.

As I said, it wasn't *intended* to support control over placement, but I acknowledge that it might be able to be used for that. Though, you will be fighting some implicit assumptions made by the tiff directory writing code.

There is kind of an implicit assumption in the directory writing code (not Checkpoint itself) that a directory will be written as "done" before switching images. So, but it's not obvious what breaking that assumption will mess up.

I wonder if there is some in-memory state caused by setting some tags which is not correctly restored after you switch directories. (Particularly thinking of things like compression, which you are changing.) There should be no ill-effect to re-setting the tags to the same values already saved, so...

You might try re-calling your "set_tags()" function with the proper values after every call to SetDirectory()

I was also wondering if there is something TIFFFlush does that would make it necessary between directory switches. So a really wild guess would be to see what it does with a TIFFFlush between the directory switches.

From Paul Heckbert <ph@cs.cmu.edu>, Mon, Nov 26, 2012 at 09:10:05PM -0500:

We have several opinions so far:

Daniel McCoy said he implemented TIFFCheckpointDirectory years ago to allow partially-written TIFF files to be read. Olivier Paquet and Edward Lam said that they use this function for that purpose, and it works for them. Edward thinks that most other people should not use TIFFCheckpointDirectory.

Joris Van Damme referred to the function as a hack that served a mysterious function for its original developers. He added that it would be unlikely to suit me or anyone else but he didn't explain why it wouldn't, other than to compare TIFFCheckpointDirectory to Stonehenge. He argued that people shouldn't care about the order of data in their TIFF files.

I side with those that are saying that TIFFCheckpointDirectory is useful and (at least sometimes) works properly. Without TIFFCheckpointDirectory, the directory would always appear at the end of a TIFF file, after the pixels, and reading partial TIFF files would be impossible. Some control of data order in the file IS useful, namely: does the directory appear before or after the pixels?

Other situations where it's useful to have the directory first: if a file is being streamed in, you might want to inspect the directory and cancel the streaming. And for very large files (I currently work with TIFFs up to 4 gigabytes, and soon, larger) it's nice to maximize performance and minimize seek times by putting directories, thumbnails, and other header-like information at the beginning of a file. That's also the order in which most of the rest of the world writes their TIFF files.

PS: If we're interested in distant history, I worked with Sam Leffler at Pixar in 1986 and he and I wrote the Pixar Software Standards document together.

Returning to my code:

I wonder if others have attempted to write two directories (IFDs) using TIFFCheckpointDirectory. Anyone got pointers to code that does this? Am I using TIFFSetDirectory properly?

My current hypothesis: TIFFCheckpointDirectory works fine if you're writing only one directory, but it's buggy when there are multiple directories.

I tested further and it seems that my program, when run with "thumb - nocheckpoint", yields a TIFF that "exiftool -hexdump" can parse properly even though the directories follow the pixels. I don't know if other TIFF-reading software can parse it properly, however.