2018.06.29 10:08 "[Tiff] Writing tiles out of order", by Roger Leigh

2018.06.29 11:37 "Re: [Tiff] Writing tiles out of order", by Even Rouault

Just to clarify, this is only time [I/O] expensive when reading and updating the tile offsets+sizes?

Yes

Or is it also wasteful of space by duplicating the whole IFD each time I do an update?

No, there's an optimization in libtiff where it detects that only the offset +size arrays have been modified, and rewrite them in place.

Looking at GDAL, I see you calling TIFFRewriteDirectory in frmts/gtiff/geotiff.cpp.

Only if bNeedsRewrite == true, and in the scenario we discuss, it won't be set (it is only set if some tags, except offset/count are added or modified)

If you're only writing additional tiles, would it be sufficient to call TIFFCheckpointDirectory to update the already written directory in place? Or will this only work before the first call to TIFFWriteDirectory? Or would TIFFWriteDirectory be possible to use here? (Apologies for the confusion, this seems a bit of a non-standard thing to do and it's not covered by the documentation.)

I'm not aware of all the subtelties of the above... GDAL could potentially be a bit clever if it knows that the directory size is not going to grow because of a change, but that's not done currently.

Thanks, I'll investigate this too. I've already wrapped libtiff in a muxtex so should be relatively safe... On the other hand, for large images with multiple planes for Z/T/C, I might run out of file descriptors!

You could have your own I/O layer that would use a single operating system file handle, but simulate at the application level multiple ones.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com