2000.03.14 01:02 "Writing subfiles out of order", by Rick LaMont

2000.03.14 03:01 "Re: Writing subfiles out of order", by Frank Warmerdam

I have an application that uses libtiff to create a file with multiple subfiles. Each subfile is tiled and may be compressed. Currently, the file is written in strictly sequential order:

    for (subfile = 0 to n) {
        set-all-TIFF-fields
        for (tile = upper-left to lower-right)
            TIFFWriteTile(data(subfile, tile))
        TIFFWriteDirectory
    }

This solution works but is slow because it has to make n passes over the input file (not shown in pseudocode). Buffering the entire input file in memory is not an option because it might be on the order of a gigabyte. It would be preferable to renest the loops to something like this:

    for (subfile = 0 to n) {
        set-all-TIFF-fields
        TIFFWriteDirectory
    }
    for (tile = upper-left to lower-right)
        for (subfile = 0 to n) {
            TIFFSetDirectory(subfile)
            TIFFWriteTile(data(subfile, tile))
        }

Is it possible to write tiles across all subfiles like this or am I just dreaming? My first attempt failed on the TIFFWriteDirectory because the tile offsets weren't initialized. If necessary, I wouldn't mind creating all black tiles and then going back to fill in the framework. My fear is that these placeholders would compress better than the actual data, not leaving enough room.

I remember reading somewhere that TIFFWriteTile *usually* appends to the end of the output file. Under what circumstances can it overwrite the middle?

Rick,

This sounds like the same issues I ran into trying to implement overview generation within a TIFF file. I wanted to be able to read a strip (or row of tiles) from one directory, and then write out strips or tiles to one or more overview directories in the same file. Then repeat.

With the 3.5.4 libtiff it wasn't possible to read from one directory in the same session that I was appending a new directory. Furthermore, all tiles (or strips) for a directory had to be written before the directory could be written out.

I made some changes so that a directory could be commited without any tiles (or strips) and to have these populated later in the session.

If you want to do this, I would suggest you check the current code out of CVS, and look at how libtiff/contrib/addtiffo handles things. In particular I have exposed TIFFWriteCheck() which forces makes a directory writable (via TIFFWriteDirectory()) even though it doesn't have tile/strip offsets yet.

Also I changed various low level code to allow reading and writing of directories when a file is opened for update ("r+") access.

These changes are "experimental" so if be wary if you want to work based on them. I would be happy to answer any questions, and open to patches as need be.

Best regards,

---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerda@home.com
light and sound - activate the windows | http://members.home.com/warmerda
and watch the world go round - Rush    | Geospatial Programmer for Rent