LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
August 2007
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2007.08.22 22:03 "Re: How to handle big single-strip images?", by Leonardo Serni
>i wonder what will happen if one opens an tiff image where all the
>data is in a SINGLE strip (some writers offer this option) and also
>compressed (G4) and someone tries to use TIFFReadScanline with it?
>I would expect that libtiff needs to read the whole strip, because
>it's compressed, into memory. Or is it possible to read compressed
>data by chunks?
It is possible, but it looks like libtiff doesn't do this. If I read
this correctly (I am now browsing 3.8.0 code, tif_read.c):
TiffReadScanLine invokes TIFFSeek
TIFFSeek calculates parent strip (in this case the first)
TIFFSeek sees it hasn't strip data, calls TIFFFillStrip to get it
TIFFFillStrip has an ominous comment:
/*
* Read the specified strip and setup for decoding.
* The data buffer is expanded, as necessary, to
* hold the strip's data.
*/
and, later on, in the case of memory mapped files (but the same
applies for unmapped ones):
/*
* Expand raw data buffer, if needed, to
* hold data strip coming from file
* (perhaps should set upper bound on
* the size of a buffer we'll use?).
*/
Anyway, what happens is that the whole byte count is read, or so it
seems to me:
bytecount = td->td_stripbytecount[strip];
...
if (TIFFReadRawStrip1(tif, strip, (unsigned char *)tif->tif_rawdata,
bytecount, module) != bytecount)
return (0);
So, yes, if you have a 700 meg compressed file, libtiff apparently reads
the whole 700 meg, then decompresses it one row at a time.
This means that a 700 meg compressed, 4GB uncompressed, single stripe image
would not require 4GB or 4.7 GB, no... but yes, it *would* require 700 MB.
Also, whenever you seek from row N to row M with M < N, libtiff appears to
"rewind" the strip and decompress-seek on it again:
} else if (row < tif->tif_row) {
/*
* Moving backwards within the same strip: backup
* to the start and then decode forward (below).
*
* NB: If you're planning on lots of random access within a
* strip, it's better to just read and decode the entire
* strip, and then access the decoded data in a random
fashion.
*/
if (!TIFFStartStrip(tif, strip))
return (0);
}
You can get good performances (memory-wise and compression-wise) using wisely
the RowsPerStrip parameter. It requires application-specific tuning (e.g., if
you care little about squeezing the last bytes of disk space, but do care for
performances, you'll choose RowsPerStrip = 1; I believe that 8 and 16 tend to
get you best performance with JPEG; otherwise the manual hints to keep stripe
size (encoded) around 64K, and I concur, but it depends on your IOSS buffers:
you might find that 1.4K, 128K or 96K give better disk throughput).
But the best choice is by far the one suggested by jcupitt: tiled TIFFs... He
who despises them will end up reinventing them (BTDTGTTS).
Best regards,
Leonardo