2010.03.02 20:38 "[Tiff] New Commiter", by Frank Warmerdam

2010.04.02 22:46 "Re: [Tiff] Tiff ReadScanline", by Richard Nolde

Regarding the topic listed below, TIFFReadScanline.

In my first implementation of tiffcrop, I used the TIFFReadScanline interface exclusively, which caused me problems with compression schemes that worked on multiple scan lines. The current verision uses reads a Strip or Tile at a time, which solves that problem but would be problematic for files such as Frank is describing. While I don't have the time or expertise to re-implement the current TIFFReadScanline interface, it seems worth considering the following:

1) Could there be a new libtiff function that reads one or more scan lines, the mimimum being determined by the compression scheme, without buffering the whole strip/tile. eg TIFFReadScanlinesUnbufferred.

The arguments to the new library function would include the number of scanlines to read, the number of the desired scanline, and pointers to variables that could be used to track values returned by each call which would allow TIFFSeek to read raw data from the appropriate point in the file each time the new function is called without having to read from the start every time. For this to work, the decompression routines would all have to be called in such a way as to satisfy their minimum payloads, starting points, etc, which may not be possible for all combinations or may be impossibly complex to determine. Perhaps starting with the TIFFReadRaw* functions and modifying them would be a starting point.

Richard Nolde

> Today's Topics:
>
> 1. TIFFReadScanline and large compressed one-strip files
> (Frank Warmerdam)
>
> Message: 1
> Date: Thu, 01 Apr 2010 17:12:53 -0400
> From: Frank Warmerdam<warmerdam@pobox.com>

> Subject: [Tiff] TIFFReadScanline and large compressed one-strip files > To: TIFF mailing list<tiff@lists.maptools.org>

> Message-ID:<4BB50C55.3070109@pobox.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed

I have an LZW compressed TIFF file of 290MB or so with the whole image in one strip. Reading the file with the TIFFReadScanline() function causes the entire 280MB of compressed data to be loaded into memory at the point the first scanline is read.

My client is finding that in some cases on 32bit systems there isn't 290MB of contiguous memory available and would like a way avoiding prereading the whole strip.

Does anyone have any suggestions for this?

I see that currently TIFFReadScanline() calls TIFFSeek() which loads the appropriate data into the working buffer. I'm wondering if I could load just part of the data into the raw buffer based on knowledge of what scanline I want. Perhaps loading enough raw data that I'm guaranteed to satisfy all the raw data needed for one scanline (say 2*uncompressed size plus 500 bytes).

I'm concerned that I would need to complicate the management of the currently loaded raw data considerably to handle this sort of rolling buffering. Does it seem ill advised to even try?