LibTiff Mailing List

TIFF and LibTiff Mailing List Archive
April 2010

Previous Thread
Next Thread

Previous by Thread
Next by Thread

Previous by Date
Next by Date

The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems

2010.04.02 22:46 "Re: Tiff ReadScanline", by Richard Nolde

Regarding the topic listed below, TIFFReadScanline.

In my first implementation of tiffcrop, I used the TIFFReadScanline 
interface exclusively, which caused me problems with compression schemes 
that worked on multiple scan lines. The current verision uses reads a 
Strip or Tile at a time, which solves that problem but would be 
problematic for files such as Frank is describing. While I don't have 
the time or expertise to re-implement the current TIFFReadScanline 
interface, it seems worth considering the following:

1) Could there be a new libtiff function that reads one or more scan 
lines, the mimimum being determined by the compression scheme, without 
buffering the whole strip/tile. eg TIFFReadScanlinesUnbufferred.

The arguments to the new library function would include the number of 
scanlines to read, the number of the desired scanline, and pointers to 
variables that could be used to track values returned by each call which 
would allow TIFFSeek to read raw data from the appropriate point in the 
file each time the new function is called without having to read from 
the start every time. For this to work, the decompression routines would 
all have to be called in such a way as to satisfy their minimum 
payloads, starting points, etc, which may not be possible for all 
combinations or may be impossibly complex to determine. Perhaps starting 
with the TIFFReadRaw* functions and modifying them would be a starting 
point.

Richard Nolde
> Today's Topics:
>
>     1. TIFFReadScanline and large compressed one-strip files
>        (Frank Warmerdam)
>
> Message: 1
> Date: Thu, 01 Apr 2010 17:12:53 -0400
> From: Frank Warmerdam<warmerdam@pobox.com>
> Subject: [Tiff] TIFFReadScanline and large compressed one-strip files
> To: TIFF mailing list<tiff@lists.maptools.org>
> Message-ID:<4BB50C55.3070109@pobox.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Folks,
>
> I have an LZW compressed TIFF file of 290MB or so with the whole image
> in one strip.  Reading the file with the TIFFReadScanline() function causes
> the entire 280MB of compressed data to be loaded into memory at the point
> the first scanline is read.
>
> My client is finding that in some cases on 32bit systems there isn't 290MB
> of contiguous memory available and would like a way avoiding prereading the
> whole strip.
>
> Does anyone have any suggestions for this?
>
> I see that currently TIFFReadScanline() calls TIFFSeek() which loads the
> appropriate data into the working buffer.  I'm wondering if I could load
> just part of the data into the raw buffer based on knowledge of what
> scanline I want.  Perhaps loading enough raw data that I'm guaranteed to
> satisfy all the raw data needed for one scanline (say 2*uncompressed size
> plus 500 bytes).
>
> I'm concerned that I would need to complicate the management of the currently
> loaded raw data considerably to handle this sort of rolling buffering.  Does
> it seem ill advised to even try?
>
> Best regards,
>