LibTiff Mailing List

Boris Filipoviè wrote:
> rastersize=2069060000

That's quite a lot.

Are you using the RGBA interface? If so, that explains your need for a complete
raster.

>   Rows/Strip: 1

This particular item here indicates that even with LibTiff, it should be
possible to cut down memory needs very substantially.

You need to build yourself a 'scanline stream'. One such design involves a
linked list of 'scanline manipulators'. You get scanlines by asking ('pulling')
the last manipulator in the list. This one pulls the previous one as needed,
etc.

The list you need to set up is the following
- interface with LibTiff, for example using TIFFReadScanline
- x resampler. This step's pull handler first pulls a 1bit per pixel scanline
from the previous step. Next, it applies a pre-calculated resampling map to it,
transforming the 118228 singlebit pixels in thumbnail width 8bit grayscale
pixels if you wish
- y resampler. This step's pull handler pulls a lot of scanlines from the
previous step. However, it does not need to store everyone of them. Instead, it
multiplies everyone with a value of a (pre-calculated if you wish) map, and adds
it to a temp buffer. The temp buffer might be 16bits per pixel. Upon having
pulled and processed all source lines that go into a destination line, it may
apply a final convertion to a result 8bit per pixel format.
- interface with your result thumbnail, probably just merely pulling a scanline
from previous step and copying it to the destination thumbnail buffer.

This cuts memory needs dramatically, and speeds up things even if you have the
kind of technology that can support memory blocks of 2 gig in size. Let's do a
quick calculation, assuming the most stupid of ways to design this and lots of
unnecessary buffering. I furthermore assume a thumbnail size of 200x200.

- interface with LibTiff:
    one 1bit per pixel scanline of 118228 pixels = 14800 bytes
- x resampler:
    one 1bit per pixel scanline of 118228 pixels = 14800 bytes
    one 8bit per pixel scanline of 200 pixels = 200 bytes
    a pre-calculated resampling map of 118228 weights, say 16bits per weight,
       all depending on implementation: 236456 bytes
- y resampler:
    one 8bit per pixel scanline of 200 pixels = 200 bytes
    one 16bit per pixel temp buffer of 200 pixels = 400 bytes
    one 8bit per pixel scanline of 200 pixels = 200 bytes
    a pre-calculated resampling map of 140000 weights, say 16bits per weight,
       all depending on implementation: 280000 bytes
- result thumbnail writer
    one 8bit per pixel scanline of 200 pixels = 200 bytes
    one 8bit per pixels result thumbnail memory block of 200x200 pixels = 40000

So, even assuming the most ridiculously stupid of implementations of this
particular 'scanline stream' scheme, with loads of duplication and stuff, this
amounts to only 587256 bytes, is about half a meg.

Means even handling of images this big is quite feasable, but avoid temp
buffers, especially the kind you get from TIFFRGBA interface costing 32bits for
each 1bits pixel in the huge source image...

Joris Van Damme
info@awaresystems.be
http://www.awaresystems.be/
Download your free TIFF tag viewer for windows here:
http://www.awaresystems.be/imaging/tiff/astifftagviewer.html


AWARE [SYSTEMS]		Imaging expertise for the Delphi developer
		TIFF and LibTiff Mailing List Archive
	LibTiff Mailing List TIFF and LibTiff Mailing List Archive August 2005 Previous Thread Next Thread Previous by Thread Next by Thread Previous by Date Next by Date Contact The TIFF Mailing List Homepage This list is run by Frank Warmerdam Archive maintained by AWare Systems	Thread 2005.08.09 21:28 "can't create thumbnail of large file", by Boris Filipoviè 2005.08.09 22:07 "Re: can't create thumbnail of large file", by Bob Friesenhahn 2005.08.10 00:46 "Re: can't create thumbnail of large file", by Joris Van Damme 2005.08.10 00:46 "Re: can't create thumbnail of large file", by Joris Van Damme Boris Filipoviè wrote: > rastersize=2069060000 That's quite a lot. Are you using the RGBA interface? If so, that explains your need for a complete raster. > Rows/Strip: 1 This particular item here indicates that even with LibTiff, it should be possible to cut down memory needs very substantially. You need to build yourself a 'scanline stream'. One such design involves a linked list of 'scanline manipulators'. You get scanlines by asking ('pulling') the last manipulator in the list. This one pulls the previous one as needed, etc. The list you need to set up is the following - interface with LibTiff, for example using TIFFReadScanline - x resampler. This step's pull handler first pulls a 1bit per pixel scanline from the previous step. Next, it applies a pre-calculated resampling map to it, transforming the 118228 singlebit pixels in thumbnail width 8bit grayscale pixels if you wish - y resampler. This step's pull handler pulls a lot of scanlines from the previous step. However, it does not need to store everyone of them. Instead, it multiplies everyone with a value of a (pre-calculated if you wish) map, and adds it to a temp buffer. The temp buffer might be 16bits per pixel. Upon having pulled and processed all source lines that go into a destination line, it may apply a final convertion to a result 8bit per pixel format. - interface with your result thumbnail, probably just merely pulling a scanline from previous step and copying it to the destination thumbnail buffer. This cuts memory needs dramatically, and speeds up things even if you have the kind of technology that can support memory blocks of 2 gig in size. Let's do a quick calculation, assuming the most stupid of ways to design this and lots of unnecessary buffering. I furthermore assume a thumbnail size of 200x200. - interface with LibTiff: one 1bit per pixel scanline of 118228 pixels = 14800 bytes - x resampler: one 1bit per pixel scanline of 118228 pixels = 14800 bytes one 8bit per pixel scanline of 200 pixels = 200 bytes a pre-calculated resampling map of 118228 weights, say 16bits per weight, all depending on implementation: 236456 bytes - y resampler: one 8bit per pixel scanline of 200 pixels = 200 bytes one 16bit per pixel temp buffer of 200 pixels = 400 bytes one 8bit per pixel scanline of 200 pixels = 200 bytes a pre-calculated resampling map of 140000 weights, say 16bits per weight, all depending on implementation: 280000 bytes - result thumbnail writer one 8bit per pixel scanline of 200 pixels = 200 bytes one 8bit per pixels result thumbnail memory block of 200x200 pixels = 40000 So, even assuming the most ridiculously stupid of implementations of this particular 'scanline stream' scheme, with loads of duplication and stuff, this amounts to only 587256 bytes, is about half a meg. Means even handling of images this big is quite feasable, but avoid temp buffers, especially the kind you get from TIFFRGBA interface costing 32bits for each 1bits pixel in the huge source image... Joris Van Damme info@awaresystems.be http://www.awaresystems.be/ Download your free TIFF tag viewer for windows here: http://www.awaresystems.be/imaging/tiff/astifftagviewer.html

LibTiff Mailing List

Thread

2005.08.10 00:46 "Re: can't create thumbnail of large file", by Joris Van Damme