2003.09.01 20:02 "[Tiff] Reading and writing to memory.", by Frank Warmerdam
V�in� J�rvel� wrote:
My software has to manipulate some very large TIFF images. The images are so big that they won't fit in the memory uncompressed, so i need to have the images compressed in the memory and uncompress it strip at a time, manipulate and compress back to the memory. When the manipulations are complete, then write it to a file.
I know i could pump up the swap space, but it might be faster to do it all in memory even with all this uncompressing and compressing than writing and reading gigabytes of data from and to swap(done automatically by kernel ofcourse) and work from there.
I have tried using TIFFClientOpen by creating my own read,write,seek,size procs which pass a struct (through thandle_t) to the functions and manipulate the data(image buffer) in the struct. But when i try to "open" the memory with
Tiff = TIFFClientOpen( "dummy", "w", (thandle_t) &mt, tiffMemReadProc, tiffMemWriteProc, tiffMemSeekProc, tiffMemCloseProc, tiffMemSizeProc, NULL, NULL );
It fails by saying: "dummy: Not a TIFF file, bad magic number 0 (0x0)."
I don't undestand this error message.. i'm not trying to read or append (to) an old TIFF file, but it apparently still tries to read one?
Is there anything specific i have to do in those procs? Return a specific value on the first run or something?
There are indeed fairly detailed semantics required of the IO functions. I don't know that these assumptions are well documented anywhere. You might be best off reviewing existing low level implementations carefully if you want to make your own.
There are already at least two memory based TIFF implementations for libtiff. in contrib/mfs there is one. I also have one I use as part of GDAL, based loosely on mfs.
I am not absolutely certain why you are getting the "dummy: Not a TIFF file, bad magic number" message. It may be that as currently setup even a writable data stream needs to support read, and that you are not allowing the tiff header to be read back by the code.
Is mmap and unmap procs mandatory?
They must be supplied, but a simple "unsupported" implementation is fine.
I can't help but wonder though if your overall approach is appropriate. Why is it that you need to hold your whole image in memory? Why not operate on the image in chunks as Pushkar suggests?
I set the clouds in motion - turn up | Frank Warmerdam, firstname.lastname@example.org
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | Geospatial Programmer for Rent