2008.11.20 04:47 "Re: [Tiff] Memory leak (TIFFOpen, TIFFReadTile)?", by Bob Friesenhahn

Actually my main complaint with mmap'ing tiffs in the current implementation is that we don't really use it to the best effect. Given the structure of tiff, it would be possible to simply mmap a file, and then set the pointers to the data structures to access it directly to the offets in that file.

There is also the shortcoming that it is not necessary to map the entire file at once. Several smaller mappings may be used to window only to the regions which are actually necessary. This allows mmap to scale to huge BigTiff files on 32 bit systems. It also helps provide info to the OS as to what pages should be disposed of first. If libtiff was more committed to memory mapping, then it could take advantage of it better.

I have done testing of memory mapping with large files and with a proper OS and the right system calls, it is possible to zing through a large file using windowing without noticing any discomfort at all. The key is to use a system call to tell the OS that you don't need a range of pages any more so they vaporize from the page cache. Unfortunately, not all popular operating systems qualify as "proper OS".

Bob
======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/