2004.04.15 00:26 "[Tiff] Large TIFF files", by Lynn Quam

2004.04.22 23:17 "Re: [Tiff] Large TIFF files", by Bob Friesenhahn

With memory managment, the task becomes simply freeing the old data block, which then becomes tracable and reusable free but managed memory, and asking the memory manager for a new data block in the file that is of suitable size for the new data, which may be (part of) an old reused block or... Very small IO operation instead. TIFF instantly becomes an 'editable' format this way. Seeing that the new file format is specifically targetted towards huge files, the benefit may be substantial.

I have been thinking along those lines for another application. As it turns out, the malloc code used by Linux is available for unencumbered (non-GPL) use. Many versions are available, including older simpler versions that humans can understand.

It should not be a big deal to modify this malloc code to allocate regions in file space rather than heap space. As evidence that this is possible, the GNU mmalloc library (delivered with gdb) already does this (its implementation started with the Linux malloc code).

The state that the allocator uses to keep track of allocations (actually free space) would need to be well documented so that independent implementations are possible.

The malloc code used by BSD Unix can also be re-written for this task. Its allocation management scheme is fairly simple and reasonably easy to document.

If the allocation scheme that Joris describes is used, then modified files may be larger than the original, and file sizes are likely to be somewhat larger by default since allocators typically have a minimum sized allocation so small allocations are padded.

Bob

======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us
http://www.simplesystems.org/users/bfriesen