LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
August 2008
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2008.08.22 20:02 "Re: creating sparse files......", by Phillip Crews
Bob Friesenhahn wrote:
> It is likely that some popular filesystems will fail to seek past the
> end of the file or will return random bytes (or parts of some
> previously deleted file) for the uninitialized portions.
This was my immediate objection to this scheme. Even if it were a
platform-specific or configuration-controlled option, there will be
cases where one O/S is writing to a filesystem on another O/S (i.e.,
over the network), resulting the same random garbage or simply ignored
seeks.
If this is for Windows, the simplest solution for the OP may be to store
the files in an NTFS compressed folder, or modify TIFFFDOpen to accept
an additional flag which would set the file as compressed on compatible
filesystems. Note that the DeviceIoControl call will fail on unsupported
file systems. No further changes should be required (i.e., non-standard
seeks or checking for zeros). Similar minimal changes should be possible
for other operating systems as well.
(NB: I apologize for the HTML mail but I thought it much more readable
for code; I'm sending in both, hoping that the plaintext version will
still be readable!)
In *tif_win32.c* (/untested /code follows, my changes in red):
|#include <winioctl.h>
...
TIFF*
TIFFFdOpen(thandle_t ifd, PCTSTR name, const char *mode)
{
TIFF* tif;
BOOL fSuppressMap = (mode[1] == 'u' || (mode[1] != 0 && mode[2] == 'u'));
static const USHORT compmode = COMPRESSION_FORMAT_DEFAULT;
DWORD retbytes = 0; /* Not used but necessary per MS docs */
if (mode[0] == 'w' && (mode[1] == 'c' || mode[2] == 'c') ) {
/* Compress new files only, otherwise no feedback on slow compression */
DeviceIoControl(ifd, FSCTL_SET_COMPRESSION, &compmode, sizeof comp,
NULL,
0, &retbytes, NULL);
/* Will fail on unsupported file systems */
}
tif = TIFFClientOpen(name, mode, ifd,
_tiffReadProc, _tiffWriteProc,
_tiffSeekProc, _tiffCloseProc, _tiffSizeProc,
fSuppressMap ? _tiffDummyMapProc : _tiffMapProc,
fSuppressMap ? _tiffDummyUnmapProc : _tiffUnmapProc);
if (tif)
tif->tif_fd = ifd;
return (tif);
}
...
|
This should greatly reduce disk consumption and I/O on holey files or
files that are simply compressed.
Reference: http://msdn.microsoft.com/en-us/library/aa364592.aspx