AWARE [SYSTEMS] Imaging expertise for the Delphi developer
AWare Systems, Imaging expertise for the Delphi developer, Home TIFF and LibTiff Mailing List Archive

LibTiff Mailing List

TIFF and LibTiff Mailing List Archive
August 2008

Previous Thread
Next Thread

Previous by Thread
Next by Thread

Previous by Date
Next by Date

Contact

The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems



Valid HTML 4.01!



Thread

2008.08.22 08:56 "creating sparse files......", by Rogier Wolff
2008.08.22 13:11 "Re: creating sparse files......", by Toby Thain
2008.08.22 15:45 "Re: creating sparse files......", by Rogier Wolff
2008.08.22 16:26 "Re: creating sparse files......", by Toby Thain
2008.08.22 14:44 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.22 15:21 "Re: creating sparse files......", by Toby Thain
2008.08.22 16:27 "Re: creating sparse files......", by Rogier Wolff
2008.08.22 16:40 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.22 16:52 "Re: creating sparse files......", by Rogier Wolff
2008.08.22 18:11 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.22 20:02 "Re: creating sparse files......", by Phillip Crews
2008.08.23 00:12 "Re: creating sparse files......", by Edward Lam
2008.08.23 15:26 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 16:07 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.23 16:23 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 16:46 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.23 15:08 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 15:54 "Re: creating sparse files......", by Bob Friesenhahn
2008.08.23 15:58 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 16:02 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 16:51 "Re: creating sparse files......", by Gene Amtower
2008.08.23 17:39 "Re: creating sparse files......", by Rogier Wolff
2008.08.23 18:03 "Re: creating sparse files......", by Toby Thain
2008.08.23 18:49 "Re: creating sparse files......", by Rogier Wolff
2008.08.25 06:28 "Re: creating sparse files......", by Andrey Kiselev
2008.08.25 09:37 "Re: creating sparse files......", by <jcupitt@gmail.com>
2008.08.25 10:41 "Re: creating sparse files......", by Rob Van Den Tillaart
2008.08.25 12:15 "Re: creating sparse files......", by <jcupitt@gmail.com>
2008.08.26 13:14 "Re: creating sparse files......", by Edward Lam

2008.08.22 14:44 "Re: creating sparse files......", by Bob Friesenhahn

On Fri, 22 Aug 2008, Rogier Wolff wrote:
>
> I'm stitching kind of large panoramas. This results in big
> intermediate files. On my last run, which took overnight to stitch,
> I thought 42 Gb of free disk space would be enough. Wrong!
>
> I got over thrity files of over 1.2Gbytes, filling up the disk.
>
> It turns out that most of the files contain lots of zeroes. On Unix
> this can be stored effciently by not issueing a "write" with a buffer
> full of zeroes, but by seeking over the area. The operating system
> will act as if the area was filled with zeroes.

This is an interesting issue.  While holey files seem like a panacea, 
there can be some drawbacks.  They are best for files which are 
written just once (like core files) and not so good for files which 
are expected to be updated in place.  For files which are updated in 
place, the updated hole is quite likely to increase disk fragmentation 
since now it takes more space and the space will need to be from some 
other place on disk.  Fragmentation behavior is quite filesystem 
dependent.

It is worth considering enabling filesystem compression, or using 
whole-file compression.  Perhaps even just enabling normal TIFF 
compression (e.g. LZW) is sufficient to eliminate the long spans of 
zeros.

Bob
======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/