AWARE [SYSTEMS] Imaging expertise for the Delphi developer
AWare Systems, Imaging expertise for the Delphi developer, Home TIFF and LibTiff Mailing List Archive

LibTiff Mailing List

TIFF and LibTiff Mailing List Archive
September 2004

Previous Thread
Next Thread

Previous by Thread
Next by Thread

Previous by Date
Next by Date

Contact

The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems



Valid HTML 4.01!



Thread

2004.09.16 19:12 "BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.16 19:23 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.16 19:36 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.16 19:38 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.16 20:08 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 01:14 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.16 22:39 "Re: BigTIFF Tag Value Count issue", by Ian Ameline
2004.09.17 00:09 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 00:32 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 01:54 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 02:29 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 02:58 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 03:08 "Re: BigTIFF Tag Value Count issue", by Chris Cox
2004.09.17 04:20 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 04:39 "Re: BigTIFF Tag Value Count issue", by Chris Cox
2004.09.17 11:25 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 14:31 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 14:40 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.17 15:02 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 21:41 "Re: BigTIFF Tag Value Count issue", by Chris Cox
2004.09.17 14:53 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 15:06 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 21:36 "Re: BigTIFF Tag Value Count issue", by Chris Cox
2004.09.17 05:46 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.17 11:40 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 13:11 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.17 13:34 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 15:10 "Re: BigTIFF Tag Value Count issue", by Fernando Loygorri
2004.09.17 15:10 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 15:55 "Re: BigTIFF Tag Value Count issue", by Fernando Loygorri
2004.09.17 16:00 "Re: BigTIFF Tag Value Count issue", by Frank Warmerdam
2004.09.17 16:33 "Re: BigTIFF Tag Value Count issue", by Fernando Loygorri
2004.09.17 16:13 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 16:34 "Re: BigTIFF Tag Value Count issue", by Fernando Loygorri
2004.09.17 16:43 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 21:48 "Re: BigTIFF Tag Value Count issue", by Ross A Finlayson
2004.09.17 22:15 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn
2004.09.17 21:41 "Re: BigTIFF Tag Value Count issue", by Chris Cox
2004.09.17 11:38 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 06:35 "Re: BigTIFF Tag Value Count issue", by Rob Tillaart
2004.09.17 11:43 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 12:46 "Re: BigTIFF Tag Value Count issue", by Rob Tillaart
2004.09.17 12:55 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 13:20 "Re: BigTIFF Tag Value Count issue", by Rob Tillaart
2004.09.17 13:47 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.17 13:08 "Re: BigTIFF Tag Value Count issue", by Ross A Finlayson
2004.09.19 12:31 "Re: BigTIFF Tag Value Count issue", by Andrey Kiselev
2004.09.19 16:43 "Re: BigTIFF Tag Value Count issue", by Joris Van Damme
2004.09.20 16:18 "Re: BigTIFF Tag Value Count issue", by Ed Grissom
2004.09.20 22:05 "Re: BigTIFF Tag Value Count issue", by Steve Carlsen
2004.09.20 23:09 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn

2004.09.17 01:54 "Re: BigTIFF Tag Value Count issue", by Bob Friesenhahn

On Fri, 17 Sep 2004, Joris wrote:
>
> Partially true. Well, completely true, I guess, if memory mapping is involved.
> However, memory mapping of files of a format that is specifically designed to be
> > 4 gig, might not be a good idea. In windows, you can have a 'window' of a file

Stop that!  We are discussing BigTIFF here!!! :-)

There was a time when 640K seemed like a hard limit but that time is 
long past.  Yesterday 4GB seemed like a hard limit but now 64-bit PCs 
can be purchased for under $1000.  In a couple of years, most PCs will 
come with 4GB of RAM.  Years later people may be cursing the 
inventors of BigTIFF because they only thought about problems based on 
past experiences and didn't think ahead.

> memory mapped, called 'view' IIRC, limiting stress on address space, but this
> too, will not be a good idea in the BigTIFF case, since TIFF is inherently
> completely random access and so with this scheme you might have to swap to
> another 'window' for just about every new read.

TIFF may be random access but it can hardly be described as 
"completely random access".  Raster image pixel data is usually 
ordered in a contiguous fashion.

64 bit CPUs are readily able to memory map huge files, limited more by 
the design of the MMU than anything else.

> As a sidenote, slightly of topic as to this particular BigTIFF issue, but on
> topic as to the memory mapping... I've never quite understood why anyone would
> want to memory map a TIFF file for use in eg LibTiff. I believe that there is no
> speed or system advantage. The main advantage of memory mapping, I believe, is
> merely the convenience of accessing memory instead of having to do IO all the
> time, which is a bit less troublesome code logic... but, if your codec is build
> to accommodate all this IO, why would one next essentially wire IO routines to
> accommodate memory mapping?

With most modern operating systems, memory-mapped accesses are 
significantly faster than stdio-based or read/write access, 
particularly if the data is accessed more than once.  The limitation 
on this is how effective the system is at forgetting mapped pages 
which are no longer needed anymore.  If unneeded pages are not purged 
when they should be the system tends to choke and "thrash" once all 
physical memory has been consumed.  Most OSs (even Windows) provide a 
call to tell the system that certain pages are no longer required.  I 
have done quite a bit of investigation and benchmarking in this area 
and have learned that some OSs do tremendously better than others. 
Linux was among those that fared the worst.

> Or am I wrong in thinking there is no actual speed/system/cache/management
> advantage?

There is always an advantage when the natural filesystem caching is 
used directly and data copies and function calls are minimized.

> Option 0: stick with 4 byte tag count members (alignment, tag data <= 4 gig,
> Frank's choice)
> Option 1: make it an 8 byte tag count (no alignment, tag data > 4 gig allowed)
> Option 2, courtesy of Bob: make it 8 byte tag count, add 4 padding bytes
> (alignment, tag data > 4 gig allowed)
>
> Can I put you down as an option 2 vote, or were you merely signalling the
> option?

Sure, there should always be a second option so put me down for option 
2. :-)

Bob
======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us
http://www.simplesystems.org/users/bfriesen