LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
January 2007
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2007.01.15 08:01 "Re: bigtiff", by Joris Van Damme
Albert, I must say that overall, I agree with Frank, except on the timing issue. In my mind, it is too late to change BigTIFF. Albert Cahalan wrote: > > The first was to provide better backward compatibility - however, it > > wasn't exactly clear to me how much backward compatibility it was > > going to give. Was the intention that TIFF applications could read > > an image? Just some of the image characteristics? > > With no modifications, just some image characteristics. This could > be very little (we have a TIFF) or it could include things such as > compression, size, and colorspace. It could also be nothing, if the > magic number changes. > > More importantly, it was intended that software modifications be easy. > Keeping the data structures the same size (particularly the IFD entry > structs and the main header) lets TIFF-handling software be upgraded > with very little change. The same data structures may be used. But the price we have to pay is considerable. If I understand you correctly, we'd have holes in the files of up to 4096 bytes. Now, remember it is typical for an IFD to have lots of tags that have a few bytes in data size, not fitting in the IFD. That means an average overhead of an IFD would cost in the order of say five to ten times 4 kilobyte. That's just wasteful. > > For the latest "cruft removal", you proposed relaxing the alignment > > requirement, or possibly enforcing it by actually shifting offsets. > > I would agree that the alignment requirements currently are somewhat > > pointless since I don't believe they are actually adhered to (even > > by libtiff?) > > Enforce it or kill it. That's not how it works. Be correct in what you write, and liberal in what you read, is our motto. Enforce it or kill it is not an option, as we've seen many times before. Some vendors still write OJPEG, we've been trying to kill that for two decades or so. > Both SSE and AltiVec have instructions that require 16-byte alignment. > Handling unaligned data is slow. If the programmer doesn't handle it, > SSE will fault and AltiVec will round off the addresses. Is why I see little use of memory mapping based codec. You need to copy, anyway. And if you do, alignment is restored, and this is not a problem. Frank wrote: > > The UUID for tags has some advantages I suppose. I can see at least one disadvantage, too. Without any threshold to creating new tags, we'd see people doing a lot less effort to reuse existing schemes. > > But overall, all these fly in the face of what I think is most > > important about BigTIFF. That is that it is in all respects the > > same as TIFF except for changing some required values to be 64bit > > to handle large data. It is exactly this similarity to the > > existing TIFF spec that > > I think is important for easy adoption. > > No, bigtiff already broke things severely by changing struct layouts. > If you're going to break things to that extent, you may as well take > the opportunity to clean up some of the ugly things. I disagree. Breaking severely, is, for example, getting a new tag labelling scheme in there. All we did was change offset bitdepth, in the most logical consistent and minimal change fashion. > > For instance moving to 128 bit uuids for tag identities would > > complicate API interfaces that are currently only accepting ints > > for tag ids. > > An initial implementation could simply require that all but the first > (or last) 16 bits be zero. Call this "base level" support if you wish. > You could even just say the extra 14 bytes are reserved padding > bytes for future expansion. It's very nice to eliminate the problem > of tag collisions, but the 8-byte IFD entry alignment is itself quite > useful for performance. This is just not TIFF. We've chosen to extend the file format, not to create a new one. Like I said, there is one point where I don't agree with Frank. I think it *is* too late to change BigTIFF design. The discussions ended over two years ago, and that's how long the 'proposal' has been up there for all to read. There is at least one codec I know of that supports it already, as is, namely mine, and it's being integrated in the new version of a widespread TIFF reading and writing imaging package of a customer as we speak. So it is too late, within a couple of months we'll see BigTIFF files that comply with the current proposal escaping in the wild on a large scale. Furthermore, the BigTIFF-in-LibTiff project has been defined. Sponsors have been found, and a date is set. I would have been happy to discuss this if you'd have been a bit quicker, and maybe I would have been a bit more open about it. But still, the consensus between a broad range of people, at the time the proposal was designed, was to stick as closely to original TIFF as possible. We don't redefine the tag labelling scheme. I for one, was eager to explore the option of actual 'memory management' inside TIFF files, so as to be able to always reclaim space that got unused. That proposal didn't make it either. It's just TIFF with 64bit offsets. Best regards, Joris Van Damme info@awaresystems.be http://www.awaresystems.be/ Download your free TIFF tag viewer for windows here: http://www.awaresystems.be/imaging/tiff/astifftagviewer.html