1994.12.14 17:38 "Alternative to Private IFD's", by Niles Ritter

1994.12.14 17:38 "Alternative to Private IFD's", by Niles Ritter

Here is an interesting alternative to private TIFF IFD's which some of you may find useful in your own applications, especially if you need a *lot* of tags.

During the process of developing a new standard for storing geographic information in TIFF tags, we ran up against the following requirements:

  1. The need for a very large number of "Tagged" information fields of various types (possibly hundreds), with an easily extensible format.
  2. The implementation must be "TIFF-visible", so that a naive TIFF-dump program would be able to display all the values, in their correct format (i.e, it is not good enough to just encode everthing into a large ASCII string; LONG values should be in a LONG tag, etc.).
  3. The ability to implement (1) and (2) with only three or four reserved public TIFF tags,
  4. Easy to understand approach, without dozens of complex order-dependent structures, and no UNKNOWN byte-streams.
  5. Must support the TIFF philosophy of data abstraction, and be modular enough that TIFF I/O modules can operate independently of the new data-storage implementation.

The usual suggestion from Aldus is to take your 5 tags and make one of them a LONG offset to a private IFD. This solution does not satisfy requirement (2), since a naive TIFF reader wont know to go to the private IFD and parse the entries, so you will not see all of the information. It also does not support (5) because the TIFF I/O module would have to be modified to make sure it doesn't overwrite invisible data referenced by the private IFD. Finally, with private IFD's you have to completely re-write the TIFF-directory parser module, including all of the byte-swapping procedures, etc, instead of letting the TIFF layer handle all of that for you. This makes code difficult to maintain.

The following is an intereresting alternative, which is described for others who may also have similar problems. It satisfies all of conditions 1-5, though (4) may be debatable to some.

The "Keyed Information" approach

We define a new entity called a "Key" which is virtually identical in function to a "Tag", but has one more level of abstraction above TIFF. If you like, it is a sort of "Meta-Tag". A Key works with formatted tag-values of a TIFF file the way that a TIFF file deals with the raw bytes of a data file.

Recall that a TIFF directory consists of a header, indicating NumberOfTags, followed by a list of Tag-entries which look like this:

   TagID, Type, Count, Value_Offset,

which gives the Tag ID number, its data format, the number of bytes in data-array, followed by either the Value itself, or else an absolute offset in the file to the data. We do the same thing with Keys, but in place of "Type" we use a TIFF Tag ID to describe the data!

Implementation Details

To set up a Keyed storage system, you reserve one of your personal TIFF Tags to be a "KeyDirectoryTag", which is of type SHORT (or possibly LONG). This will contain several values of header information, followed by a list of KeyEntries. In our implementation there are four header values, which are:

   KeyDirectoryVersion, KeyVersion, NumberOfKeys, Reserved

where KeyDirectoryVersion is like the TIFFVersion (42), and will only change if this Tag's Key *structure* is changed. The KeyVersion merely indicates what version of Key-Sets are used. The next value, NumberOfKeys, indicates how many Keys are defined in this Tag, followed by a "Reserved" value which only serves as padding.

The 4-SHORT header is immediately followed by a collection of <NumberOfKeys> KeyEntry sets, each of which is also 4-SHORTS long. Each KeyEntry is of the form:

  KeyID, TIFFTagLocation, Count, Value_Offset,

where KeyID gives the ID value of your Key (identical in function to TIFF tag ID), and the TIFFTagLocation indicates which TIFF tag contains the value(s) of the Key. The Type (format) of the Key-value is therefore implied by the Type of the TIFFTagLocation ID. Finally, the Count indicates the number of TIFF *values* (rather than the number of bytes) in this key.

Now here's the trick: if TIFFTagLocation is 0, then the Value_Offset contains the actual (SHORT) value of the Key (and Count=1 is implied), or else, Value_Offset indicates the offset *into* the TagArray indicated by TIFFTagLocation.

Following the KeyEntry definitions, the KeyDirectory tag may also contain additional values. For example, if a Key requires multiple SHORT values, they could be placed at the end of this tag, and the KeyEntry will set TIFFTagLocation=KeyDirectoryTag, with the Value_Offset pointing to the end of the tag.

Thus, if you have only 5 real TIFF tags to play with, then you can define the first to be the KeyDirectory Tag, the next to be the LONGValuesTag, the next one ASCIIValuesTag, and perhaps another one as FLOATValuesTag (with one left over). The first tag defines a collection of Keys, which may point to various places in the other three tag-value-arrays.

Here's an illustrative example, with three tags:

  KeyDirectoryTag=( 1,    1,4,0,
                    1,    0,1,5,
                    4,32764,2,3,
                    5,32765,7,5 
                    8,32765,5,0 
                                )
  Tag32764=(10,20,30,40,50,60,70,80)
  Tag32765=("ABLE\0BAKERS")

The first line indicates that this is a Verson 1 KeyDirectory tag, the keys are version 1, and there are 4 Keys defined in this tag.

The next line indicates that the first Key (ID=1) has the value 5, explicitly placed in the entry list (since TIFFLoc=0). The next two lines indicate that the Keys 4 and 5 have their values listed in Tags 32764 and 32765 respectively. Key 4 has two integer values, starting at index 3 (the fourth in array), which consist of (40,50), while Key 5 has a 7-character string value, which is at offset 5 into 32765, and so has the value "BAKERS" (with a null character at the end). The value of Key 8 is "ABLE".

The TIFF layer handles all the problems of data structure, platform independence, format types, etc, by specifying byte-offsets, byte-order format and count, while the Key describes its key values at the TIFF level by specifying Tag number, array-index, and count. Since all TIFF information occurs in TIFF arrays of some sort, we have a robust method for storing anything in a Key that would occur in a Tag.

With this Keyed-value approach, you have 65536 Keys which have all the flexibility of TIFF tag, with the added advantage that a TIFF dump will provide you with all the information that exists in your private implementation.

We welcome your comments.

  --Niles.