LibTiff Mailing List

Tiff users,

In regard to storing elevation data in TIFF images: At the most abstract 
level there are two distinct approaches depending on your intended 
audience. One is to pre-process the data to fit in a "normal" TIFF image 
that any reasonably sophisticated graphics application can read and the 
other is to encode the raw data in a format that retains as much of the 
original information as possible at the expense of portability and 
usefulness in a general graphics context.

I first became involved with LibTIFF when I wrote imaging software for a 
magnetic resonance imaging center. The data was collected on a German 
made spectrometer with using 24 bits words, non IEEE 48 bit floating 
point format with signed mantissa and signed exponent and interleaved 
data words of mantissa and exponent with part of each word being 
assigned to both the mantissa and the exponent.  Only Bruker Instruments 
could dream up such a format. We ported the program to a Dec VAX running 
VMS which has 4 different floating point formats, none of which were 
IEEE compliant and we did Fourier transforms on the data with a 
Skywarrior vector processor at first and then in C code when SKY quit 
supporting the vector processor. Reading the raw data from the 
spectrometer and converting it to every known machine format in the 
world was not an option.  I chose to support big endian and little 
endian variants for 8, 16, 32 bit integer, IEEE float and double 
floating point formats, and TIFF and I contributed some patches to the 
bitbashing macros for the VAX C floating point to IEEE format conversions.

The data represented signal strength values collected over time in a 
three dimensional space, slices if you will, through the brain. The 
intensity varied logarithmically rather than linearly. To put this raw 
data in a TIFF file would be meaningless even if it could have been 
done. Preprocessing was everything; to find the range of values that 
defined the region of interest and an acceptable range to present in a 
graphic format. The histogram analysis, clipping, thresholding, etc 
could only be performed by someone who was an expert in the field and 
knew what they were looking for. While a field like MRI might be an 
extreme case, I doubt that it is the only case where assuming the data 
can be reconstructed meaningfully in any automated fashion would be a 
false assumption. To people in the field that wanted to write their own 
software, we offered the raw data in any of the exported formats with an 
ASCII header that described all the acquisition parameters used in the 
study. There was no way to put all this in a header and add it to the 
TIFF file.  For grant writers we offered TIFFs as grayscale or palette 
color RGB.  AFTER clipping, min/max filtering, etc by the scientist, we 
scaled the image to fit in the range of the TIFF image type, ie 0 - 255 
for grayscale images, MIN_IS_BLACK.  We also offered an option to chose 
among a series of custom palettes to provide various visual 
interpretations of the data. For example, we created an RGB palette that 
mapped the top 16 values to 16 distinct colors  and then mapped the 
remaining remaining palette entries linearly to shades of gray, a sort 
of pseudo-color over grayscale approach. Visually, this conveyed the 
information in the data very nicely, the "hot" signal strength spots 
were highlighted against a background that appeared to be grayscale even 
though there was never any red, green, or blue component of the data. I 
also wrote an algorithm that created isolines mapped to the color 
palette entries at the points where the data values changed between 
discrete ranges so that the regions would be outlined rather than 
obscured, a contour map of the intensity levels as it were. These were 
artificial as well. but it was common practice in the field to use color 
to map to intensity just as any false color images of space objects are 
frequently used by NASA.

In that application, the original data would have no meaning to anyone 
but a spectrometer in raw form. Digital Elevation Model data can be 
negative but at least it is linear. It is my opinion that it is wrong to 
assume that you can auto scale any scientific data and not do an 
injustice to the result in at least some special area of scientific 
application. Set values for SMinSampleValue and SMaxSampleValue for the 
agreed upon logical min/max in that scientific field, not the specific 
image in question. If you are using TIFF as a scientific data 
interchange format instead of a picture viewer format, then the people 
in each field need to get together and decide what the logical range of 
values in the data set is going to be, not the TIFF community who can't 
possibly know these things. If you just want TIFF to show a 
representation of the data for general viewing and passing amongst 
desktop applications, then pick a set of minima/maxima that lend 
themselves to viewing and printing, scale your data to those values and 
be done with it. Having long double floating point numbers in a TIFF 
file won't do you any good if all the values are zero except for one 
pixel that has a value of 2**38 - 1 if your intention is to view the 
image on screen without your own custom software.

I would vote for adding new TAGS to provide the additional information 
that explains how the stored image data relates to the original 
collected range, ie scale factor, and the range of possible values for 
that data. You really need both for scientific data.  If there are 
standard algorithms that are applied to the data that can be identified 
in a TAG, add them too. How does GeoTIFF stores the information about 
the Geodetic reference model and projection used for the image? Aren't 
these comparable "scientific" issues that a non-geotiff enhanced reader 
ignores? That will limit the ability to get this information to those 
applications that support those tags of course, but anybody could read 
the scaled data into a graphics package and manipulate the image to view 
it in some fashion if the data is pre-scaled to the ranges normally 
supported in integer formats say 0 - 255, but only the specialists in 
each field will know what the original range of values are and how they 
were preprocessed to get to those values. Anybody can write a propriety 
binary file format that no one else can read, but TIFF is still my best 
vote for platform independence, speed, flexibility, and scalability. 
There are also other formats for cross platform data interchange that 
are public I believe if viewing is not your real concern. Look at netcdf 
among others if all you want is portable data interchange.

My two cents worth and nothing more. Descendit soapbox.

Richard Nolde


AWARE [SYSTEMS]		Imaging expertise for the Delphi developer
		TIFF and LibTiff Mailing List Archive
	LibTiff Mailing List TIFF and LibTiff Mailing List Archive January 2007 Previous Thread Next Thread Previous by Thread Next by Thread Previous by Date Next by Date Contact The TIFF Mailing List Homepage This list is run by Frank Warmerdam Archive maintained by AWare Systems	Thread 2007.01.18 05:40 "Scientific Data in Tiff images", by Richard Nolde 2007.01.18 11:27 "Re: Scientific Data in Tiff images", by Andrey Kiselev 2007.01.18 05:40 "Scientific Data in Tiff images", by Richard Nolde Tiff users, In regard to storing elevation data in TIFF images: At the most abstract level there are two distinct approaches depending on your intended audience. One is to pre-process the data to fit in a "normal" TIFF image that any reasonably sophisticated graphics application can read and the other is to encode the raw data in a format that retains as much of the original information as possible at the expense of portability and usefulness in a general graphics context. I first became involved with LibTIFF when I wrote imaging software for a magnetic resonance imaging center. The data was collected on a German made spectrometer with using 24 bits words, non IEEE 48 bit floating point format with signed mantissa and signed exponent and interleaved data words of mantissa and exponent with part of each word being assigned to both the mantissa and the exponent. Only Bruker Instruments could dream up such a format. We ported the program to a Dec VAX running VMS which has 4 different floating point formats, none of which were IEEE compliant and we did Fourier transforms on the data with a Skywarrior vector processor at first and then in C code when SKY quit supporting the vector processor. Reading the raw data from the spectrometer and converting it to every known machine format in the world was not an option. I chose to support big endian and little endian variants for 8, 16, 32 bit integer, IEEE float and double floating point formats, and TIFF and I contributed some patches to the bitbashing macros for the VAX C floating point to IEEE format conversions. The data represented signal strength values collected over time in a three dimensional space, slices if you will, through the brain. The intensity varied logarithmically rather than linearly. To put this raw data in a TIFF file would be meaningless even if it could have been done. Preprocessing was everything; to find the range of values that defined the region of interest and an acceptable range to present in a graphic format. The histogram analysis, clipping, thresholding, etc could only be performed by someone who was an expert in the field and knew what they were looking for. While a field like MRI might be an extreme case, I doubt that it is the only case where assuming the data can be reconstructed meaningfully in any automated fashion would be a false assumption. To people in the field that wanted to write their own software, we offered the raw data in any of the exported formats with an ASCII header that described all the acquisition parameters used in the study. There was no way to put all this in a header and add it to the TIFF file. For grant writers we offered TIFFs as grayscale or palette color RGB. AFTER clipping, min/max filtering, etc by the scientist, we scaled the image to fit in the range of the TIFF image type, ie 0 - 255 for grayscale images, MIN_IS_BLACK. We also offered an option to chose among a series of custom palettes to provide various visual interpretations of the data. For example, we created an RGB palette that mapped the top 16 values to 16 distinct colors and then mapped the remaining remaining palette entries linearly to shades of gray, a sort of pseudo-color over grayscale approach. Visually, this conveyed the information in the data very nicely, the "hot" signal strength spots were highlighted against a background that appeared to be grayscale even though there was never any red, green, or blue component of the data. I also wrote an algorithm that created isolines mapped to the color palette entries at the points where the data values changed between discrete ranges so that the regions would be outlined rather than obscured, a contour map of the intensity levels as it were. These were artificial as well. but it was common practice in the field to use color to map to intensity just as any false color images of space objects are frequently used by NASA. In that application, the original data would have no meaning to anyone but a spectrometer in raw form. Digital Elevation Model data can be negative but at least it is linear. It is my opinion that it is wrong to assume that you can auto scale any scientific data and not do an injustice to the result in at least some special area of scientific application. Set values for SMinSampleValue and SMaxSampleValue for the agreed upon logical min/max in that scientific field, not the specific image in question. If you are using TIFF as a scientific data interchange format instead of a picture viewer format, then the people in each field need to get together and decide what the logical range of values in the data set is going to be, not the TIFF community who can't possibly know these things. If you just want TIFF to show a representation of the data for general viewing and passing amongst desktop applications, then pick a set of minima/maxima that lend themselves to viewing and printing, scale your data to those values and be done with it. Having long double floating point numbers in a TIFF file won't do you any good if all the values are zero except for one pixel that has a value of 2**38 - 1 if your intention is to view the image on screen without your own custom software. I would vote for adding new TAGS to provide the additional information that explains how the stored image data relates to the original collected range, ie scale factor, and the range of possible values for that data. You really need both for scientific data. If there are standard algorithms that are applied to the data that can be identified in a TAG, add them too. How does GeoTIFF stores the information about the Geodetic reference model and projection used for the image? Aren't these comparable "scientific" issues that a non-geotiff enhanced reader ignores? That will limit the ability to get this information to those applications that support those tags of course, but anybody could read the scaled data into a graphics package and manipulate the image to view it in some fashion if the data is pre-scaled to the ranges normally supported in integer formats say 0 - 255, but only the specialists in each field will know what the original range of values are and how they were preprocessed to get to those values. Anybody can write a propriety binary file format that no one else can read, but TIFF is still my best vote for platform independence, speed, flexibility, and scalability. There are also other formats for cross platform data interchange that are public I believe if viewing is not your real concern. Look at netcdf among others if all you want is portable data interchange. My two cents worth and nothing more. Descendit soapbox. Richard Nolde

LibTiff Mailing List

Thread

2007.01.18 05:40 "Scientific Data in Tiff images", by Richard Nolde