LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
July 2011
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2011.07.29 00:49 "Re: Using photon lists rather than rasters", by Ryan Wong
Actually, the sticking point is not whether something is raster. Here is the real sticking point: http://www.awaresystems.be/imaging/tiff/tifftags/baseline.html Any tags that are listed on this TIFF Baseline page, are not subject to modification or re-interpretation. To give an example, an TIFF page must be either Tiled or Striped, and if it is Tiled it must have certain required fields indicating the tile size and data locations, likewise if it is striped it must have some other required fields. The number of strip/tile must be a positive integer. A file that has a TIFF header but violates the Baseline assumption, must not use the TIFF magic number (0x49 0x49 0x2a 0x00, or 0x4d 0x4d 0x00 0x2a) or else it would be properly called a "malicious TIFF image file". It could even be automatically deleted by virus scanners. These are about the only "dogmatic" rules about TIFF. Beyond that, TIFF is quite a liberal container format. Thanks, Ryan Wong rwong_002@hotmail.com <mailto:rwong_002@hotmail.com> Disclaimer: The opinions expressed here are the views of the writer(s) and do not necessarily reflect the views and opinions of the writers’ employer(s). Neither the writer(s), nor the writers’ employers, make any claim as to the accuracy of the opinions expressed here. Date: Thu, 28 Jul 2011 11:30:48 -0700 From: Terry.Sprout@Agile-Automation.com To: tiff@lists.maptools.org Subject: Re: [Tiff] Using photon lists rather than rasters This has been an interesting thread. The final decision is to include the image in the TIFF file and simply add the photon list as private meta data. We will use integration methods to achieve compression (integrating 1000 16-bit images gives a 2000:1 compression ratio) and the TIFF file will include photon lists from all images used in the integration. That will allow all TIFF readers to display the integrated image and it will allow my software (and any other software that knows how to read the photon lists) to use the photon lists to create images with shorter exposures if desired. Including the image also allows users to browse the hard drive when trying to find a specific set of images. In the future, I may choose to compress images further by setting the bits/sample to the largest bit-depth in the image and then packing the image. So images that are not integrated will be 1-bit/sample images, which gives me another 8:1 compression ratio. I’m not sure if there is enough value in packing bits, however, so I may never do it. I do want to mention one other thought. Some of you have indicated that images described with a coordinate map is not an image. I am not aware that the definition of an image has anything to do with its representation. An image is the final result that is presented to the user. A 2D array of intensities is no more an image than a list of vectors or a list of coordinates. If an image is defined as a 2D array of intensities, then I guess a 2D array of any A/D devices would also be an image. So to argue that a representation of an image as a list of coordinates does not fit the TIFF specification simply because it doesn’t fit your definition of an image is a bad argument. It is better to say a TIFF image is expected to be comprised of rasters and not coordinate maps. So I am in agreement that trying to use the TIFF structure for a photon list with no raster image is an incorrect use of TIFF. If I want a private use of a TIFF file, I might as well create my own file structure (which would be easier) and provide instructions to anyone that needs to read them. Thank you all for the opportunity to have this discussion. Terry L. Sprout From: tiff-bounces@lists.maptools.org [mailto:tiff-bounces@lists.maptools.org] On Behalf Of rwong_002@hotmail.com Sent: Thursday, July 28, 2011 10:46 AM To: libtiff mailing list Subject: Re: [Tiff] Using photon lists rather than rasters The idea about a better predictor to improve compression of niche scientific images is interesting. Along the same direction, Terry could really use the new compression type for that. Think about that: if it's a new predictor that nobody today knows how to decode, and if it's much more advanced than today's "TIFF predictor" (even though it is indeed a predictor in mathematical/pedantic terms), it doesn't hurt to make its own compression type. As for the photon images, it will require something superbly advanced algorithms, something that blends several existing advanced algorithms together that nobody has seen before. Like: multi-resolution (or progressive-resolution), some kind of CABAC (which is both context-sensitive and uses arithmetic coding), etc. That said, the larger TIFF community may try to compel you to (1) open source your implementation and (2) give up, or sign over all intellectual rights and copyrights of that specification, and for that matter (3) it still leaves you vulnerable to patent lawsuits while the larger TIFF community just sit and watch. Therefore, it's important to remember that ultimately you have to decide what to do with your invention; you may decide whether to open it or not. If in doubt, talk to your company or employer lawyer. Thanks, rwong_002@hotmail.com <mailto:rwong_002@hotmail.com> Disclaimer: The opinions expressed here are the views of the writer(s) and do not necessarily reflect the views and opinions of the writers’ employer(s). Neither the writer(s), nor the writers’ employers, make any claim as to the accuracy of the opinions expressed here. From: ccox@adobe.com To: tiff@lists.maptools.org Date: Wed, 27 Jul 2011 16:57:30 -0700 Subject: Re: [Tiff] Using photon lists rather than rasters Tagged Image File Format. I would rather not see a TIFF “image” that is not an image and is not readable by other software. TIFF already gets too much abuse with undocumented custom compression types that render the files useless. In this case you are creating a vector display list, for which there are more appropriate types. Personally, I’d just apply a better predictor to improve compression and keep the image data. If all you need are locations/centroids of photon hits – you could easily convert the image to a 1 bit/sample representation and compress that. Chris On 7/27/11 3:51 PM, "Terry L. Sprout" <Terry.Sprout@Agile-Automation.com> wrote: Ryan Wong wrote the following message and accidently sent it directly to my email: *************************** Hello Terry, TIFF is a raster image format, not a vector image format. Inside a TIFF file, each page must contain exactly one raster image. It can contain vector annotations in metadata, but viewers are not required to render it. It will most likely be ignored by most TIFF viewers that do not understand it. A TIFF page can contain thumbnail images, which, like any other metadata, can be ignored by viewers that do not support it. No matter how hard you try, no current software would be able to decode from your photons list. Even far into the future, mainstream software do not have a need to support “photon lists”, but they will happily render the raster image part of a processed TIFF file. My suggestion is to store the coordinates as a metadata tag, which accompanies with the original (unprocessed) image file. TIFF metadata tags can store arbitrary byte data and do not have size limits (unless going into the gigabytes range). This is the recommended way of storing your coordinates data. And yes, floating point (32-bit, 64-bit and all IEEE-approved ones) are supported in metadata. Viewers that do not understand your photons list will simply ignore it, and will still display the unprocessed image. Similar to the way EXIF can be added as an optional part of any TIFF file. If you do not take this approach, I don’t think your proposal will be accepted by the larger TIFF community at all. In particular, modifications to NewSubFileType, proposals to support a StripByteCount of zero, will be shot down immediately. Thanks, rwong_002@hotmail.com <mailto:rwong_002@hotmail.com> Disclaimer: The opinions expressed here are the views of the writer(s) and do not necessarily reflect the views and opinions of the writers’ employer(s). Neither the writer(s), nor the writers’ employers, make any claim as to the accuracy of the opinions expressed here. *************************** Hello Ryan, Thank you for your comments. Including the original image doesn’t help since one purpose is to create smaller files so the drive doesn’t fill up so fast. My software can currently fill up a hard drive at 240 MB/s. Some people are using a camera that outputs very sparse photons (maybe 100 photons from a 300 KB image 120 times per second) and they only want to know where the photons are located. They can then playback the photon list and create images with different exposures. The original image will consume over 600 KB per image (they’re 2 bytes per pixel) or 73 MB/s; or 4.8 GB/m; or 262 GB/h. In contrast, the photon list will only consume about 400 bytes per image; or 48 KB/s; or 2.9 MB/m; or 173 MB/h. That’s a very good compression ratio. I’ve decided to treat the coordinate list as a new compression type, since that’s basically what it is. When my new compression type is used (34927), my new coordinate list tag (51098) must be used as well. The coordinate list tag is a pointer to a linked list of IFDs that will contain a list of pixel coordinates along with a list of intensity values (and a timestamp). TIFF readers that don’t understand the compression type should ignore the image. I don’t really expect most people that provide TIFF readers to add this capability, but they can if they want to view the images. The scientists that use this data will add this capability because it helps them in their analysis. Terry L. Sprout (510) 324-5501 w7-32 _______________________________________________ Tiff mailing list: Tiff@lists.maptools.org http://lists.maptools.org/mailman/listinfo/tiff http://www.remotesensing.org/libtiff/ _______________________________________________ Tiff mailing list: Tiff@lists.maptools.org http://lists.maptools.org/mailman/listinfo/tiff http://www.remotesensing.org/libtiff/