2009.11.30 09:05 "Re: [Tiff] TIFFTAG_JPEGTABLES mandatory for JPEG-in-TIFF since 3.8.x ?", by Lee Howard

Joris Van Damme (AWare Systems) wrote:

From a JPEG point of view, there roughly exists two options.

One is to use standard tables. One good reason for doing this, is that you want compression to be a single-pass process, when you need compression speed and/or want to avoid the buffer involved in a two-pass compression process.

The other option is to use optimized tables. One good reason for doing this, is if you want the best possible compression ratio.

In this particular case I'm dealing with a JPEG image that is provided from a fax machine, and I'm merely wrapping it with TIFF in order to provide the application with multi-page color fax reception support in one file... as well as the ability to use other TIFF tags for data storage. So unless I intend to alter the image that is provided by the fax machine (and I don't), then I just will be dealing with the JPEG as-is.

This is complicated somewhat by two things: 1) fax machines generally omit image length from the SOF marker and instead use a DNL marker - I take the data from the DNL marker and "fix" the SOF marker because DNL is largely unsupported by JPEG viewers/processors other than fax machines, and 2) fax machines use ITULAB colorspace - and I am now using LittleCMS to convert the colorspace to sRGB because ITULAB is largely unsupported by JPEG viewers/processors. So those are two exceptions to the statement I made above.

From a TIFF point of view, there roughly exist three options that make sense, and then there's a fourth that sneaks in when there's editing and recompressing being done on a limited number of strips or tiles.

One option corresponds to the use of standard tables in JPEG. When you want the best possible compression speed and/or want to avoid the buffer involved in a two-pass compression process, you can prefer to have standard tables on all strips/tiles. In this case, it makes sense not to include the tables in every single strip/tile, but to write them only once in the JpegTables tag.

The entire JPEG image is written in one strip with TIFFWriteRawStrip. So I don't think that there's any savings to be had in this option.

Another option corresponds to the use of optimized tables. If you want the best possible compression ratio, and your strips/tiles are not too small, it may very well make sense to not include the JpegTables tag, but include tables in every strip/tile, individually optimized for the best possible compression ratio of that particular strip/tile.

I can't speak to the optimization - I'd like to leave this as provided by the fax machine as much as possible although I am processing the image with LittleCMS - but this sounds like the case I am dealing with. I'd prefer to trust the fax machine (or LittleCMS) to be responsible for optimization.

If your strips/tile are reasonably small, and/or your picture is reasonably uniform, I guess it may turn out the best possible compression ratio may be to use tables that are optimized for the complete picture instead of a single strip or tile, and include those in a JpegTables tag, and not include them in every strip/tile. It may be that the gain from including tables only once outweights the gain from table optimization on an individual strip/tile basis.

I don't know exactly what "reasonably small" means, but I don't think that this is the case - see the sample to which I linked in the last mail. But since there is only one strip I don't think that this option is applicable.

Last but not least, TIFF can support editing and rewriting of a single tile or strip. In such cases, it can happen that the rewriter includes tables in an individual strip/tile, that may be optimized for that particular strip/tile, while the original writer choose to use the JpegTables tag instead. As such, there can be tables in both the JpegTables tag and in some or all of the strips/tiles. This too, is perfectly legal, the latter taking precedence on any individual strip/tile.

As the strip data will contain the same JPEG table information as would be included in any JPEGTABLES TIFF tag, I don't think this option is applicable, either.

Since I need to make the application be compatible with libtiff 3.8.2 - if there's not a way to avoid writing TIFFTAG_JPEGTABLES with that version when writing JPEG-in-TIFF, then I'll probably just need to write TIFFTAG_JPEGTABLES properly... but I wanted to understand if this was intentional behavior by libtiff, or if this is a bug, or if I just don't know what I'm doing (likely).

Must be categorized as a bug.

In libtiff/tif_jpeg.c there is this code at the very bottom of the file:

        /*
        ** Create a JPEGTables field if no directory has yet been created.
        ** We do this just to ensure that sufficient space is reserved for
        ** the JPEGTables field.  It will be properly created the right
        ** size later.
        */
        if( tif->tif_diroff == 0 )

        {
#define SIZE_OF_JPEGTABLES 2000
            TIFFSetFieldBit(tif, FIELD_JPEGTABLES);
            sp->jpegtables_length = SIZE_OF_JPEGTABLES;
            sp->jpegtables = (void *) _TIFFmalloc(sp->jpegtables_length);
            _TIFFmemset(sp->jpegtables, 0, SIZE_OF_JPEGTABLES);
#undef SIZE_OF_JPEGTABLES
        }

This code was added after 3.7.2 and before 3.8.2, and it is this code which is causing TIFFTAG_JPEGTABLES to be added to the created TIFF file. If I comment-out this code then the tag is not put there.

Unfortunately, without the tag there I'm still not out of the woods. For example, running the TIFF file through tiff2ps gives me...

JPEGPreDecode: Warning, Improper JPEG sampling factors 2,2 Apparently should be 1,1..

JPEGPreDecode: Cannot honour JPEG sampling factors that exceed those specified..

Does this mean that I should be setting YCBCRSUBSAMPLING?

Thanks,

Lee.