1994.03.03 12:20 "Re: Steve Carlsen's comments on TIFF/JPEG draft", by Niles Ritter

Tom Lane writes,

>> 1. How do we maximize the chances that this will actually get used, and
>> implemented correctly?
>Sam Leffler has indicated that he would like to support the revised definition
>in LIBTIFF. I expect that once he's worked through his inbox stack from his
>sabbatical, he and I will start on integrating the IJG codec with LIBTIFF to
>produce an implementation. Once that's in hand, making some sample files
>should be easy enough.
>Dunno whether you think that LIBTIFF is adequately "influential". If we need
>to get some commercial developers to commit to it too, then I don't know who
>to ask.

I'll bet there's a pretty good user-base of libtiff out there that will attest to its influence. Allow me to volunteer to send you the intermediary modules I wrote to get a working version of the old JPEG-TIFF implementation to function, using the IJG/LIBTIFF libraries. To keep users of our version of the libtiff routines happy (that is, to make the JPEG compression almost as transparent as all the others) I found I had to break both libraries approaches in exactly those places that sparked a lot of discussion in the tiffjpeg mailing list...

For example, the LIBTIFF code expects that the colorspace of the data you send it in the TIFFRead/WriteTile (or strip) is the colorspace indicated by the tag. However, the JPEG codec really wants to take in rgb, and then do the YCBCR conversion itself. The same also holds for the Subsampling tags. Since most of our stuff involves writing unsampled RGB, we broke the way LIBTIFF works by passing in un-subsampled RGB, and made the lib_jpeg.c module pass on to the JPEG codec the subsampling and colorspace information, and let the codec do the work. Unfortunately, this means that the LIBTIFF code is no longer treating compression/color/subsampling in an orthogonal manner, so we would be in trouble if we encountered YCBCR that wasn't JPEG.

I seem to recall that we had to make some changes in the IJG JPEG codec in the way that some of the buffers were allocated. For example, the way we process a tile is to create a "jselrbuffer" module which overrides the JPEG cinfo->methods->get_input_row method to read/write the data from the LIBTIFF raw_data buffer instead of a file. But the jcpipe controller assumes that it's running the whole show, and so doesn't deallocate all of the memory it allocated on startup until the free_all() routine is called, which means that when the lib_jpeg routines call cinfo->methods->d_pipeline_controller for each tile there is a progressive memory leak. Our solution was to modify the simple_controller to deallocate the leaking memory, thusly:

  (*cinfo->methods->extract_MCUs) (cinfo, sampled_data, 
                (int) (cinfo->MCU_rows_in_scan - mcu_rows_output),

   * Release working memory. We have to do it here because
   * the buffers are local and we can't wait for free_all
   * because this routine will be called multiple times.

  free_sampling_buffer(cinfo, fullsize_data);
  for (ci = 0; ci < cinfo->num_components; ci++) {
    (*cinfo->emethods->free_small_sarray) (sampled_data[ci]);
  (*cinfo->emethods->free_small) (sampled_data);

Anyway, as I said, the intermediary modules, together with the enabled tif_jpeg.c code, are available for you and Sam to mull over, if you'd like. I don't want to make my stuff public because it still uses the old TIFF-JPEG specs, such as they are, and I have no interest in propagating an obsolete implementation.

As of this date, we have a *STRONG* interest in supporting JPEG- in-TIFF. One of more recent projects involves compressing a single RGB color scanned map, around 3 Gigabytes uncompressed, into a nice little 100Megabyte JPEG-TIFF compressed 128x128 tiled file. The next best TIFF compression, LZW, only gives us a 3-to-1 ratio, but of course that's lossless...