2008.12.17 20:54 "[Tiff] Deleting tags from a directory", by Frank Warmerdam

2009.02.07 14:10 "Re: [Tiff] The OJPEG bug 1996", by

Frank, folks,

This is about http://bugzilla.maptools.org/show_bug.cgi?id=1996. I'll try and get a Bugzilla account again right away and append this mail to that report, too.

In OJPEG, it is relatively common to find that the Strip/Tile ByteCounts tag is missing. In this case, I've sofar found that following the JpegInterchangeFormat tag as the primary and mostly reliable source of JPEG compressed information is sufficient to resolve the situation. Hence, the OJPEG-specific hack in TIFFReadDirectory that makes sure no fatal error is thrown complaining about the situation:

 if (!TIFFFieldSet(tif, FIELD_STRIPOFFSETS)) {
  if ((tif->tif_dir.td_compression==COMPRESSION_OJPEG) &&
      (isTiled(tif)==0) &&
      (tif->tif_dir.td_nstrips==1)) {

   /*
    * XXX: OJPEG hack.
    * If a) compression is OJPEG, b) it's not a tiled TIFF,
    * and c) the number of strips is 1,
    * then we tolerate the absence of stripoffsets tag,
    * because, presumably, all required data is in the
    * JpegInterchangeFormat stream.
    */
   TIFFSetFieldBit(tif, FIELD_STRIPOFFSETS);

  } else {
   MissingRequired(tif,
    isTiled(tif)? "TileOffsets": "StripOffsets");
   goto bad;
  }
 }

The code above works only for strips. The file in the bugreport is missing the TileByteCounts tag, actually, as it's pseudo-tiled. However, it turns out that LibTiff doesn't actually check the presence of the ByteCounts tag at all for tiled TIFFs, resulting in much the same behaviour. This might cause other codecs to blow up with a NULL dereferencing too, and we should check this out, keeping in mind that there's much naming confusion in LibTiff where stuff with 'strip' in its name is reused for tiles and the other way around. But for OJPEG it is unintentionally proper behaviour: there's nothing to OJPEG-specifically forgive here because the problem goes undetected independently of the compression scheme, though I'm not quite sure at this stage why or how nor how to cure it.

Inside the OJPEG decoder, following the JpegInterchangeFormat tag as primary source of information doesn't work in this case, as the tag is absent. Thus, the OJPEG decoder next tries to access the first strile (=strip or tile) trying to find the required information. This is where things blow up, the code incorrectly relies on the presence of a bytecounts array, and it's absent, thus dereferencing NULL.

The safe cure is to test for the presence of a bytecounts array, and if it's not fall back on the same robust bytecount that is used when the bytecount that is present is totally out of range. In other words, inside OJPEGReadBufferFill in tif_ojpeg.c, near the bottom of that procedure, change the handling of the osibsStrile source case like this:

   case osibsStrile:
    if (sp->in_buffer_next_strile==sp->in_buffer_strile_count)
     sp->in_buffer_source=osibsEof;
    else
    {
     sp->in_buffer_file_pos=sp->tif->tif_dir.td_stripoffset[sp->in_buffer_next_strile];
     if (sp->in_buffer_file_pos!=0)
     {
      if (sp->in_buffer_file_pos>=sp->file_size)
       sp->in_buffer_file_pos=0;
      else if (sp->tif->tif_dir.td_stripbytecount==NULL)
       sp->in_buffer_file_togo=sp->file_size-sp->in_buffer_file_pos;
      else
      {
       sp->in_buffer_file_togo=sp->tif->tif_dir.td_stripbytecount[sp->in_buffer_next_strile];
       if (sp->in_buffer_file_togo==0)
        sp->in_buffer_file_pos=0;
       else if
(sp->in_buffer_file_pos+sp->in_buffer_file_togo>sp->file_size)
        sp->in_buffer_file_togo=sp->file_size-sp->in_buffer_file_pos;
      }
     }
     sp->in_buffer_next_strile++;
    }
    break;

The file this way decodes and renders fine.

This was a mistake of mine, I hadn't thought of this situation and it resulted in a bad failure. I don't have CVS set up right now, so I hope someone will urgently apply the fix for me.

Best regards,

Joris