2005.01.09 05:46 "Re: [Tiff] JPEG (T.30-E, fax) in TIFF", by Lee Howard

I strongly feel that you really should state the correct information

I couldn't agree with you more. :-)

** StripByteCounts/StripOffsets **

single value of StripOffsets and StripByteCounts

fax000003467.tif: Warning, incorrect count for field "StripOffsets" (3, expecting 1); tag trimmed.
fax000003467.tif: Warning, incorrect count for field "StripByteCounts" (3, expecting 1); tag trimmed.

It seems to be affected by the SAMPLESPERPIXEL tag. If I change the SAMPLESPERPIXEL tag to 1 I only get one single value of StripOffsets and StripByteCounts. If I just change the SAMPLESPERPIXEL tag to 3 then I get three values in each of StripOffsets and StripByteCounts. I don't know if I'm doing something wrong or if there is a libtiff bug in this.

This may be related to the PlanarConfiguration setting, if SamplesPerPixel influences this. Have you tried setting PlanarConfiguration, even though you've found LibTiff to use the proper default?

I've now tried setting TIFFTAG_PLANARCONFIG as PLANARCONFIG_CONTIG explicitly, and unfortunately there is no improvement. As best I can tell, it looks like a bug in libtiff creating three values for StripByteCounts and StripOffsets when it is not supposed to. I suppose that the next step to take on this one is to file a bug report?

** More general **

Unfortunately, the YCBCRSUBSAMPLING tag appears to be ignored by tiffcp when the photometric tag is not set for YCbCr.

In my opinion, you should give up on trying to build your files such that they are handled by tiffcp, in this first stage. If this attempt involves 'forging' the Photometric indication, it's bound to lead to trouble later in the process, anyway, and it build you incorrect TIFF files that you cannot hope will ever get supported any better.

Okay, you've convinced me. I'll stop trying to use tiffcp as a gauge for TIFF correctness... as obviously it is broken in some degrees, when handling this kind of TIFF. I was trying to use tiffcp in that way, though, because libtiff is really the only library that HylaFAX relies on for TIFF processing.

Instead, try to build a correct TIFF.

Okay, following your advice as best as I can I have come up with:

http://gateway.howardsilvan.com/fax000003470.tif

Here's what tiffdump tells me:

fax000003470.tif:
Magic: 0x4949 <little-endian> Version: 0x2a
Directory 0: offset 266952 (0x412c8) next 0 (0)
SubFileType (254) LONG (4) 1 <2>
ImageWidth (256) SHORT (3) 1 <1728>
ImageLength (257) SHORT (3) 1 <2128>
BitsPerSample (258) SHORT (3) 3 <8 8 8>
Compression (259) SHORT (3) 1 <7>
Photometric (262) SHORT (3) 1 <10>
FillOrder (266) SHORT (3) 1 <2>
ImageDescription (270) ASCII (2) 11 <3604278160\0>
Make (271) ASCII (2) 62 <LT V.92 1.0 MT5634ZBA-V9 ...>
Model (272) ASCII (2) 62 <LT V.92 1.0 MT5634ZBA-V9 ...>
StripOffsets (273) LONG (4) 3 <133480 0 0>
Orientation (274) SHORT (3) 1 <1>
SamplesPerPixel (277) SHORT (3) 1 <3>
RowsPerStrip (278) LONG (4) 1 <4294967295>
StripByteCounts (279) LONG (4) 3 <133472 0 0>
XResolution (282) RATIONAL (5) 1 <204>
YResolution (283) RATIONAL (5) 1 <196>
PlanarConfig (284) SHORT (3) 1 <1>
ResolutionUnit (296) SHORT (3) 1 <2>
Software (305) ASCII (2) 27 <HylaFAX (tm) Version 4.2 ...>
DateTime (306) ASCII (2) 20 <2005:01:08 20:58:55\0>
HostComputer (316) ASCII (2) 16 <gollum.x101.com\0>
YCbCrSubsampling (530) SHORT (3) 2 <1 1>

As you can see, the StripOffsets and StripByteCounts contain three values each rather than one. Looks to be a bug in libtiff as I mentioned above. Also, notice that the HylaFAX-specific tags (SGI) are not present even though they were "set" with TIFFSetField(), as I mention below.

You seem to have been on that track before, (almost?) successfully. Next, try extending tiffcp, possibly Andrey might be persuaded to help you in that area. Then, all will be well, and your files wil be correct.

Looks like I'm going to need Andrey's help on getting tiffcp enhanced to support PHOTOMETRIC_ITULAB. I attempted to test this TIFF file in Adobe Photoshop 5.5, but Photoshop was unable to open it due to a "colorspace" error.

If you want, I can maybe help out by double-checking a correctly build TIFF with independent non-LibTiff software. That's only going to be an additional indication of the TIFF being valid, not a defenite proof, and I'll need to extend that independent software to deal with it. But at least it is an additional indication. Let me know.

If you can do what you can to verify the accuracy of the TIFF I linked above I would appreciate it. But, at the same time, I don't want you to do anything that you don't want to do. So, don't let me abuse your kindness.

If LibTiff doesn't write TIFFTAG_FAXRECVPARAMS and TIFFTAG_FAXRECVTIME tags for you, I believe this too, you'll probably need to ask Andrey's help.

Yeah. Looks like I'll need to file a bug report on this one, also. Or is Andrey listening in and does he work on these kinds of things without a bug report?

** The code **

The repeated calling of TIFFWriteRawStrip seems very extremely weird. Could you try changing the DNL issue correction stuff to work on your JPEG data buffer, before calling TIFFWriteRawStrip, and next making one single TIFFWriteRawStrip call instead of the current three calls and the TIFFSetWriteOffset call.

Well, the situation is like this... the fax sender will deliver to HylaFAX the image in a byte-by-byte stream of data that is interrupted by fax protocol every 64KB. Because the SOF marker comes before the DNL marker, and because the SOF marker does not indicate the image length, it is requisite that, when the DNL marker is encountered, that we "go back" and "fix" the SOF marker to indicate the page length. So, we have to buffer all of that data *somewhere*, and because we don't know how much total data we'll actually receive I didn't want to get into any risks of buffer overflows using RAM, and I didn't want to have to complicate the process by opening up a temp file on disk (yet another file to open and unlink later), and so chose to do what I did - storing the data as-it-comes in the already-open TIFF and then overwriting it later.

Only the "overwriting it later" part is new to HylaFAX (well, and not even that, because we "overwrite" the TIFF data on page retransmissions - when HylaFAX rejects a page and requires it to be resent). We constantly write single-strip TIFFs (the usual MH, MR, or MMR compressions) using multiple calls to TIFFWriteRawStrip, and there seems to be no problem with this. All TIFF viewers that I've tested have been happy with the resulting files.

Lee.