2004.10.02 15:41 "Re: [Tiff] quad-tile", by Bob Friesenhahn

Two minor complications I foresee:

Closed-circuit testing is a tiny bit problematic when lossy storage is involved. I'm not just thinking jpeg, even mere YCbCr subsampled images, whatever compression scheme, are lossy. This tiny problem can perhaps be solved by having the data include tolerance levels for the closed-circuit testing?

For many of GraphicsMagick's tests we use an image comparison function which computes statistical error values. For algorithms which should not produce error, the allowed error is zero, but for algorithms which produce error (e.g. JPEG) we manually inspect the result to ensure that it seems ok, and then set the maximum allowed error value to slighly higher than the actual result. The error values include the maximum pixel error (in case a single pixel is wrong), and the total mean error.

Provided that the libtiff test suite provides an image compare function, the allowed degree of error may be included with the test specification.

Note that this does not verify that the file format was correct, only that when the file is read back, the results are equivalent.

If there is no master reference image, then testing based on observed error is limited since only the ability to reproduce locally-prepared results can be evaluated, and the locally-prepared reference results may be wrong.

For operations which must always produce a certain result (the case for most libtiff operations), we use a checksum (SHA-128) and compare the resulting checksum with a checksum included with the test. This is a very solid way to prepare a test, but is a bit tedious since each result must be validated by the person who writes the test.

Bob

======================================
Bob Friesenhahn
bfriesen@simple.dallas.tx.us
http://www.simplesystems.org/users/bfriesen