TIFF and LibTiff Mail List Archive

[...]
[...]

# 2004.01.14 19:48 "Re: [Tiff] COLORMAP and byte padding", by Andy Cave

Hi Marti.

Your code will produce false results. Although I'd divide by 256, I would not multiple by 256 (which is where your errors come in). You'd still need to multiple by 257 so as to re-map the range back correctly, so a16_by256 should use the same calculation as a16_by257.

_andy.

----- Original Message -----

From: "Marti Maria" <sales@littlecms.com> To: "Andy Cave" <andy.cave@hamillroad.com>; <tiff@remotesensing.org> Sent: Wednesday, January 14, 2004 7:36 PM Subject: Re: [Tiff] COLORMAP and byte padding

>

Here is a small program to check the error by using both methods, shifting 8, (same as *, / by 256) and the "good" method, which is by 257. But this should be not so surprising. If you are scaling a value with range 0.. 255 to a range 0..65535, the factor would be 65535 / 255, and that equals to 257 :-)

--- Cut ---

#include <stdio.h>
#include <stdlib.h>

int main()
{
int i;
unsigned char a8_by256, a8_by257;
unsigned short a16_by256, a16_by257;
int error256, error257;
int max256 = 0, max257 = 0;
double sum256=0., sum257=0.;

for (i=0; i < 0xFFFF; i++) {

a8_by256 = (unsigned char) (i >> 8);
a8_by257 = (unsigned char) (double) floor(i / 257. + .5);

a16_by256 = ((unsigned short) a8_by256 << 8);
a16_by257 = ((unsigned short) a8_by257 << 8) | a8_by257;

error256 = abs(a16_by256 - i);
error257 = abs(a16_by257 - i);

if (error256 > max256) max256 = error256;
if (error257 > max257) max257 = error257;

sum256 += error256;
sum257 += error257;

}

printf("Max error:  by 256 = %d, by 257 = %d\n", max256, max257);
printf("Mean error: by 256 = %g, by 257 = %g\n", sum256 / 65535., sum257 / 65535.);

return 0;
}

--- Cut --

From: "Andy Cave" <andy.cave@hamillroad.com> To: "Marti Maria" <sales@littlecms.com>; <tiff@remotesensing.org> Sent: Wednesday, January 14, 2004 8:02 PM

Hi Marti.

What "significant errors"? Please show us. As far as I know there are no significant errors - as I said in my previous post, 0 to 255 -> 0, 256 to 511 -> 1, etc... gives an even spread reduction. How else would one want to map these numbers? For example what do you think 127 should map to, or 128, or 255, 256, or 511,512, etc...?

(rgb * 65281) = (rgb * (65536 - 255)) = rgb * 65536 - rgb * 255.

If I understand this correctly, all it does is map 0 to 128 -> 0, 129 to 385 -> 1, etc... which gives a non-even spread (where the end 'bands' are half all the other bands). Why would anyone prefer this (slower method)?

_andy.

For the perfectionists who are wondering: multiplying by 257 works perfectly. In effect, it makes the least significant byte a copy of the most significant byte of the 16-bit value. In C:

uint16val = ((uint8val << 8) | uint8val);

Note that when reducing the 16-bit value to 8-bit, you could divide by 257 and round, but it's way faster to simply take the upper byte (the same as an integer division by 256, truncating everything after the decimal point). Don't divide by 256 and round, because you will end up 1 too high in 50% of the cases. In C:

uint8val = (uint16val >> 8);

Yep, that is fast, but unfortunately can give significant errors, since that is same than dividing by 256.

Here are a couple of macros that does the trick.

#define RGB_8_TO_16(rgb) (WORD) ((((WORD) (rgb)) << 8)|(rgb))

#define RGB_16_TO_8(rgb) (BYTE) ((((rgb) * 65281 + 8388608) >> 24) & 0xFF)

Marti Maria
The little cms project
http://www.littlecms.com
marti@littlecms.com