
Thread
2004.01.15 17:37 "Re: [Tiff] COLORMAP and byte padding", by Andy Cave
Hi Marti.
I think that both methods are fine, depending on what you want to do. The /256 gives an equal spread of resultant values (visually better), where /257 does not. /256 is much faster than /257 (or the macro). /256 gives larger 'errors' to the floating point value (but the error is small - diff is 20/65535). When doing an equal spread, it is pretty obvious that the max 'error' is 255. The difference between the mean error is neglible. This error is also /65535, so is always pretty small. Personally I don't see the need to use one over the other (they are both pretty much as accurate as the other), so would go for simplicity and speed.
_andy.
----- Original Message -----
From: <sales@littlecms.com>
To: "Andy Cave" <andy.cave@hamillroad.com>; "Marti Maria" <sales@littlecms.com>; <tiff@remotesensing.org> Sent: Thursday, January 15, 2004 5:24 PM Subject: Re: [Tiff] COLORMAP and byte padding
Your code will produce false results. Although I'd divide by 256, I would not multiple by 256 (which is where your errors come in). You'd still need to multiple by 257 so as to re-map the range back correctly, so a16_by256 should use the same calculation as a16_by257.
Ok, I have replaced the multiplication by
a16_by256 = ((unsigned short) (a8_by256 << 8) | a8_by256 );
And the results are still:
Max error: by 256 = 255, by 257 = 128
Mean error: by 256 = 85.3333, by 257 = 64.249I was proposing the macro:
#define RGB_16_TO_8(rgb) (BYTE) ((((rgb)*65281+8388608)>>24)&0xFF)
Because this *does* produce same results as floor(rgb / 257.0 + 0.5)
Max error: by macro = 128, by double = 128
Mean error: by macro = 64.249, by double = 64.249Check it by yourself :-)
Regards,
Marti.#include <stdio.h>
#include <stdlib.h>
#include <math.h>#define RGB_16_TO_8(rgb) (unsigned char) ((((rgb)*65281+8388608)>>24)&0xFF) #define RGB_8_TO_16(rgb) (unsigned short)((((unsigned short)(rgb))<<8)|rgb))
int main()
{
int i;
unsigned char byMacro_8, byDouble_8;
unsigned short byMacro_16, byDouble_16;
int errorMacro, errorDouble;
int maxMacro = 0, maxDouble = 0;
double sumMacro=0., sumDouble=0.;for (i=0; i < 0xFFFF; i++) {
byMacro_8 = (unsigned char) RGB_16_TO_8(i);
byDouble_8 = (unsigned char) (double) floor(i/257.+.5);byMacro_16 = (unsigned short) RGB_8_TO_16(byMacro_8);
byDouble_16 = (unsigned short) (double) floor(byDouble_8*257.0+0.5);errorMacro = abs(byMacro_16 - i);
errorDouble = abs(byDouble_16 - i);if (errorMacro > maxMacro) maxMacro = errorMacro;
if (errorDouble > maxDouble) maxDouble = errorDouble;sumMacro += errorMacro;
sumDouble += errorDouble;
}printf("Max error: by macro = %d, by double = %d\n", maxMacro,
maxDouble);
printf("Mean error: by macro = %g, by double = %g\n", sumMacro / 65535.,
sumDouble / 65535.);return 0;
}