LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
January 2004
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2004.01.14 19:36 "Re: COLORMAP and byte padding", by Marti Maria
Hi,
Here is a small program to check the error by using both methods,
shifting 8, (same as *, / by 256) and the "good" method, which is
by 257. But this should be not so surprising. If you are scaling a value
with range 0 .. 255 to a range 0..65535, the factor would be
65535 / 255, and that equals to 257 :-)
--- Cut ---
#include <stdio.h>
#include <stdlib.h>
int main()
{
int i;
unsigned char a8_by256, a8_by257;
unsigned short a16_by256, a16_by257;
int error256, error257;
int max256 = 0, max257 = 0;
double sum256=0., sum257=0.;
for (i=0; i < 0xFFFF; i++) {
a8_by256 = (unsigned char) (i >> 8);
a8_by257 = (unsigned char) (double) floor(i / 257. + .5);
a16_by256 = ((unsigned short) a8_by256 << 8);
a16_by257 = ((unsigned short) a8_by257 << 8) | a8_by257;
error256 = abs(a16_by256 - i);
error257 = abs(a16_by257 - i);
if (error256 > max256) max256 = error256;
if (error257 > max257) max257 = error257;
sum256 += error256;
sum257 += error257;
}
printf("Max error: by 256 = %d, by 257 = %d\n", max256, max257);
printf("Mean error: by 256 = %g, by 257 = %g\n", sum256 / 65535., sum257 / 65535.);
return 0;
}
--- Cut --