2004.01.15 17:24 "Re: [Tiff] COLORMAP and byte padding", by Marti Maria

Hi,

Your code will produce false results. Although I'd divide by 256, I would not multiple by 256 (which is where your errors come in). You'd still need to multiple by 257 so as to re-map the range back correctly, so a16_by256 should use the same calculation as a16_by257.

Ok, I have replaced the multiplication by

a16_by256 = ((unsigned short) (a8_by256 << 8) | a8_by256 );

And the results are still:

Max error: by 256 = 255, by 257 = 128
Mean error: by 256 = 85.3333, by 257 = 64.249

I was proposing the macro:

#define RGB_16_TO_8(rgb) (BYTE) ((((rgb)*65281+8388608)>>24)&0xFF)

Because this *does* produce same results as floor(rgb / 257.0 + 0.5)

Max error: by macro = 128, by double = 128
Mean error: by macro = 64.249, by double = 64.249

Check it by yourself :-)

Regards,
Marti.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

#define RGB_16_TO_8(rgb) (unsigned char) ((((rgb)*65281+8388608)>>24)&0xFF)
#define RGB_8_TO_16(rgb) (unsigned short)((((unsigned short)(rgb))<<8)|rgb))

 int main()
 {
    int i;
    unsigned char byMacro_8, byDouble_8;
    unsigned short byMacro_16, byDouble_16;
    int errorMacro, errorDouble;
    int maxMacro = 0, maxDouble = 0;
    double sumMacro=0., sumDouble=0.;

    for (i=0; i < 0xFFFF; i++) {

       byMacro_8 = (unsigned char) RGB_16_TO_8(i);
       byDouble_8 = (unsigned char) (double) floor(i/257.+.5);

       byMacro_16 = (unsigned short) RGB_8_TO_16(byMacro_8);
       byDouble_16 = (unsigned short) (double) floor(byDouble_8*257.0+0.5);

       errorMacro = abs(byMacro_16 - i);
       errorDouble = abs(byDouble_16 - i);

       if (errorMacro > maxMacro) maxMacro = errorMacro;
       if (errorDouble > maxDouble) maxDouble = errorDouble;

       sumMacro += errorMacro;
       sumDouble += errorDouble;
    }

    printf("Max error:  by macro = %d, by double = %d\n", maxMacro, maxDouble);
    printf("Mean error: by macro = %g, by double = %g\n", sumMacro / 65535., sumDouble / 65535.);

    return 0;
 }