LibTiff Mailing List

• TIFF and LibTiff Mailing List Archive
• December 2003

• Previous Thread
• Next Thread

• Previous by Thread
• Next by Thread

• Previous by Date
• Next by Date

Contact

The TIFF Mailing List Homepage
Archive maintained by AWare Systems

Thread

2004.01.15 17:37 "Re: [Tiff] COLORMAP and byte padding", by Andy Cave

Hi Marti.

I think that both methods are fine, depending on what you want to do. The /256 gives an equal spread of resultant values (visually better), where /257 does not. /256 is much faster than /257 (or the macro). /256 gives larger 'errors' to the floating point value (but the error is small - diff is 20/65535). When doing an equal spread, it is pretty obvious that the max 'error' is 255. The difference between the mean error is neglible. This error is also /65535, so is always pretty small. Personally I don't see the need to use one over the other (they are both pretty much as accurate as the other), so would go for simplicity and speed.

_andy.

----- Original Message -----
From: <sales@littlecms.com>

To: "Andy Cave" <andy.cave@hamillroad.com>; "Marti Maria" <sales@littlecms.com>; <tiff@remotesensing.org> Sent: Thursday, January 15, 2004 5:24 PM Subject: Re: [Tiff] COLORMAP and byte padding

Your code will produce false results. Although I'd divide by 256, I would not multiple by 256 (which is where your errors come in). You'd still need to multiple by 257 so as to re-map the range back correctly, so a16_by256 should use the same calculation as a16_by257.

Ok, I have replaced the multiplication by

a16_by256 = ((unsigned short) (a8_by256 << 8) | a8_by256 );

And the results are still:

Max error: by 256 = 255, by 257 = 128
Mean error: by 256 = 85.3333, by 257 = 64.249

I was proposing the macro:

#define RGB_16_TO_8(rgb) (BYTE) ((((rgb)*65281+8388608)>>24)&0xFF)

Because this *does* produce same results as floor(rgb / 257.0 + 0.5)

Max error: by macro = 128, by double = 128
Mean error: by macro = 64.249, by double = 64.249

Check it by yourself :-)

Regards,
Marti.

#include <stdio.h>
#include <stdlib.h>
#include <math.h>

#define RGB_16_TO_8(rgb) (unsigned char) ((((rgb)*65281+8388608)>>24)&0xFF) #define RGB_8_TO_16(rgb) (unsigned short)((((unsigned short)(rgb))<<8)|rgb))

 int main()
 {
    int i;
    unsigned char byMacro_8, byDouble_8;
    unsigned short byMacro_16, byDouble_16;
    int errorMacro, errorDouble;
    int maxMacro = 0, maxDouble = 0;
    double sumMacro=0., sumDouble=0.;

    for (i=0; i < 0xFFFF; i++) {

       byMacro_8 = (unsigned char) RGB_16_TO_8(i);
       byDouble_8 = (unsigned char) (double) floor(i/257.+.5);

       byMacro_16 = (unsigned short) RGB_8_TO_16(byMacro_8);
       byDouble_16 = (unsigned short) (double) floor(byDouble_8*257.0+0.5);

       errorMacro = abs(byMacro_16 - i);
       errorDouble = abs(byDouble_16 - i);

       if (errorMacro > maxMacro) maxMacro = errorMacro;
       if (errorDouble > maxDouble) maxDouble = errorDouble;

       sumMacro += errorMacro;
       sumDouble += errorDouble;
    }

    printf("Max error: by macro = %d, by double = %d\n", maxMacro,
maxDouble);
    printf("Mean error: by macro = %g, by double = %g\n", sumMacro / 65535.,
sumDouble / 65535.);

    return 0;
 }