LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
November 2008
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2008.11.14 22:03 "Re: Memory leak (TIFFOpen, TIFFReadTile)?", by Phillip Crews
Ilkka, For Windows: Don't use Task manager to monitor a process's memory use. Get Process Explorer from www.sysitnernals.com, which has been bought by Microsoft and redirects there now. Add a column for and watch the "Private bytes" for the process. This is the committed, writeable, non-shared pages used by a process and is a quick indicator to watch for when either memory allocation patterns go wonky or memory leaks occur. For large memory allocations, you may want to switch to VirtualAlloc() and VirtualFree() rather than malloc/free. VirtualFree() actually returns the pages to the O/S rather than leave them hanging around for future mallocs. (I believe some newer Windows C runtimes will automatically use VirtualAlloc/Free under some conditions, but I've had my own memory management library for so long I'm not sure how it works. realloc() can interfere with this too, since pages have to be wholly free before they can be released.) Note that there is no VirtualRealloc(); but realloc() itself is sometimes an easy way to cause runaway page allocations and unusable holes in the process address space. The often unpredictable distribution of sizes and lifetimes of memory allocations contribute to heap fragmentation; there's also the low fragmentation heap in XP and later, but that's more geared to programs that allocate memory in smaller sizes. It attempts to minimize unusable holes in the heap. See: http://msdn.microsoft.com/en-us/library/aa366750(VS.85).aspx for more information. Sorry if that's too rambling. Been a long week. - Phillip Ilkka Korpela wrote: > Hi > > I have some huge TIFFs, size upto 2 gB, 1, 3 or 4 bands, 2 bytes > per band, tiled. Huge aerial image-scanner mats with the ads80 > instrument, > > I'm trying to make untiled greyscale/rgb pixel-interleaved > raw-image versions, I am working with MS Vis studio 6 and C > in an XP machine. > > I believe I am using the 3.6.1 version of libbtiff. libtiff.lib > is from Feb 2004. > > I now have a working solution for the task of untiling and writing > a bip-image, but there is a memory leak somewhere? I am reading > and converting a 1517454784 bytes file. XP's task manager tells me > the process is gradually consuming nearly 1.3 Gbytes at the end. > > I tried with _TIFFmalloc() and malloc() but it does not make > a difference - memory consumption goes up. Any suggestions out there? > Here is the simple code (include statements not there),: > > ilkka > > struct NRGB16 { > uint16 N; > uint16 R; > uint16 G; > uint16 B; > }; > > int main(int argc, char* argv[]) > { > // Open the 1-2 GByte block file (ads40-imagery) > TIFF* tif = TIFFOpen("c:\\temp\\ads40\\O-08230753NRGBN00AL2_0_0.tif", > "r"); > FILE *fOut; > uint32 i; > if (tif) { > uint32 imageWidth, imageLength; > uint32 tileWidth, tileLength; > uint32 x, y; > NRGB16* buf; > unsigned char Red, Green, Nir; > int Column, Row, OldRow; > // Query the Image & Tile dimensions > TIFFGetField(tif, TIFFTAG_IMAGEWIDTH, &imageWidth); > TIFFGetField(tif, TIFFTAG_IMAGELENGTH, &imageLength); > TIFFGetField(tif, TIFFTAG_TILEWIDTH, &tileWidth); > TIFFGetField(tif, TIFFTAG_TILELENGTH, &tileLength); > // Open the output file, 3-channel viewable version of the aerial image > fOut=fopen("c:\\temp\\ads40\\O-08230753NRGBN00AL2_0_0_CIR_uint8.raw","wb"); > > OldRow = -1; > // Allocate storage for the 4-channel, 2 bytes per channel image TILE > buf = (NRGB16*)malloc(tileLength*tileWidth*4*2); > // Start untiling the image > for (y = 0; y < imageLength; y += tileLength) > for (x = 0; x < imageWidth; x += tileWidth) > { > i=TIFFReadTile(tif, &buf[0], x, y, 0, 0); > for (i=0;i < tileLength*tileWidth;i=i+1) > { // Carry out simple 16-8 bit reduction > Nir = 255; Green = 255; Red = 255; > if ( double(buf[i].N)/20 <= 255) Nir = unsigned char(buf[i].N/20); > if ( double(buf[i].R)/15 <= 255) Red = unsigned char(buf[i].R/15); > if ( double(buf[i].G)/15 <= 255) Green = unsigned > char(buf[i].G/15); > // Compute the global col,row coordinates in the untiled image > Column = x + (i % tileWidth); > Row = y + (i / tileWidth); > // Check if we started a new row, set the file pointer accordingly > if (Row != OldRow) {fseek(fOut,Row*imageWidth*3+Column*3, SEEK_SET);} > OldRow = Row; > // Write the RGB-bytes > fwrite(&Nir, 1, 1, fOut); > fwrite(&Green, 1, 1, fOut); > fwrite(&Red, 1, 1, fOut); > } > // Memory leak? Try reallocating the buf > buf = (NRGB16*)realloc(buf,tileLength*tileWidth*4*2); > // Watch for memory sizes of these blocks (showing no leak here) > printf("%d %d\n",_msize(tif),_msize(buf)); > } > // Finally free everything > free(buf); > fclose(fOut); > TIFFClose(tif); > } > return 0; > } > > > > > > > > > >