AWare Systems, Home TIFF and LibTiff Mail List Archive

LibTiff Mailing List

TIFF and LibTiff Mailing List Archive
January 2016

Previous Thread
Next Thread

Previous by Thread
Next by Thread

Previous by Date
Next by Date


The TIFF Mailing List Homepage
Archive maintained by AWare Systems

New Datamatrix section

Valid HTML 4.01!


2016.01.25 18:25 "[Tiff] OpenMP enabled libtiff", by Aaron Boxer
2016.01.26 09:36 "Re: [Tiff] OpenMP enabled libtiff", by Mat Maher
2016.01.26 09:52 "Re: [Tiff] OpenMP enabled libtiff", by Even Rouault
2016.01.26 14:22 "Re: [Tiff] OpenMP enabled libtiff", by Fred Rothganger

2016.01.26 09:52 "Re: [Tiff] OpenMP enabled libtiff", by Even Rouault

Le mardi 26 janvier 2016 10:36:16, Mat Maher a écrit:

Hello List,

Is anyone interested in having OpenMP support in libtiff?

Is this feasible? planning on adding this feature for another I am project I am working on, so I am trying to gauge interest.

libTiff performance is governed by two factors:

  1. . Performance of disk read/writes
  2. . Performance of compression/decompression routines (eg CPU)

The latter would benefit enormously from multi-threaded implementation, BUT, will ultimately be throttled by the former.

There are a few simple things which could be done to improve matters:

  1. . Buffer the entire file in memory for processing. HOWEVER, this would only work with a user switch input, as it's always possible the application TIFFs would be larger than available memory. 'Blasting' an entire file to/from disk maximises the transfer rate due to seek-times and DMA-setup latencies. In this scenario, you will end up reading/writing at the line-rate of the disk: which about as fast as you can get (and actually, for write operations, shows SATA as the faster medium for file sizes > SSD cache)
  2. . Implement thread-per-strip CPU processing.

For these changes, I actually wouldn't use OpenMP. The TIFF strip arrangement is a perfect division for multi-threading utilisation. Trying to multi-thread at a lower granularity than this is likely to yield lower performance due to the compression/decompression libraries which are largely sequential in operation.

For what is worth, I've experimented some time ago in the GDAL TIFF driver, multi-treaded compression. This works like this:

  1. Uncompressed strips/tiles to write are sent to a queue
  2. Worker threads fetch that uncompressed data from the queue, creates a in- memory TIFF file of the dimension of the tile/strip with the desired compression options, and extract the resulting compressed data from the writen in-memory TIFF file.
  3. The main thread collects the compressed data and write it to the main TIFF file with TIFFWriteRawTile / TIFFWriteRawStrip
  4. This gives performance boost on slow compression methods such as Deflate.


Spatialys - Geospatial professional services