LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
Previous by Thread
Next by Thread
Previous by Date
Next by Date
The TIFF Mailing List Homepage
Archive maintained by AWare Systems
2016.01.26 09:52 "Re: [Tiff] OpenMP enabled libtiff", by Even Rouault
Le mardi 26 janvier 2016 10:36:16, Mat Maher a écrit:
Is anyone interested in having OpenMP support in libtiff?
Is this feasible? planning on adding this feature for another I am project I am working on, so I am trying to gauge interest.
libTiff performance is governed by two factors:
- . Performance of disk read/writes
- . Performance of compression/decompression routines (eg CPU)
The latter would benefit enormously from multi-threaded implementation, BUT, will ultimately be throttled by the former.
There are a few simple things which could be done to improve matters:
. Buffer the entire file in memory for processing. HOWEVER, this would only work with a user switch input, as it's always possible the application TIFFs would be larger than available
memory. 'Blasting' an entire file to/from disk maximises the transfer rate due to seek-times and DMA-setup latencies. In this scenario, you will end up reading/writing at the line-rate
of the disk: which about as fast as you can get (and actually, for write operations, shows SATA as the faster medium for file sizes > SSD cache)
- . Implement thread-per-strip CPU processing.
For these changes, I actually wouldn't use OpenMP. The TIFF strip arrangement is a perfect division for multi-threading utilisation. Trying to multi-thread at a lower granularity than
this is likely to yield lower performance due to the compression/decompression libraries which are largely sequential in operation.
For what is worth, I've experimented some time ago in the GDAL TIFF driver, multi-treaded compression. This works like this:
- Uncompressed strips/tiles to write are sent to a queue
Worker threads fetch that uncompressed data from the queue, creates a in- memory TIFF file of the dimension of the tile/strip with the desired compression options, and extract the
resulting compressed data from the writen in-memory TIFF file.
- The main thread collects the compressed data and write it to the main TIFF file with TIFFWriteRawTile / TIFFWriteRawStrip
- This gives performance boost on slow compression methods such as Deflate.
Spatialys - Geospatial professional services