2016.01.26 09:52 "Re: [Tiff] OpenMP enabled libtiff", by Even Rouault
Le mardi 26 janvier 2016 10:36:16, Mat Maher a écrit:
Is anyone interested in having OpenMP support in libtiff?
Is this feasible? planning on adding this feature for another I am project I am working on, so I am trying to gauge interest.
libTiff performance is governed by two factors:
- . Performance of disk read/writes
- . Performance of compression/decompression routines (eg CPU)
The latter would benefit enormously from multi-threaded implementation, BUT, will ultimately be throttled by the former.
There are a few simple things which could be done to improve matters:
- . Buffer the entire file in memory for processing. HOWEVER, this would only work with a user switch input, as it's always possible the application TIFFs would be larger than available memory. 'Blasting' an entire file to/from disk maximises the transfer rate due to seek-times and DMA-setup latencies. In this scenario, you will end up reading/writing at the line-rate of the disk: which about as fast as you can get (and actually, for write operations, shows SATA as the faster medium for file sizes > SSD cache)
- . Implement thread-per-strip CPU processing.
For these changes, I actually wouldn't use OpenMP. The TIFF strip arrangement is a perfect division for multi-threading utilisation. Trying to multi-thread at a lower granularity than this is likely to yield lower performance due to the compression/decompression libraries which are largely sequential in operation.
For what is worth, I've experimented some time ago in the GDAL TIFF driver, multi-treaded compression. This works like this:
- Uncompressed strips/tiles to write are sent to a queue
- Worker threads fetch that uncompressed data from the queue, creates a in- memory TIFF file of the dimension of the tile/strip with the desired compression options, and extract the resulting compressed data from the writen in-memory TIFF file.
- The main thread collects the compressed data and write it to the main TIFF file with TIFFWriteRawTile / TIFFWriteRawStrip
- This gives performance boost on slow compression methods such as Deflate.
Spatialys - Geospatial professional services