2015.01.07 10:53 "Re: [Tiff] Fast TIFF Reading on Windows", by Roger Leigh

On Mon, Jan 05, 2015 at 09:05:17AM -0600, Bob Friesenhahn wrote:

What would be interesting is to use multiple threads to read a single TIFF image. This would be particularly advantageous for BigTIFF. is using C++11, so threading support is baked in. easy, keeping in mind   My library  Its quite

that threads are evil.

For libtiff, one would need to find a good cross platform C threading library.

For a single file, the approach could be similar: a thread safe list of strips, and each thread gets the next strip and reads it in.

>Actually, this would be a perfect job for OpenMP. you wouldn't need a thread safe list, just divide up the   And >strips evenly across threads.

Libtiff does not use any locking or use thread-specific contexts so I think it is definitely not safe for using code to request multiple strips from the same image at a time with the same tiff handle using multiple threads. It should be mostly safe to read multiple images at one time (different tiff handles) using multiple threads. It should be safe to open the same input TIFF file several times so that there is a different handle for each reading thread. The threads would then need to agree on which strips/tiles they would read.

It would be good for libtiff to support multiple strip/tile reads at a time using the same handle in different threads. This would require thread-specific context and some locking.

Regardless, in my own experience, libtiff strip/tile reads are quite fast and the bottleneck is normally in the using code. Using a semaphore to protect tiff handle use would allow multiple threads to use libtiff to obtain strips/tiles and threads can be dealing with data returned while one thread is getting the next strip/tile. This could result in a 2X to 4X speed-up.

I have wrapped libtiff to use a mutex here: https://github.com/openmicroscopy/bioformats/tree/develop/cpp/lib/ome/bioformats/tiff

The main point of interest is the sentry object used to get exclusive access to libtiff using RAII: https://github.com/openmicroscopy/bioformats/blob/develop/cpp/lib/ome/bioformats/tiff/Sentry.cpp

The major annoyance is the use of a global error handler which precludes the multiple callers being able to accurately report errors. I know the use of a mutex is a bit of a blunt instrument, but it's needed to save and restore the error handler and throw exceptions correctly in the calling thread. If libtiff could make the error reporting use thread- local structures, that would avoid the need to have threads contending on a mutex or other synchronisation primitive just to get an error message. If it was also thread-safe internally that would also be a huge improvement.

The above wrapper also provides type-safe field access and a few other features. If there is any interest in a robust C++ API for libtiff, please do feel free to make use of any of this work.

Regards,
Roger

--
  .''`. Roger Leigh

 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools

   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800