LibTiff Mailing List
TIFF and LibTiff Mailing List Archive
January 2015
Previous Thread
Next Thread
Previous by Thread
Next by Thread
Previous by Date
Next by Date
Contact
The TIFF Mailing List Homepage
This list is run by Frank Warmerdam
Archive maintained by AWare Systems
Thread
2015.01.07 10:53 "Re: Fast TIFF Reading on Windows", by Roger Leigh
On Mon, Jan 05, 2015 at 09:05:17AM -0600, Bob Friesenhahn wrote: > On Mon, 5 Jan 2015, Aaron Boxer wrote: > > > >What would be interesting is to use multiple threads to read a single > >TIFF image. This would be particularly > >advantageous > >for BigTIFF. My library is using C++11, so threading support is baked > >in. Its quite easy, keeping in mind > >that threads are evil. > >For libtiff, one would need to find a good cross platform C threading > >library. > > > >For a single file, the approach could be similar: a thread safe list of > >strips, and each thread gets the next > >strip and reads it in. > > > >Actually, this would be a perfect job for OpenMP. And you wouldn't need > >a thread safe list, just divide up the > >strips evenly across threads. > > Libtiff does not use any locking or use thread-specific contexts so > I think it is definitely not safe for using code to request multiple > strips from the same image at a time with the same tiff handle using > multiple threads. It should be mostly safe to read multiple images > at one time (different tiff handles) using multiple threads. It > should be safe to open the same input TIFF file several times so > that there is a different handle for each reading thread. The > threads would then need to agree on which strips/tiles they would > read. > > It would be good for libtiff to support multiple strip/tile reads at > a time using the same handle in different threads. This would > require thread-specific context and some locking. > > Regardless, in my own experience, libtiff strip/tile reads are quite > fast and the bottleneck is normally in the using code. Using a > semaphore to protect tiff handle use would allow multiple threads to > use libtiff to obtain strips/tiles and threads can be dealing with > data returned while one thread is getting the next strip/tile. This > could result in a 2X to 4X speed-up. I have wrapped libtiff to use a mutex here: https://github.com/openmicroscopy/bioformats/tree/develop/cpp/lib/ome/bioformats/tiff The main point of interest is the sentry object used to get exclusive access to libtiff using RAII: https://github.com/openmicroscopy/bioformats/blob/develop/cpp/lib/ome/bioformats/tiff/Sentry.cpp The major annoyance is the use of a global error handler which precludes the multiple callers being able to accurately report errors. I know the use of a mutex is a bit of a blunt instrument, but it's needed to save and restore the error handler and throw exceptions correctly in the calling thread. If libtiff could make the error reporting use thread- local structures, that would avoid the need to have threads contending on a mutex or other synchronisation primitive just to get an error message. If it was also thread-safe internally that would also be a huge improvement. The above wrapper also provides type-safe field access and a few other features. If there is any interest in a robust C++ API for libtiff, please do feel free to make use of any of this work. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `- GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800