2016.01.27 18:51 "Re: [Tiff] OpenMP enabled libtiff", by Larry Gritz

It may be instructive to look at what OpenEXR did (technically, libIlmImf). If you use the API calls that read or write more than one scanline or thread at a time, it will use a thread pool to work in parallel on the compression/decompression or data conversion of the separate compressed chunks that comprise that section of the image, which often is the bulk of the wall clock time. If you have threads to spare, it really does speed up the reads/writes almost linearly (it's the disk I/O that is still serialized). Of course, if you use the API calls to read/write a single scanline or tile, the threads don't do anything.

This turns out to be a very valuable feature, and in many applications that tend to read or write whole images at once, can tremendously speed up throughput. For certain access patterns and thread availability, it can be used to read or write an exr file many times faster than the equivalent TIFF file.

> On Jan 27, 2016, at 9:00 AM, Bob Friesenhahn <bfriesen@simple.dallas.tx.us> wrote:

>
> Date: Tue, 26 Jan 2016 11:10:59 -0600 (CST)
> From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
> Subject: Re: [Tiff] OpenMP enabled libtiff
> To: Aaron Boxer <boxerab@gmail.com>
> Cc: TIFF mailing list <tiff@lists.maptools.org>
> Message-ID:
> <alpine.GSO.2.01.1601261103040.24533@freddy.simplesystems.org>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

Yes, so you don't think there is a "quick and dirty" solution for speeding up compressed reading?

For my purposes, I am only interested in reading uncompressed files, so TIFFClientOpen

on a memory buffer should be good enough.

Libtiff is very fast at reading from uncompressed files and I don't see much performance penalty as compared with accessing files using system APIs.

But, I still think it is a shame to leave the library single threaded in this multi-core world.

It is just finding the time to work on improvements.

It is likely that adding threading would just make it slower for many common cases and it adds a link-time dependency which otherwise does not exist. It is much better to enable threaded applications to make better use of libtiff when accessing one file. What works in one use case may be totally inappropriate for another. Preserve the application writer's freedom to chose the best solution for the application. Most importantly, we can not afford to break the existing libtiff API and ABI.

--
Larry Gritz
lg@larrygritz.com