AWARE SYSTEMS
TIFF and LibTiff Mail List Archive

Thread

2017.11.20 17:04 "[Tiff] TIFF tag and compression registration", by Kemp Watson
2017.11.20 17:34 "Re: [Tiff] TIFF tag and compression registration", by Bob Friesenhahn
2017.11.20 21:01 "Re: [Tiff] TIFF tag and compression registration", by Kemp Watson
2018.01.16 20:08 "Re: [Tiff] Strategies for multi-core speedups", by
2018.01.16 00:15 "Re: [Tiff] Strategies for multi-core speedups", by Bob Friesenhahn
2018.01.15 21:41 "Re: [Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.15 19:17 "[Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.15 20:23 "Re: [Tiff] Strategies for multi-core speedups", by Bob Friesenhahn
2018.01.15 22:02 "Re: [Tiff] Strategies for multi-core speedups", by Olivier Paquet
2018.01.15 22:47 "Re: [Tiff] Strategies for multi-core speedups", by Craig Bruce
2018.01.15 22:38 "Re: [Tiff] Strategies for multi-core speedups", by Craig Bruce
2018.01.15 22:43 "Re: [Tiff] Strategies for multi-core speedups", by scott ribe
2018.01.15 21:46 "Re: [Tiff] Strategies for multi-core speedups", by scott ribe
2018.01.15 21:51 "Re: [Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.16 01:09 "Re: [Tiff] Strategies for multi-core speedups", by Craig Bruce
2018.01.16 18:36 "Re: [Tiff] Strategies for multi-core speedups", by
2018.01.15 23:29 "Re: [Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.16 00:24 "Re: [Tiff] Strategies for multi-core speedups", by Bob Friesenhahn
2018.01.16 00:43 "Re: [Tiff] Strategies for multi-core speedups", by Paul Hemmer
2018.01.16 00:58 "Re: [Tiff] Strategies for multi-core speedups", by scott ribe
2018.01.16 07:31 "Re: [Tiff] Strategies for multi-core speedups", by Paavo Helde
2018.01.17 02:00 "Re: [Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.24 00:33 "Re: [Tiff] Strategies for multi-core speedups", by Larry Gritz
2018.01.16 18:48 "Re: [Tiff] Strategies for multi-core speedups", by scott ribe
2018.01.16 19:01 "Re: [Tiff] Strategies for multi-core speedups", by Kemp Watson
2018.01.16 19:17 "Re: [Tiff] Strategies for multi-core speedups", by
2018.01.16 19:34 "Re: [Tiff] Strategies for multi-core speedups", by Paavo Helde
2018.01.16 19:52 "Re: [Tiff] Strategies for multi-core speedups", by Kemp Watson
2018.01.16 19:57 "Re: [Tiff] Strategies for multi-core speedups", by
2018.01.16 20:18 "Re: [Tiff] Strategies for multi-core speedups", by Kemp Watson
2018.01.16 20:20 "Re: [Tiff] Strategies for multi-core speedups", by Kemp Watson
2018.01.16 20:23 "Re: [Tiff] Strategies for multi-core speedups", by

2018.01.16 07:31 "Re: [Tiff] Strategies for multi-core speedups", by Paavo Helde

On 16.01.2018 2:58, scott ribe wrote:

On Jan 15, 2018, at 5:24 PM, Bob Friesenhahn <bfriesen@simple.dallas.tx.us> wrote:

The I/O is not necessarily serialized. When using mmap, pread/pwrite, or asynchronous I/O, the I/O can also benefit from multi-threading given that the underlying storage is a multi-disk array or SSDs using PCIe NVMe (capable of up to 1024 simultaneous I/Os).

Also, I/O calls are likely only to RAM, not to disk, thanks to caching...

With TIFFClientOpen() it is pretty easy to set up one's own I/O so that everything happens in RAM definitely, not only likely. All our libtiff reading happens via memory, either mmap-ped or obtained otherwise.

That does not mean RAM is always super fast. If the files are large enough and there are many frames then traversing them takes a lot of time because they do not fit in L3 caches. The main culprit in libtiff is TIFFSetDirectory() which always traverses the whole file. This becomes pretty slow when you have something like 500 frames. In our app we are trying to cache the known IFD offsets externally and (ab)using TIFFSetSubDirectory() for frame hopping instead.

Cheers

Paavo