2018.01.16 20:08 "Re: [Tiff] Strategies for multi-core speedups", by

I did experiment with HTML5 + webGL in the past and I actually ran into draw call limitations by the browsers. But looking back I'm sure I did things quite incorrectly setting images up. But even then the thought did cross our minds and we basically came to the same conclusions. We may be getting some 10Gb network capabilities soon, but I'm not banking on it so I went the desktop application route.

Thanks againm

Joseph Maniaci
Staff Software Engineer
RPPS

From:   Kemp Watson <kemp@objectivepathology.com>
To:     Paavo Helde <paavo@osa.pri.ee>
Cc:     tiff@lists.maptools.org
Date:   01/16/2018 01:02 PM

Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]
Sent by:        tiff-bounces@lists.maptools.org

Also Joseph, you might find yourself limited by network bandwidth or BDP long before you are limited by application threading, good idea to think about network simulation (if relevant) before detailed design.

On Tue, Jan 16, 2018 at 2:34 PM, Paavo Helde <paavo@osa.pri.ee> wrote:

On 16.01.2018 20:36, Joe.Maniaci@ricoh-usa.com wrote:

I am a complete newbie to digital imagery and TIFFs in general, so please forgive any ignorance.

I'm an automated test tool developer and I am actually going to start
working on a viewer for the TIFFs that my company generates, to
potentially include relying on the bigtiff format and this is the thing
I've spent most of my time thinking about. I've come to the conclusion
that for the images we generate, up to 10GB uncompressed(soon to be 40GB),
that I will need to preprocess a single TIFF into multiple TIFFs
containing intelligently placed tiles just to get around the I/O issue. By
intelligently placed, I mean that if a user were to scroll their view
window to the right, and five tiles(vertically aligned) had to be loaded
into memory, I could theoretically have those files intelligently located
in different TIFF files(subTIFFs?) that would allow multiple threads to
operate on. So each thread only had to find one tile. So for my tool it
would be a staggering amount of pre-processing, but real time manipulation
is extremely fluid for the user which is basically my number one

requirement.

So for a case like mine, thread overhead is nothing to worry about. That and the more I read about the C++11 thread library, the less I worry. I can just leave it to the library to determine how many threads should operate in my 4-core environments, or my 16-core environments. At least that's the impression I've garnered.

If all you want is a viewer then you are limited by the screen sizes which are typically not more than ca 2000x2000 pixels. And this is pretty small image nowadays, there is no need to multithread anything, you could just read the needed tiles from the TIFF file in real time and maybe add a small cache of the read tiles so you don't need to read them again if the user pans left 1 pixel.

For zooming out the TIFF file should have different frames of the same image stored in decreasing resolutions so that a suitable resolution can be chosen for display - these are called pyramidal tiffs. The TIFF standard suggests that such varying resolution images should be stored as sub-IFD-s but as many viewers do not support sub-IFD-s they are typically stored just in standard IFD-s, and some XML piece in the first IFD provides the details. I am sure there are multiple variants of such pyramidal tiff formats, one from our company as well, and of course there are multiple readers supporting such tiff files.

hth
Paavo

https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.maptools.org_mailman_listinfo_tiff&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=WLZ0J37-6mmRsbEmF36lp3XgYKTTDercqcosYw1nJns&s=wO0wn6GcM--MZsgJUVNSMq-kStUGc-sLcgZqlQ-Oz74&e=

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.remotesensing.org_libtiff_&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=WLZ0J37-6mmRsbEmF36lp3XgYKTTDercqcosYw1nJns&s=qTeqEySSaE0asy3GUU2r61dgmAm_ZVfcdQMhnB4qbrQ&e=