2018.01.16 20:23 "Re: [Tiff] Strategies for multi-core speedups", by

I'll definitely look into it a bit more.

Thanks,

Joseph Maniaci
Staff Software Engineer
RPPS

From:   Kemp Watson <kemp@objectivepathology.com>

To:     Joe.Maniaci@ricoh-usa.com
Cc:     Paavo Helde <paavo@osa.pri.ee>, tiff@lists.maptools.org, 

        tiff-bounces@lists.maptools.org

Date:   01/16/2018 01:20 PM

Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]

Actually, to follow that up, required bandwidth is pretty easy to calculate, based on your screen size, % of screen refreshed on a pan or zoom, required response time, and compression ratio.

Same for BDP limits, with packet size and packets in flight.

Kemp

On Tue, Jan 16, 2018 at 3:18 PM, Kemp Watson <kemp@objectivepathology.com> wrote:

We found that we were getting very reasonable interactivity in web browsers, with no 'fancy' WebGL, WebSockets, WebRTC, etc, just raw XMLHTTP, at about 0.5-0.6 Mbps per user. That's for viewing at screen pixel data sizes.

Kemp

On Tue, Jan 16, 2018 at 3:08 PM, <Joe.Maniaci@ricoh-usa.com> wrote:

I did experiment with HTML5 + webGL in the past and I actually ran into draw call limitations by the browsers. But looking back I'm sure I did things quite incorrectly setting images up. But even then the thought did cross our minds and we basically came to the same conclusions. We may be getting some 10Gb network capabilities soon, but I'm not banking on it so I went the desktop application route.

Thanks againm

Joseph Maniaci
Staff Software Engineer
RPPS

From:   Kemp Watson <kemp@objectivepathology.com>
To:     Paavo Helde <paavo@osa.pri.ee>
Cc:     tiff@lists.maptools.org
Date:   01/16/2018 01:02 PM

Subject:        Re: [Tiff] Strategies for multi-core speedups [EXTERNAL]
Sent by:        tiff-bounces@lists.maptools.org

Also Joseph, you might find yourself limited by network bandwidth or BDP long before you are limited by application threading, good idea to think about network simulation (if relevant) before detailed design.

On Tue, Jan 16, 2018 at 2:34 PM, Paavo Helde <paavo@osa.pri.ee> wrote:

On 16.01.2018 20:36, Joe.Maniaci@ricoh-usa.com wrote:

I am a complete newbie to digital imagery and TIFFs in general, so please forgive any ignorance.

I'm an automated test tool developer and I am actually going to start
working on a viewer for the TIFFs that my company generates, to
potentially include relying on the bigtiff format and this is the thing
I've spent most of my time thinking about. I've come to the conclusion
that for the images we generate, up to 10GB uncompressed(soon to be 40GB),
that I will need to preprocess a single TIFF into multiple TIFFs
containing intelligently placed tiles just to get around the I/O issue. By
intelligently placed, I mean that if a user were to scroll their view
window to the right, and five tiles(vertically aligned) had to be loaded
into memory, I could theoretically have those files intelligently located
in different TIFF files(subTIFFs?) that would allow multiple threads to
operate on. So each thread only had to find one tile. So for my tool it
would be a staggering amount of pre-processing, but real time manipulation
is extremely fluid for the user which is basically my number one
requirement.

So for a case like mine, thread overhead is nothing to worry about. That
and the more I read about the C++11 thread library, the less I worry. I
can just leave it to the library to determine how many threads should
operate in my 4-core environments, or my 16-core environments. At least
that's the impression I've garnered.

If all you want is a viewer then you are limited by the screen sizes which
are typically not more than ca 2000x2000 pixels. And this is pretty small
image nowadays, there is no need to multithread anything, you could just
read the needed tiles from the TIFF file in real time and maybe add a
small cache of the read tiles so you don't need to read them again if the
user pans left 1 pixel.

For zooming out the TIFF file should have different frames of the same
image stored in decreasing resolutions so that a suitable resolution can
be chosen for display - these are called pyramidal tiffs. The TIFF
standard suggests that such varying resolution images should be stored as
sub-IFD-s but as many viewers do not support sub-IFD-s they are typically
stored just in standard IFD-s, and some XML piece in the first IFD
provides the details. I am sure there are multiple variants of such
pyramidal tiff formats, one from our company as well, and of course there
are multiple readers supporting such tiff files.

hth
Paavo

https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.maptools.org_mailman_listinfo_tiff&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=WLZ0J37-6mmRsbEmF36lp3XgYKTTDercqcosYw1nJns&s=wO0wn6GcM--MZsgJUVNSMq-kStUGc-sLcgZqlQ-Oz74&e=

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.remotesensing.org_libtiff_&d=DwICAg&c=5hYF0Zu0Yz-C6S-kaHDItw&r=HnVw9YJBS4ST0tOBucT3NvWUJRRGhDxXu4zHnS5AkgY&m=WLZ0J37-6mmRsbEmF36lp3XgYKTTDercqcosYw1nJns&s=qTeqEySSaE0asy3GUU2r61dgmAm_ZVfcdQMhnB4qbrQ&e=