2018.01.16 01:09 "Re: [Tiff] Strategies for multi-core speedups", by Craig Bruce

On 01/15/2018 08:15 PM, Bob Friesenhahn wrote:

On Mon, 15 Jan 2018, Mike Stanton wrote:

> To begin with I am an Operating Systems guy. When you are looking > at going to multi-threading, what have you determined as your

> setup/teardown cost for the individual threads? How large a chunk > will each thread be processing? Will the individual Scan-lines

> and/or Tiles be sufficiently large to absorb the cost of that setup > and teardown by the inherent performance you are projecting?

> >
> >

It is normal to use a pool of threads where threads are created in advance (or added to the pool on demand) and obtained from, and returned to, the pool. This significantly reduces overheads caused by starting/stopping threads.

Creating and destroying threads ought to take on the scale of a few microseconds (on Linux anyway). Larry Gritz was saying it takes 16.2 seconds to read what could be 1024 tiles of size 256×256, which is 15.8 ms per tile, which is nearly four orders of magnitude longer than thread creation for reading. Writing with Deflate compression is about an order of magnitude slower than reading.

However, you don't need to launch a new thread for every tile if you just break up the job into N=number_of_cpus pieces and launch N threads where each is pre-assigned to read or write a particular group of tiles of the overall image. The thread-creation overhead would be around six or seven orders of magnitude smaller than the reading or writing.

--
Dr. Craig S. Bruce
Senior Software Developer
CubeWerx Inc.
http://www.cubewerx.com