2016.01.25 18:25 "[Tiff] OpenMP enabled libtiff", by Aaron Boxer

2016.01.29 18:12 "Re: [Tiff] OpenMP enabled libtiff", by Bob Friesenhahn

On Fri, 29 Jan 2016, Aaron Boxer wrote:

> On Unix type systems, the madvise() function can be used to help control behavior of memory-mapped data
> (e.g. MADV_DONTNEED), but again behavior is highly OS implementation dependent and the only way to know
> for sure is to test.

Thanks, Bob. windows, since there is no madvise function,   On wouldn't reading a file and memory mapping the file both give the same result: file gets cached in the OS file cache, causing   the memory pressure when the cache gets filled up?

I think that Windows also has memory advisory functions.

Memory mapping is not necessarily based on the OS file cache. The file cache may be owned by the filesystem implementation rather than the VM system.

When reading a memory mapped file the first time, major page faults are caused when encountering a page of data the first time. When immediately reading the file the second time, only minor page faults are encountered for the same page of data. The difference is that major page faults trap to the kernel, and the kernel goes to disk for the data (long wait), and the minor page fault traps to the kernel, but the kernel only maps the existing page into the process (tiny wait).

Even after the program quits, the memory pages associated with the file are in memory. When reading a second file, the memory pages from the first file are still in memory. After reading enough files, the kernel needs to purge old pages and this takes time. During the time old pages are being purged (typically by a kernel thread), new programs have difficulty running, and major faults on freshly mapped files take much longer. The whole computer seems slower.

Does this make sense?

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/