AWARE SYSTEMS
TIFF and LibTiff Mail List Archive

Thread

2002.08.20 07:31 "memory allocation", by Peter Majer
2002.08.20 16:13 "RE: memory allocation", by Roger Bedell
2002.08.20 20:48 "OT: large memory allocation in Windows", by Dante Allegria
2002.08.21 08:34 "Re: OT: large memory allocation in Windows", by Rob van den Tillaart
2002.08.21 11:30 "Re: OT: large memory allocation in Windows", by Peter Majer
2002.08.21 13:47 "Re: OT: large memory allocation in Windows", by Rob van den Tillaart
2002.08.21 16:05 "Re: OT: large memory allocation in Windows", by Bob Friesenhahn
2002.08.21 20:47 "Re: OT: large memory allocation in Windows", by Peter Nielsen
2002.08.22 08:33 "Re: OT: large memory allocation in Windows", by Andreas R. Kleinert
2002.08.22 15:30 "RE: OT: large memory allocation in Windows", by Roger Bedell
2002.08.22 16:29 "Re: OT: large memory allocation in Windows", by Chris 'Xenon' Hanson
2002.08.22 17:20 "RE: OT: large memory allocation in Windows", by Roger Bedell
2002.08.22 17:46 "RE: OT: large memory allocation in Windows", by Dante Allegria
2002.08.22 17:33 "Re: OT: large memory allocation in Windows", by Peter Montgomery
2002.08.22 18:45 "Re: OT: large memory allocation in Windows", by Chris 'Xenon' Hanson
2002.08.22 21:19 "Re: OT: large memory allocation in Windows", by Peter Montgomery
2002.09.01 17:14 "Re: OT: large memory allocation in Windows", by Peter Montgomery
2002.09.05 17:00 "Re: OT: large memory allocation in Windows", by Andreas R. Kleinert
2002.09.05 17:03 "Re: OT: large memory allocation in Windows", by Peter Montgomery
2002.08.22 15:59 "RE: OT: large memory allocation in Windows", by Ed Grissom
2002.08.22 22:09 "RE: OT: large memory allocation in Windows", by Andreas R. Kleinert
2002.08.21 11:48 "Re: OT: large memory allocation in Windows", by Peter Nielsen
2002.08.22 18:00 "Re: OT: large memory allocation in Windows", by Bob Friesenhahn
2002.08.22 18:18 "Re: OT: large memory allocation in Windows", by Peter Montgomery

2002.08.22 22:09 "RE: OT: large memory allocation in Windows", by Andreas R. Kleinert

A 1 GB file occupies a 1 GB address range (within the 2 GB virtual user address space area).

If you map the whole file at once, sure.

A more reasonable approach is to map the current region of the file you are interested in. We typically map a few hundred K of the file at a time. I probably ought to revisit this and map a few meg at a time.

Well, of course. But this again means a tiled/segmented memory architecture - with a slide of the address window and/or its content, which needs to be covered by the algorithm that processes the data.

I actually hide this from the rest of my library in a file i/o layer that I wrote that mimics fopen/fseeek/fread with mmap'ed calls using MapViewOfFile.

I like the idea, but you (functionally) also could achieve this with normal fopen/fseek/fread on a fixed malloc'ed buffer. No fancy 4K mmu-page wise "write back" support but anyway...

If I want linear addressing of a file's content, I need to fully map it to memory. If not, all sorts of elegant/appropriate algorithms certainly come in handy ;-)

And a "linear pointer space" was mentioned at the start. Here's the quote again:

>>>How about a memory mapped file? Could that possibly give you a
>>>linear "pointer space" of 2GB or maybe even more? Of course the file
>>>would not be completely mapped at all times, but maybe it would do the
>>>trick... (Just a crazy idea. I don't know if it would work at all...).

Andreas_Kleinert@t-online.de  | http://www.ar-kleinert.de             |
Freelance Consultant & Writer | Software Engineering                  |
 *** PerSuaSiVe SoftWorX ***  | x86 Win/Linux, 68k/PPC Amiga and more |