Re: MappedByteBuffer
On 23/07/10 14:27, Alan Gutierrez wrote:
Kevin McMurtrie wrote:
In article <i2av27$9h7$1@news.eternal-september.org>,
Alan Gutierrez <alan@blogometer.com> wrote:
I'm implementing file storage for a B+Tree for deployment on Linux,
development on OS X, and wanted to use MappedByteBuffer. This seems
like an easy way to address large pages of data and not have
implement buffering myself.
I've done a little testing...
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
...
Or am I misunderstanding the usage of MappedByteBuffer?
Java 1.5 is more commonly a 32 bit process while Java 1.6 is more
commonly a 64 bit process. You're running out of memory addresses in
32 bit mode.
Adding '-d64' as the first JVM option will force the 64 bit version or
fail with a message that it's not installed.
Adding -d64 to JDK 1.5 does allow it to address more than 6 pages. Thank
you.
Can you help me understand what is going on here? Is it that, Even
through `MappedByteBuffer` will page in and out of memory, you still
need a 64 bits to address entire region, and the addresses cannot overlap?
In the dim and distant past I did some work using memory mapped files in
C, and I presume that the mechanism behind Java memory mapped files is
similar. A memory mapped file is mapped onto a specific address in the
virtual memory address space of the application. Although they are not
actually occupying the memory there they still require a unique, and
contiguous, address space equal to the size of the memory mapped region.
So if you want to map a region of 256MB you need 256MB of contiguous
address space.
A value is read from a memory mapped region by accessing a particular
address in the virtual memory of the application e.g. if the file is
mapped at address 0x20000, and you want to read a value at offset 0x500
from that start of the file then the data at 0x20500 is read. This
address must be unique, so the mapped regions cannot overlap, they must
be contiguous, and all mapped regions must lie within the addressable
space of the application.
In a 32bit application your maximum address space is 4GB, and could be
a good deal less than that due to system dependent memory limitations.
Thus, your attempt to allocate multiple regions of that address space in
blocks of 256MB will have a limited amount of success, depending on what
contiguous regions of the address space are currently available.
When you run the 64bit version of the JVM the available address space
has increased by a factor of 1 billion. Therefore you are able to
allocate those regions of contiguous address space for your application,
even though there is still the same amount of physical RAM and system VM.
--
Nigel Wade