Re: Memory allocation program

Javeed wrote:


CreateDIBSection has an hSection argument, which is a handle to a file
mapping object (aka memory mapped file). You can create such an object
using CreateFileMapping, where the fProtect argument can contain the
SEC_RESERVE flag. That should reserve virtual memory pages without
actually physically allocating it. Reserved pages can later be committed
using MapViewOfFile, followed by VirtualAlloc, with flAllocationType =
MEM_COMMIT -- this is when the physical memory gets claimed.

The big problem is that you can't decommit a file mapping:
http://support.microsoft.com/kb/108231

<quote>
The SEC_RESERVE flag is intended for file mappings that are backed by
the paging file, and therefore use SEC_RESERVE only when hFile is -1.
The pages are reserved just as they are when the MEM_RESERVE flag is
used in VirtualAlloc(). The pages can be committed by subsequently using
the VirtualAlloc() application programming interface (API), specifying
MEM_COMMIT. *Once committed, these pages cannot be decommitted*.
</quote>

I actually tried it and that seems to be the situation:

Test1:

    const size_t mem_size = 100000000;
    void* reserved_ptr = VirtualAlloc(NULL, mem_size, MEM_RESERVE,
       PAGE_READWRITE);
    if(reserved_ptr)
    {
       void* ptr = VirtualAlloc(reserved_ptr, mem_size, MEM_COMMIT,
          PAGE_READWRITE);
       if(ptr)
       {
          MessageBox(0, "committed", "", MB_OK);
          //
          VirtualFree(ptr, mem_size, MEM_DECOMMIT);
       }
       MessageBox(0, "decommitted", "", MB_OK);
       VirtualFree(reserved_ptr, mem_size, MEM_RELEASE);
    }

This works as expected. When the "committed" message box appears, my
memory usage goes up, and when I click it, it goes back by 100MB, as the
physical memory gets decommitted.

You can reserve a file mapping object and commit it later, but it can
never be decommitted without releasing the virtual address space with it.

Test2:

    const size_t mem_size = 100000000;
    const HANDLE map_handle = CreateFileMapping(INVALID_HANDLE_VALUE,
       NULL, PAGE_READWRITE | SEC_RESERVE, 0, mem_size, 0);
    if(map_handle)
    {
       void* map_ptr = MapViewOfFile(map_handle,
          FILE_MAP_WRITE | FILE_MAP_READ, 0, 0, mem_size);
       if(map_ptr)
       {
          void* ptr = VirtualAlloc(map_ptr, mem_size, MEM_COMMIT,
             PAGE_READWRITE);
          if(ptr)
          {
             MessageBox(0, "committed", "", MB_OK);
             //
             VirtualFree(ptr, mem_size, MEM_DECOMMIT);
          }
          UnmapViewOfFile(map_ptr);
          MessageBox(0, "decommitted", "", MB_OK);
       }
       CloseHandle(map_handle);
    }

When the "committed" message box appears, my memory usage goes up, and
when I click it, it stays there, until the file mapping's handle gets
closed.

CreateDIBSection can only be used in two modes:

1. Either it manages the bitmap's memory on its own, without the
possibility of commit/decommit,

2. Or you can use a file mapping object to store the bitmap's pixels,
which you can reserve at program startup, and commit it later, but you
will never be able to decommit it without destroying the entire file
mapping.

It looks like there is no documented mechanism that allows you to
decommit virtual memory belonging to a DIB section, with keeping a
large, contiguous virtual memory block reserved at all times.

If you explain why you need to stick with CreateDIBSection, we could
probably offer an alternative method.

My take on this is that I never use a DIB section larger than the
screen. If I have to work with really large images, I handle it on my
own. You can't use Windows functions to display your images anyway,
because they don't have reasonably good interpolating routines. I just
take my image, in whatever memory format I want (which can be committed
and decommitted if needed), and render it on a DIB section on the fly
just for display, or just use the DrawDibDraw API to display.
DrawDibDraw is faster than CreateDIBSection + BitBlt.

If you want to render stuff of an off-screen bitmap, once again, a DIB
section is not needed if you get a library like D-Type
(http://d-type.com), which draws vector and text objects in far superior
quality than the Windows GDI. That doesn't require a DIB section either.

Once again, my recommended solution would be what Tom Widmer recommended
(outlined in my Test1 sample code) -- you reserve a contiguous virtual
memory block, but don't allocate actual physical memory. Then commit and
decommit the memory as needed, but never completely free the virtual
address space until your application exits. This way you avoid
fragmentation, without having to hold on to the large physical memory at
all times.

Tom