Re: Exceeding memory while using STL containers
Jeff Koftinoff wrote:
kanze wrote:
<snip>
I've never experienced this. I know that Linux used to (and
probably still does in some configurations) use lazy
allocation, and would start terminating random processes when
it ran out of memory -- AIX also had this problem in the
distant past.
I can attest to the typical linux installation's lazy
allocation effectively making bad_alloc an exception that is
almost never thrown!
For clarification for others, new and malloc will happily
return virtual memory pointers. The actual memory pages are
only allocated when you actually touch the data.
At this point, if there is a memory full condition, ie: all
ram and swap is taken, then all the O/S can do is start
killing processes via the 'OOM Killer'.
The tricky part here is that new and malloc are not the only
places where pages of virtual memory space are allocated. The
stack is as well.
And fork, and possibly some other operations as well. (Does
pthread_create use sbrk/brk to allocate stacks, or some other
mechanism?) And of course, it's quite possible to write a
version of malloc or operator new which doesn't use space in
virtual memory -- say by creating a file an mmapping it.
The really tricky part, usually, is finding out what your system
does, because this sort of thing is usually very poorly
documented.
So in effect, the STL 'compliance' of throwing bad_alloc is fully
dependent on the operating system.
Which was my point. More generally, even outside of the STL,
you generally cannot count on bad_alloc -- you have to take
draconian, system dependant (and usually undocumented) actions
to ensure getting it when you run out of memory.
On linux, one can change the kernel setting:
/proc/sys/vm/overcommit_memory
to avoid this, at the expense of much higher system memory usage.
People who are trying to write fail-safe, robust, secure
programs need to know about this behavour and test all these
corner cases. It makes worrying about bad_alloc kind of
pointless if the system almost never throws it!
Note that they know about this behavior *because* they test all
those corner cases. It's far from obvious to the typical Linux
user that overcommit is active, and it's even less obvious how
to turn it off. And in communities where robust, secure
programs are the rule, the programmers usually refuse to believe
that any system would be so dumb as to use it, until they
encounter the case in their tests.
--
James Kanze kanze.james@neuf.fr
Conseils en informatique orient?e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S?mard, 78210 St.-Cyr-l'?cole, France +33 (0)1 30 23 00 34
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]