Re: bad alloc
On Sep 2, 5:01 am, Adam Skutt <ask...@gmail.com> wrote:
On Sep 1, 7:58 pm, James Kanze <james.ka...@gmail.com> wrote:
If you can't write your output, then logging will fail. But
that's a different problem from running out of memory.
Not particularly. Our goal here is to never fail. It's by
definition impossible.
Logging is not normally part of the "essential" job of the
application.
I tried, a long time ago, to eat all my memory and then proceed to
disk I/O. This works on e.g. Unix and windows. Why wouldn't it?
Sure, if you're using read(2) and write(2) (or equivalents) and have
already allocated your buffers, then being out of memory won't require
any additional allocations on the part of your process. Of course,
performing I/O requires more effort than just the read and write
calls, and many (most?) people don't write code that uses such low-
level interfaces. Those interfaces frequently do not (e.g., C++
iostreams) make it easy or even possible to ensure that any given I/O
operation will not cause memory allocation to occur.
It's very simple to ensure that a write to an ostream doesn't do
any allocations, if you design your streambuf correctly.
Writing my own streambuf is more work than I ever want to do, and
really more work than anyone should ever have to do, in order to
handle an exception.
I've yet to see a logging system in a large system which didn't
use custom streambuf. (For that matter, at least on large scale
servers, I'd say that outputting to custom streambuf's is far
more frequent than outputting to the standard streambuf's.)
Which was my whole point initially. C++ doesn't
provide the facilities out of the box for doing anything other than
some sort of termination on an OOM condition.
I'm not sure what you mean by "the facilities out of the box".
No language that I know provides the logging facilities
necessary for a large scale server "out of the box"; C++
actually does far better than most in this regard. And no
language, to my knowledge, provides any sort of transaction
management (e.g. with rollback).
Yes, but if you're logging an out of memory condition, you don't
need any of those conversions (or you know which ones you need,
and you can use static or pre-allocated memory for them).
Yes, but that's not the only behavior that was advocated in an OOM
condition. Attempting to save program state was advocated, and that
may well require converting state.
Or may not, if it is designed correctly. (This is one of the
rare cases where just dumping the memory image of certain
struct's might be appropriate.)
In fact, in context, it started as discussion about trying to
save after std::bad_alloc was thrown, not just merely log the
OOM condition.
I've not followed the discussion completely, but most of what
I've seen seems to concern returning an error for the request
triggering the problem, and continuing to handle other requests.
Those due to handling a specific request. One obvious example
is parsing filters in LDAP; the filter can contain an
arbitrarily complex expression, which must be represented in
memory. If you run out of memory to represent it, you abort the
request (freeing the memory) with an "insufficient resources"
error. That doesn't mean that you can't handle more reasonable
requests.
It doesn't mean you can, either.
Unless the reason you've run out of memory is a memory leak,
you can.
And it hardly justifies the effort involved in handling the
OOM condition.
The effort isn't that hard, and it's a basic requirement for
some applications. If you don't handle OOM correctly, you're
application isn't correct.
If the operating system's virtual memory allows for memory allocation
by other processes to cause allocation failure in my own, then
ultimately I may be forced to crash anyway. Many operating systems
kernel panic (i.e., stop completely) if they reach their commit limit
and have no way of raising the limit (e.g., adding swap automatically
or expanding an existing file). Talking about other processes when
all mainstream systems provide robust virtual memory systems is
tomfoolery.
All mainstream systems except Linux (and I think Windows, and
some versions of AIX, and I think some versions of HP/UX as
well), you mean. The default configuration of Linux will start
killing random processes when memory gets tight (rather than
returning an error from the system request for memory).
I'm not sure how what you wrote has anything whatsoever to do with
what I said.
I'm not sure either. But it is something that must be kept in
mind when you are writing an application which has to handle
OOM.
--
James Kanze