Re: Memory Leak/Profiling
On May 19, 1:06 pm, Branimir Maksimovic <b...@hotmail.com> wrote:
On May 19, 10:35 am, James Kanze <james.ka...@gmail.com> wrote:
On May 19, 3:01 am, Branimir Maksimovic <b...@hotmail.com> wrote:
On May 19, 2:03 am, jeungs...@gmail.com wrote:
I'm trying to track down a memory issue with a C++ application that
I'm working on:
In a nutshell, the resident memory usage of my program continues to
grow as the program runs. It starts off at a nice 4% of memory, th=
en
slowly grows up to 50% and beyond. This translates to around 2 gigs
of physical memory, and that's really way more memory than this
program should be taking up.
I'm looking for a tool that can tell me where this memory is being
allocated to.
I've fiddled around with both purify and ccmalloc without much
success. I suspect that what I'm seeing here is not a memory leak =
in
the traditional sense -- by which I mean, memory that has been
allocated and no longer has any pointers pointing to it.
I have a suspicion that somewhere in my program, a vector is growing
out of control.
What you are describing looks like memory leak caused by
fragmentation of memory. This type of leak is probably not
caused by vector, but buy large number of alloc's/free's
of varying sizes.
Not necessarily. Modern collectors are pretty good at reducing
fragmentation.
Yes, those are moving collectors that can't be implemented for C++
because of raw memory pointers and pointer arithmetic.
Altough one can imeplement such easilly with disciplined programing
and usage of objects as pointers instead of raw ones.
But, non moving collectors have same problem as manual
memory management.
Not necessarily.
To solve this problem you have to actually
do same thing as vector. To preallocate pools for each alloc
size and in that way keep fragmentation at bay.
That's a strategy used by some mallocs. If he's having such a
problem, switching to a different malloc/free might help. (For
the most part, the system operator new will just call malloc.)
I have tried to replace malloc in different ways
with my implementation on linux (because of same problem), but
unsucessfully.
I've never had any problem on Solaris. If the default malloc
has caused problems, switching to one of the others delivered
with the system generally solved them.
It's important to realize, too, that while fragmentation will
generally increase the memory footprint of the program, it
almost never causes real leaking; the footprint may be larger,
but it is stable.
LD_PRELOAD won;t work because libc functions
call malloc during init time (so nobody knows which private function
of malloc interface will be called), and also malloc hooks can;t be
used because malloc uses those for it's internal purposes,
even that linux docs say opposite.
Problem is that when programers play smart and don;t use
global operator new or some function that wraps calls to malloc, then
since C don;t have interface for replacing that function
this is sometimes not an option.
Which malloc/free you use is decided at link time. So there's
no problem with static initializers (unless the malloc/free
requires some special initialization).
In order to solve this successfully, you have to find out
what sizes are allocated/freed most.
If you can't determine that, even pool allocator won't help
you, because it will just preallocate too much blocks
for different sizes.
A good implementation of malloc will detect this automatically,
and adjust its strategy accordingly.
I never saw such malloc. I can imagine that allocs would be
very costly with such implementation.
Not really. All it entails is maintaining a bit of statistical
data. But it's true that it is generally easier for the vendor
to provide several different mallocs, and let the user
explicitly choose at link time.
But my colleague and I have solved problem by aligning
allocation for different sizes into predefined sizes.
This uses more memory but does not have fragmentation that
much. This wasn't satisfactory solution for applications
that hold lot of memory since memory usage was a bit high.
So I really can't imagine general purpose allocator that
can solve all problems. In practice application programmer
always have to think about allocation/deallocation strategy,
if writing apps that will allocate/free
non-stop and run for a long time.
That hasn't been my experience. Of course, very few of my
applications run for more than about ten years; maybe that's not
long enough.
--
James Kanze (Gabi Software) email: james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34