Re: Exception Misconceptions
In article <4b2f4d16$0$1109$4fafbaef@reader4.news.tin.it>, "io_x" <a@b.c.invalid> wrote:
"tanix" <tanix@mongo.net> ha scritto nel messaggio
news:hgnee8$dag$1@news.eternal-september.org...
In article <4b2f3095$0$1110$4fafbaef@reader3.news.tin.it>, "io_x"
<a@b.c.invalid> wrote:
"James Kanze" <james.kanze@gmail.com> ha scritto nel messaggio
news:4f3858fa-d8e2-4c92-970f-ae4d6d9913cc@s31g2000yqs.googlegroups.com...
On Dec 19, 4:51 pm, "io_x" <a...@b.c.invalid> wrote:
"tanix" <ta...@mongo.net> ha scritto nel
messaggionews:hgg839$ot7$1@news.eternal-september.org...
In article
<3eacbb7a-4318-4fed-b71c-f5da24cfa...@s20g2000yqd.googlegroups.com>,
James
Kanze <james.ka...@gmail.com> wrote:
Wasting weeks on cleaning up the memory leaks?
I have wasted MONTHS on trying to catch all the subtle
memory leaks in a sophisticated async based program because
of some network availability issues make you maintain all
sorts of queues, depending on user interaction, causing such
headaches, that you can not even begin to imagine from the
standpoint of memory leaks.
do you know, exist wrapper for malloc, that at end of the
program check if there are "memory leak" and report the result
to the screen
If the program ends, it doesn't leak memory, and practically,
there's no way any tool can tell whether there is a leak or not.
Well, flip some bits in Visual Studio IDE and you'll get
a dump of ALL your memory leaks when the program terminates.
Not exactly a nuclear science. But some things are not as simple
as it looks. Sure, if you are willing to waste days on it,
assuming those leaks are bad enough to make you worrry about it,
you can design some things to exactly identify the situation.
But there are times when you pass the same buffer around
to several methods
and hold a reference to in on various queues
for efficiency reasons for example.
in the case i think, there is a place where the buffer born, and
a place where the buffer die
Yep, at least in theory AND in SYNCHRONOUS environment.
malloc/realloc has to do all the "efficiency work" (if possible)
Nope. Does not work. The buffer size varies with the packet size.
The buffer generation is pertty much a random event.
One buffer has no relations to any other buffer for related
classes. They all exist on their own and there is nothing common
between them except of the fact that they are of the same
class. But WHO will eventually be a client or "consumer"
depends on ALL sorts of things. Those buffers may become
a part of one class or another. There may be several different
complex objects that would hold those buffers for various reasons.
The only time the buffers are released is either when you
run over the limits to a number of monitor entries, or you
exit the program.
At all other times, these buffers may be temporarily held
until the user decides what he wants to do, in which cause
the references from some queues are released, but even there,
there may exist a copy of that buffer on a monitor view
list box.
The solution, finally adopted was to specifically make
a copy of the buffer so that there is no "competition"
for the same buffer.
The buffer deallocation in this situation is one of the
nastiest things you can imagine.
As far as logical design of the system, to this day,
I am not aware of a better way to design it as it is
probably the most efficient way to accomodate for ALL
sorts of different situations.
Unless this magic "smart pointer" works and is not a royal
pain on the neck to even bother about and it really does
its "automatic" whatever to make sure there is not
leaks possible, which I doubt, you have about the toughest
memory management situation you can imagine.
In those cases, your buffer
may be releasable by several different routines
this can not exist, for what i write up, and double allocation
of the same pointer without free is not allow
It is not a DOUBLE allocation.
The allocation is single.
But the buffer may be temporarily owned by a different
subsystem. Those subsystems are independent and are not even
aware of each other.
Depending on logical operation, which ultimately is determined
by the user's choice, the buffer may be passed to a different
subsystem and, in some cases, may be released instantly,
in some cases it may be released at a predictable point,
and in other cases it may be held for "indefinetely" long
period of time within the scope of realistic events.
and in the async
environment, you don't have a luxury of seeing a sequential
allocation and deallocation within more or less same scope of
events.
You may allocate it in one place. Then, depending on what kinds
of things happen, deallocate it DAYS later for that matter,
and from SEVERAL different places. Not just one, and I could
care less what anybody has to say on "good design" issue.
So, considering the fact there are several clients, which one has
a bug?
Yes, memory deallocations are non issues in trivial applications.
But if you do not have a gc equvalent mechanism, no matter how
you cut it, it is going to be a headache one way or another.
it is not one headache if all the functions to debug
has one place (at start) where all memory is created
and what place (at end) where all memory is free
Except when you have thousands if not millions of allocations
and deallocation does not happen in ALL cases more or less
immediately. Try to debug that one. Or even log it.
Would you like to look at 10 mile long logs?
then there would be something, that when the function end
and there is some leak for pointer allocated in that function,
for debug reason,
program stop and
print the line of the function and the pointer that is never free
That would be nice if it was THAT simple.
:--}
But you are talking about SYNCHRONOUS environment.
A different animal altogether.
You can not simply single step this issue in async environment.
Yes, you CAN log things into some log file.
But when you start analyzing it, you'll get a 3 mile long log.
How do you even LOOK for something in that log?
Well, DAYS of work.
in not a multithread environment, i could ask to some malloc function
the size of all the memory
allocated for the program in the start of the function
and check that in the end of that function the memory is the same
something like
Not a good idea. To do your own memory management is WAY
too ineficient and simply does not make sense.
The end result though looks nice.
Because you know you only have one buffer, which you can
easily deallocate at the end.
But then you have to worry about ALL sorts of side effects.
You see, there IS already a system for memory management
for you on the O/S level. Do you think you can manage memory
MORE efficiently than the O/S does?
Well, people spent YEARS on designing those mechanisms.
It may take you YEARS to make it finally work,
after you see those nasty special cases, hitting your like
a brick.
The problem could be solved by buffer duplication.
That is. You could make a copy of the buffer for a different
subsystem. So, each subsystem knows it is an owner of the
buffer and no matter what happens with other subsystem,
it is guaranteed to be able to deallocate the buffer
whenever it decides to do so.
But...
If the number of those allocations is huge,
you will be consuming at least double the amount of memory
by your program and will overload the O/S unnecessarily
with essentially idle things that do not have to exist
in vast majority of cases.
That is why this program is probably one of the leanest
programs you can see in your windows task manager.
It consumes about 10-20 megs. in toto, no matter what
happens. Compare it to hundreds of megs taken by just
about any modern program.
Sure, I COULD swallow a couple of hundreds megs just like
anybody else does and I COULD solve this problem in a
matter of hours if I decided to be sloppy, just like
everybody else. But hey, it is no fun that way.
I can pretty much guarantee you:
You won't find a more efficient program, more leaner
program and more robust program for ANY money, more or
less, within the reasonable limitations.
Just take ANY firewall out there, and most of them
simply create an ILLUSION that you are protected
while silently passing some of the nastiest and most
dangerous traffic without even knowing you do.
Because some of that traffic happens to be on remote
port 80, whichs your web browser. One of the most lethal
and potent rootkit versions operates a global network.
If your box is infected, it will send out a packet
on remote port 80 to their root host. No firewall
program will even notice it.
Now, once that packet is sent out, your box will be
essentially attacked by a number of hosts on their
network, and some of those boxes belong to people,
who do not even know their boxes are used in a global
most lethal network known to mankind.
So, some of those hosts may send you the ICMP type 4
packets to probe if your box is responsive.
Some other hosts may send you viruses and trojans
of the latest version.
All these things are done either in temp port range
that no firewall will even attempt to block, or in
high port range.
The bottom line is that when you use most of the firewalls
out there, you are like blindfolded.
One of the things I have in my firewall is the traffic
indication lights. As soon as I suspect something, I just
click on icon and simply look at the traffic light.
If I am not doing anything and see those lights flashing,
then something funky may be going on. At THAT point, I
may just push a single checkbox and it will display me
every single packet my box is dealing with. If I right
click on any of those packets and push a single button,
my firewall will perform a whois request from all the
major world's servers and will automatically filter
the server results and display it in the window.
That is another set of buffers. Also fully async.
There are also the DNS buffers. The forward/backward
DNS requests are performed automatically for ANY
funky pakcet, so when I look at the monitor listbox,
it shows me not only the remote IP, but remote host name.
The DNS requests are fully async and will automatically
update the monitor entries as soon as DNS server
sends me the data.
There are trace buffer, raw socket buffers, and you
name it.
When some jack says he has a magic solution for me,
I just say: go to hell.
Ok, Cya.
unsigned z;
z=All_malloc_memory();
f(a,b,c,c,s);
x=All_malloc_memory();
if(x!=z) {printf("Memory problem for f line=%u \n", 1111); exit(0); }
Cool.
:--}
I wish it would be THAT simple!
Well, you can have an alloc wrapper.
What I did in my last version for one of the most critical
and massive allocation related methods is to ID stamp the
allocations. The buffers are allocated at a single place,
a driver interface. Once they are allocated, they are stamped
with ID of 0.
Once you start processing and pass those buffers around
and incorporate them into different objects, you increase the
processing ID.
Once you terminate the program and get your memory leak dump
from VC, the 1st byte of buffer is that ID. So you can see
exactly what routines had that buffer and could or should have
released it. That took care of some of the nastiest deallocation
issues I had that could potentially lead to MASSIVE leaks
under certain conditions.
Some of
them, you may have used, without knowing it, since many of the
programs I've worked on aren't visible to the user---they do
things like routing your telephone call to the correct
destination. (And the proof that there isn't a leak: the
program has run over five years without running out of memory.)
possibily the leak is small and/or in some corner case
Even considering 5 YEARS?
:--}
so memory leak can not be one problem if one use these special
"malloc" functions (like i use always with all bell that
sound)
I'll say it again: there's no silver bullet.
the silver bullet exist in all
In the end, good
software engineering is the only solution. Thus, for example, I
prefer using garbage collection
when I can, but it's a tool
which reduces my workload, not something which miraculously
elimninates all memory leaks (and some of the early Java
applications were noted for leaking, fast and furious).
are you sure is it good for think less?
are you sure is it good for not doing formal
correct memory allocation-deallocations?
--
James Kanze
--
Programmer's Goldmine collections:
http://preciseinfo.org
Tens of thousands of code examples and expert discussions on
C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript,
organized by major topics of language, tools, methods, techniques.
--
Programmer's Goldmine collections:
http://preciseinfo.org
Tens of thousands of code examples and expert discussions on
C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript,
organized by major topics of language, tools, methods, techniques.