Re: How to add thread-safety in a logging library?
ZHENG Zhong <heavyzheng@gmail.com> wrote in news:1186136408.642053.42890
@r34g2000hsd.googlegroups.com:
Hi,
I implemented a small logging library with the API like this:
[snip]
logger& log = log_manager::instance().get_logger("my_logger");
log.stream(DEBUG) << "this is a debug message" << std::endl;
log.stream(INFO) << "this is an info message" << std::endl;
[/snip]
Every logger has a unique name, and manages an output stream
(std::ostream). The 'stream(log_level_t)' member function returns an
ostream_proxy object, which is implemented as the following:
[snip]
class ostream_proxy {
public:
// other member functions...
template<typename T>
ostream_proxy& operator<<(T const& t) {
(*os_) << t;
return *this;
}
private:
std::ostream* os_; // ostream_proxy objects from the same logger
object share
// the same output stream.
};
[/snip]
Thus it is possible that two threads retrieve the same logger object
and write messages to the logging stream at the same time.
Now i want to add thread-safety to my logging library, and i realize
that in ostream_proxy, the statement "(*os_) << t;" is not thread-safe
(std::ostream is not thread-safe, right?). So i need to re-implement
the operator << in ostream_proxy like this:
[snip]
class ostream_proxy {
public:
// other member functions...
template<typename T>
ostream_proxy& operator<<(T const& t) {
{
boost::mutex::scoped_lock lock(*os_mutex_);
(*os_) << t;
}
return *this;
}
private:
std::ostream* os_; // ostream_proxy objects from the same
logger object
boost::mutex* os_mutex_; // share the same output stream and the
same mutex.
};
[/snip]
In this way, i can guarantee that at any moment, there is at most one
thread calling "(*os_) << t;".
As another posted, you'll only ensure the individual function calls on
the ostream are atomic. However east line of logging contains
multiple function calls, so you'll get logs interspersed. There's
another problem with this approach. You're blocking threads on
logging. This will hinder performance and the ability to diagnose any
timing issues of your application that logging might help identify
(although logging affects timing).
Consider instead a separate log object per thread. You might use the
boost::thread::tss class. Morever, consider separating the interface
(e.g., ostream) from the buffering (e.g., streambuf), and the end
destination from those two. The Standard C++ IOStreams does not
achieve this separation because of how the streambufs are intimately
connected with the target of read/write calls.
Depending on your operating system's threading model and IO system
call interface, you may be able to achieve atomicity at the system
call level, without affecting performance. For example, if your write
call is atomic, then that problem is solved. Otherwise, you might
want to consider a lock-free algorithm for queuing buffer objects to
be output. From reading discussions of fwrite(), that might not be
the solution. Emprically, it seems to have worked OK on Linux, but
that's not the best way to judge.
But since logging may be an action that is frequently performed, the
code above may be too expensive to bear... Surely i can use a policy
to allow user to choose if s/he want thread-safety or not. But in a
multi-threaded application, user still has to pay for logging...
There are different ways to tackle this issue. As another poster
mentioned, you can use macros. It gets a little ugly and doesn't
allow you fine-grained filtering, by logging level. You have to
decide at compile time if "INFO" or "ERROR" level logs are warranted.
However macros ensure code is removed.
Another approach is to evaluate in the logging operation whether to
copy the value. Suppose you have a log subsystem that associates levels
through a configuration setting. Consider something like this:
.....
log << level(LOG_LEVEL_INFO) << "value=" << value << send;
The various implementation of operatior <<() would decide whether to
bother copying each value into the buffer. However, it gets to be
very difficult to implement this with the IOStream interface and the
manipulators make it even harder. If the function calls are inlined,
you only take a hit with something like:
Log & operator <<(Log & log, T value) // for some T
{
if (log.meetsThreshold())
{
copyTo(log.getBuffer(), value); // where copyTo is overloaded,
etc.
}
return T;
}
bool Log::meetsThreshold() const
{
return (this->currThreshold & this->necThreshold) != 0;
}
So i would like to know if such implementation is proper, or if there
is a way to make that better.
Having implemented a logging subsystem used by a team of six
developers, I think it's good to allow class users different levels of
filtering at run-time and some ability to remove logs at compile-time.
In the performance critical system I worked on, all debug statements
were removed in release builds (NDEBUG == 1). Anything that was in a
tight processing area would only log things that indicated errors.
Moreover, the constructor of the Log class was very cheap. Buffers
were held on a per-thread stack that didn't destory buffers when they
were no longer used.
Filtering should be delegated to another lightweight object that can
use bit operations as much as possible. Delegation also facilitates
changing the filtering interface later, allowing richer capabilities
as the need arises.
I didn't use IOStreams, but implemented a bunch of overloaded
operators. It was more coding, but it had less of the baggage I
didn't need (e.g., locales). The targets were designated on a
per-thread basis and could be chosen at run-time.
Some other thoughts: I would really take some time to design your
classes before implementing them. If you can use UML, you'll probably
see many more abstractions. On the other hand, you'll definitely want
to prototype the interfaces. See what the state of the art is.
--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]