Re: What influences C++ I/O performance?
On Feb 17, 1:04 am, "Alf P. Steinbach" <al...@start.no> wrote:
* peter koch:
On 16 Feb., 22:19, rpbg...@yahoo.com (Roland Pibinger) wrote:
On Sun, 3 Feb 2008 06:04:39 -0800 (PST), SzH wrote:
I would like to read in large integer matrices from text files, as
quickly as possible.
...
It is more likely
that there are some relevant settings for iostreams (with a
significant impact on performance) that I am not aware of.
Any suggestions for making this program perform well portably would me=
most welcome!
Avoid iostreams. They are slow 'by design'. Just try to figure out how
often reallocation and copying occur in your code.
Why do you believe that iostreams are slow by design?
I think the belief that the inefficiency may or must be
related to the design, primarily stems from the virtual
absence of efficient implementations.
Totally agreed. Dietmar Kuehl did design a very, very efficient
implementation at one point. He also made it freely available,
allowing even commercial implementations to incorporate it for
free. I know that he was very frustrated that none did.
Note that making filebuf efficient in the presence of a possibly
non-trivial codecvt is not trivial, and if I understand
correctly, Dietmar's solution involved some pretty clever
meta-programming to avoid unnecessary work whtn the codecvt was
trivial. It may be that the commercial implementers didn't
think it worth the extra complexity, and supposed that any extra
overhead would not be noticeable compared to actual I/O times.
Bjarne indicated in 2001[1] that an efficient iostream
implementation was very high on his wish-list, but that he
only expected that to be possible for a simple subset of
operations, "I believe that aggressive optimization techniques
will allow us to regain the efficiency of the original in the
many common cases where the full generality of iostreams is
not used". As far as I know, as of 2008 Bjarne has not yet
had his wish granted:
I wonder what the status of Dietmar's implementation is.
FWIW: I've used classical iostreams which were faster than the
systems printf. The classical iostreams, of course, were not
templates, didn't support wide character I/O, and didn't support
on the fly code conversions via the codecvt in filebuf. Knowing
"up front" that you're dealing with the actual "ascii"
characters (and not having to deal with things like "narrow")
can certainly result in a faster implementation.
e.g., Andrei's fabled YASTLI
implementation seems to still only have status of Good Idea.
And what would you suggest instead? The printf family
certainly is slow by design, replacing a compiled language
with an (obscure) interpreted one - namely the language
specified by the format specifier.
There is unfortunately, AFAIK, no general, good replacement.
However, where efficiency matters, one may try using FILE*,
which is generally more efficient, before going down to system
APIs.
One might, but in practice, most of us have to deal with at most
two system API's, so even going down to that level isn't always
that much of a problem. (And mmap can often beat both FILE* and
filebuf by a significant amount.)
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34