Re: is std::ifstream buffered or not?
Karl wrote:
I've got a quick question on whether std::ifstream is buffered
or not.
The answer is yes. The size of the buffer depends on the
implementation, however.
The reason is that I have a homework assignment that
requires me to benchmark copying files using different
buffer sizes. I ended up doing this using
std::istream::readsome() and std::ostream::write():
I'm not sure what you were expecting from readsome(), but I'm
pretty sure that it isn't appropriate for what you are trying to
do.
// now try writing to the destination file
std::ifstream sourceStream(sourceFilename.c_str(),
std::ios::binary);
std::ofstream destinationStream(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);
// determine the size of the file
sourceStream.seekg(0, std::ios::end);
const std::streamsize totalLength = sourceStream.tellg();
I hope you realize that this is not guaranteed to give you
anything significant. I don't think it is even guaranteed to
compile. The only portable way to find the length of a file is
to read it, byte by byte, and count the number of bytes you
read.
sourceStream.seekg(0, std::ios::beg);
// now writing the actual data
char buffer[numberOfBytes];
std::streamsize length = 0;
while(length != totalLength)
{
int l = sourceStream.readsome(buffer, numberOfBytes);
And this is very implementation dependant. All that readsome
guarantees is 1) if there are characters present in the buffer,
they, and only they, will be read, and 2) it will never block.
(Even 2 is a bit tricky, since it really isn't defined what is
meant be block. In practice, I think you can count on it not
waiting indefinitly for keyboard or pipe input, but that's about
it. An implementation which can determine file size may do so,
and read all of the bytes in the file, or it may start reading 0
bytes as soon as the internal buffer is exhausted, just like it
would for keyboard input.)
destinationStream.write(buffer, l);
length += l;
}
// now close the stream
sourceStream.close();
destinationStream.close();
Now, the validity of benchmarking this could possibly be in
question because std::istream::readsome() could be doing some
buffering (readahead?) in the back, so using different buffer
sizes would possibly be meaningless.
It's also very, very possible that the code loops forever,
regardless of the size of your buffer.
I understand std::cout and std::cin are buffered, and it makes
sense that they are. However, I do not see why std::ifstream
and std::ofstream would be buffered because the filesystem and
even the harddrive does some buffering. Wouldn't that be
meaningless?
No. System requests are (or were) expensive. In fact, I would
generally say that the reverse is true: cout and cin are often
connected to interactive devices, where you don't want
buffering; ifstream and ofstream rarely are.
Then again, this could depend on the implementation. I'm not
familiar enough with the C++ STL specification.
There is a function in streambuf: setbuf, which can be used to
set a user defined buffer. But an implementation is only
required to respect it if it is used to request unbuffered IO
(which means in practice a buffer size of 1).
Can anyone help me out? Any help would be greatly
appreciated. :)
The real question is what is the goal of the homework
assignment. There are so many indirections involved with
istream and ostream that it is almost impossible to manage
buffering in a portable manner. (Remember, for example, that
filebuf code translates according to the locale.) If the goal
is really to learn about optimizing buffering, then you should
be going to the system level functions (open, read, write and
close under Posix). If, on the other hand, the goal is for you
to discover on your own the futility of trying to buffer stream
I/O externally, then the intent is probably that you use
istream::read.
In either case, under Unix, it would also be interesting to
compare the results when reading /dev/null, a file filled with
random data, and a file created by a seek to some very distant
place and the write of a single byte. It also might be
interesting to see what happens when you use some "odd" buffer
sizes, rather than a power of two.
More relevant to the real world, of course, would be to compare
read times on a file on local disk with those for a remote
mounted file. At different times of the day, when the network
load varies. Or to compare synchronous writes, with
asynchronous. (My experience is that FILE* and std::filebuf
generally use the best possible buffer size by default. But
note that the optimal buffer size for a file may depend on the
system which hosts it, rather than the system you are running
on, and so be different for different files.)
--
James Kanze GABI Software
Conseils en informatique orient?e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S?mard, 78210 St.-Cyr-l'?cole, France, +33 (0)1 30 23 00 34
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]