Re: How to GET multi-word input from a *file* stream as opposed to a *console* stream?
In message <20060426190655.cd826ab3.alex.buell@munted.org.uk>, Alex
Buell <alex.buell@munted.org.uk> writes
On Wed, 26 Apr 2006 17:56:18 +0100 Richard Herring <junk@[127.0.0.1]>
waved a wand and this message magically appeared:
thanks for letting me in on the joke! :)
Look up token(), it'll help with reading words.
Where? I can't find it in ISO14882.
Ok well tokenising is a trick I use to parse sentences.
std::string input("hello world!");
std::stringstream ss(input);
std::vector<string> tokens;
std::string buffer;
while (ss >> buffer)
tokens.push_back(buffer);
Now you get a nice little vector<std::string> full of words. In this
case; tokens.at(0) has "hello", and tokens.at(1) has "world!".
Capise?
Capisco, though I see nothing called token() above ;-).
But if the input is a big file, you may get a nice _big_ vector filling
most of your memory with strings.
That's an unnecessary overhead if the intention is to scan the strings
sequentially and then discard them. This is what istream_iterators are
for.
std::string input("hello world");
std::stringstream ss(input);
for (std::istream_iterator<std::string, char> p(ss), e; p!=e; ++p)
{
// do stuff with *p
}
The above also uses a rather simplistic definition of "word" - if your
text has punctuation, you probably need something more complex. Time to
start looking at boost::tokenizer, or even the Spirit parser.
--
Richard Herring