Re: CStdioFile... with a twist (please)
Thank you very very much, Joe. I now have all I need and then some.
On Aug 5, 12:37 am, Joseph M. Newcomer <newco...@flounder.com> wrote:
See below...
On Mon, 4 Aug 2008 07:12:40 -0700 (PDT), Alexander <the44s...@yahoo.com> =
wrote:
Thank you for taking the time to post an algorithm, Joe.
Your observations have, of course, hit on the issues that I've come
across. CString operations are the biggest time suckers for sure (the
file is ASCII but the code must be UNICODE).
****
Note I used a CStringA. If, at some point, you need to convert, you ca=
n do
CStringA t;
CString s(t);
and s will be the Unicode version of the ANSI characters in t. I use this=
trick fairly
often when I have to deal with 8-bit data streams in Unicode apps. But=
if you can
postpone the conversion so the conversion is folded in with the creation,=
it will work
better; for example
CString s(p, n);
where p is an LPSTR/LPCSTR (not LPTSTR/LPCTSTR) and n is the number of ch=
aracters to
convert will give you a Unicode string with just one copy operation (and =
a
MultiByteToWideChar conversion)
****>I wonder, though, about
loading the entire file at once... somehow I had convinced myself that
it would slow things down.
****
Why? You have to read all the file eventually...
****
The test file is small, approx. 4 MB (the "real" one 60~90 MB). I'll
add this code to the benchmark and see how it fares.
****
Memory-mapped files may also give better performance than a large ReadFil=
e; they're harder
to use but you don't actually bring the data in until it is touched. I=
f you window data,
it is a bit tricky because the windows have to fall on 64K boundaries, so=
you have to
"back up" to the previous 64K boundary if you hit an edge, e.g., if each =
of the letters
below is a 64K block
ABCDEFG
and you map AB into memory, when you get to the end of B you have only a =
partial line, so
you would then have to map
BC
and then
CD
and so on so you could see the entire string (assuming it is <64K; otherw=
ise you would map
more pages to cover the maximum string length). Note that you don't ha=
ve to map the
minimum number of pages; you could map 100 pages or 300 pages or whatever=
you want, but
when you hit a boundary, you have to include the 64K block in which the r=
ecord started in
the next mapping.
joe
****
On Aug 4, 10:15 pm, Joseph M. Newcomer <newco...@flounder.com> wrote:
How large are your files?
CFile f;
if(!f.Open(....))
deal with error
ULONGLONG size = f.GetLength();
ASSERT(size <= 0x0FFFFFFFull);
// Arbitrary choice of maximum length; you are probably in trouble
// if it is > 100MB or so, and the code below won't work, so you
// might choose a smaller length, e.g.
// ASSERT(size < 100000000ull);
CStringA b;
LPSTR p = b.GetBuffer(size + 1);
f,Read(p, size);
p[size] = '\0';
b,ReleaseBufferSetLength((int)size);
// Using ReleaseBufferSetLength means it doesn't have to search
// for the terminating NUL character to determine the length
int start = 0;
while(true)
{
int n = b.Find('X', start);
if(n < 0)
{ /* not found */
everything from start to end of string is the recor=
d of interest
break;
} /* not found */
everything from start to n-1 is the record of interest
start = n + 1;
}
This is rather simplistic, but if you don't try to create intermediate=
CStrings it can be
very fast. It works well only for small files (say, < 100MB). Fo=
r larger files, you
would apply the technique above to a memory-mapped file (there is some=
trickiness and you
can't use CStringA in this case, you have to go a bit lower-level beca=
use the strings will
not necessarily be NULL-terminated at the endpoint, and you have to de=
al with windowing
the mapping view into the larger file, but I'll assume your files are =
of moderate size and
therefore this more complex solution is not needed)
=
joe
On Mon, 4 Aug 2008 00:00:30 -0700 (PDT), Alexander <the44s...@yahoo.co=
m> wrote:
Ok. I've written a couple of implementations (derived from CStdioFile
and streams). They work fine but are much slower than CStdioFile whic=
h
is slow to being with. I need something fast for this.
Any ideas?
On Aug 4, 2:05 pm, "Check Abdoul" <check abdoul at mvps dot org>
wrote:
Derive a subclass from CStdioFile and overwrite ReadString(=
) function
and change its implementation[ ReadString() is virtual ]
Cheers
Check Abdoul
---------------------
"Alexander" <the44s...@yahoo.com> wrote in message
news:94225650-b700-490b-a1c8-c62a71f52700@a6g2000prm.googlegroups.co=
m...
I need a class exactly like CStdioFile but that on ReadString fetc=
hes
up to a character other than EOL.
Does such a thing exists? Thank you all.
Joseph M. Newcomer [MVP]
email: newco...@flounder.com
Web:http://www.flounder.com
MVP Tips:http://www.flounder.com/mvp_tips.htm
Joseph M. Newcomer [MVP]
email: newco...@flounder.com
Web:http://www.flounder.com
MVP Tips:http://www.flounder.com/mvp_tips.htm