Re: CStdioFile... with a twist (please)

From:

Alexander <the44secs@yahoo.com>

Newsgroups:

microsoft.public.vc.mfc

Date:

Tue, 5 Aug 2008 08:41:20 -0700 (PDT)

Message-ID:

<880e96b5-0ffe-4c29-9118-af1432cd4a30@v13g2000pro.googlegroups.com>

Ouch!

On Aug 5, 9:04 pm, Joseph M. Newcomer <newco...@flounder.com> wrote:

CStringA/CStringW do not exist in VC6. In fact, that was what finally =

got one of my VC6

"holdout" customers to switch to VS2003, because they work a lot with emb=

edded (ANSI)

systems but sell internationally (Unicode).
=

joe

On Mon, 4 Aug 2008 19:01:04 -0700 (PDT), Alexander <the44s...@yahoo.com> =

wrote:

The benchmark results are positive. Your approach matches the speed of
CStdioFile and since I have to read the file several times, loading
the contents only once actually makes this approach faster.

Memory mapped files look interesting. I'll take a look around and see
what I can... understand.

Speaking of (lack of) understanding, I can't get CStringA to
compile... gosh darnit, I'm dumb! What's the include? I'm using VC6...

On Aug 5, 5:16 am, Joseph M. Newcomer <newco...@flounder.com> wrote:

Another advantage of the file mapping is that it keeps your entire wor=

king set size down.

joe

On Mon, 4 Aug 2008 10:40:22 -0700 (PDT), Alexander <the44s...@yahoo.co=

m> wrote:

Thank you very very much, Joe. I now have all I need and then some.

On Aug 5, 12:37 am, Joseph M. Newcomer <newco...@flounder.com> wrot=

See below...

On Mon, 4 Aug 2008 07:12:40 -0700 (PDT), Alexander <the44s...@yahoo=

..com> wrote:

Thank you for taking the time to post an algorithm, Joe.

Your observations have, of course, hit on the issues that I've com=

across. CString operations are the biggest time suckers for sure (=

the

file is ASCII but the code must be UNICODE).

****
Note I used a CStringA. If, at some point, you need to convert, =

you can do

CStringA t;
CString s(t);
and s will be the Unicode version of the ANSI characters in t. I us=

e this trick fairly

often when I have to deal with 8-bit data streams in Unicode apps. =

But if you can

postpone the conversion so the conversion is folded in with the cre=

ation, it will work

better; for example
CString s(p, n);
where p is an LPSTR/LPCSTR (not LPTSTR/LPCTSTR) and n is the number=

of characters to

convert will give you a Unicode string with just one copy operation=

(and a

MultiByteToWideChar conversion)
****>I wonder, though, about

loading the entire file at once... somehow I had convinced myself =

that

it would slow things down.

****
Why? You have to read all the file eventually...
****

The test file is small, approx. 4 MB (the "real" one 60~90 MB). I'=

add this code to the benchmark and see how it fares.

****
Memory-mapped files may also give better performance than a large R=

eadFile; they're harder

to use but you don't actually bring the data in until it is touched=

.. If you window data,

it is a bit tricky because the windows have to fall on 64K boundari=

es, so you have to

"back up" to the previous 64K boundary if you hit an edge, e.g., if=

each of the letters

below is a 64K block
ABCDEFG
and you map AB into memory, when you get to the end of B you have o=

nly a partial line, so

you would then have to map
BC
and then
CD
and so on so you could see the entire string (assuming it is <64K; =

otherwise you would map

more pages to cover the maximum string length). Note that you do=

n't have to map the

minimum number of pages; you could map 100 pages or 300 pages or wh=

atever you want, but

when you hit a boundary, you have to include the 64K block in which=

the record started in

the next mapping.
joe
****

On Aug 4, 10:15 pm, Joseph M. Newcomer <newco...@flounder.com> w=

rote:

How large are your files?

CFile f;
if(!f.Open(....))
deal with error

ULONGLONG size = f.GetLength();
ASSERT(size <= 0x0FFFFFFFull);
// Arbitrary choice of maximum length; you are probably in troub=

// if it is > 100MB or so, and the code below won't work, so you
// might choose a smaller length, e.g.
// ASSERT(size < 100000000ull);

CStringA b;
LPSTR p = b.GetBuffer(size + 1);

f,Read(p, size);
p[size] = '\0';

b,ReleaseBufferSetLength((int)size);

// Using ReleaseBufferSetLength means it doesn't have to sear=

// for the terminating NUL character to determine the length

int start = 0;
while(true)
   {
    int n = b.Find('X', start);
    if(n < 0)
        { /* not found */
         everything from start to end of string is the=

record of interest

         break;
        } /* not found */
    everything from start to n-1 is the record of interest
    start = n + 1;
   }

This is rather simplistic, but if you don't try to create interm=

ediate CStrings it can be

very fast. It works well only for small files (say, < 100MB).=

For larger files, you

would apply the technique above to a memory-mapped file (there i=

s some trickiness and you

can't use CStringA in this case, you have to go a bit lower-leve=

l because the strings will

not necessarily be NULL-terminated at the endpoint, and you have=

to deal with windowing

the mapping view into the larger file, but I'll assume your file=

s are of moderate size and

therefore this more complex solution is not needed)
=

joe

On Mon, 4 Aug 2008 00:00:30 -0700 (PDT), Alexander <the44s...@ya=

hoo.com> wrote:

Ok. I've written a couple of implementations (derived from CStd=

ioFile

and streams). They work fine but are much slower than CStdioFil=

e which

is slow to being with. I need something fast for this.

Any ideas?

On Aug 4, 2:05 pm, "Check Abdoul" <check abdoul at mvps dot o=

rg>

wrote:

Derive a subclass from CStdioFile and overwrite ReadS=

tring() function

and change its implementation[ ReadString() is virtual ]

Cheers
Check Abdoul
---------------------

"Alexander" <the44s...@yahoo.com> wrote in message

news:94225650-b700-490b-a1c8-c62a71f52700@a6g2000prm.googlegro=

ups.com...

I need a class exactly like CStdioFile but that on ReadStrin=

g fetches

up to a character other than EOL.

Does such a thing exists? Thank you all.

Joseph M. Newcomer [MVP]
email: newco...@flounder.com
Web:http://www.flounder.com
MVP Tips:http://www.flounder.com/mvp_tips.htm