Re: Can extra processing threads help in this case?

From:

"Peter Olcott" <NoSpam@OCR4Screen.com>

Newsgroups:

microsoft.public.vc.mfc

Date:

Tue, 23 Mar 2010 00:32:10 -0500

Message-ID:

<lM2dnZWy4JjBzzXWnZ2dnUVZ_qCdnZ2d@giganews.com>

"Hector Santos" <sant9442@nospam.gmail.com> wrote in message
news:etOekekyKHA.5036@TK2MSFTNGP02.phx.gbl...

Hmmmmm, you mean two threads in one process?

What is this:

num = Data[num]

Do you mean:

num = Data[i];

No I mean it just like it is. I init all of memory with
random numbers and then access the memory location
references by these random numbers in a tight loop. This
attempts to force memory bandwidth use to its limit. Even
with four cores I do not reach the limit.

What are the heuristics for making a process thread safe?
(1) keep all data in locals to the best extent possible.
(2) Eliminate the need for global data that must be written
to if possible.
(3) Global data that must be read from is OK
(4) Only use thread safe libraries.

I think If I can follow all those rules, then the much more
complex rules aren't even needed.

Did I miss anything?

Take the posted code I gave you and change this part:

#define USE_STD_VECTOR
#include <vector>

//------------------------------------------------------
// Parameters to play with
//------------------------------------------------------

#define KIND DWORD // array element
type
#define MAX_THREADS 64 // # of threads
DWORD nRepeat = 10; // data access
repeats
DWORD nTotalThreads = 2; // # of threads
DWORD size = MAXLONG/6; // ~1.4GB
#idef USE_STD_VECTOR
std::vector<KIND> *data = NULL;
#else
KIND *data = NULL;
#endif

//------------------------------------------------------
// Functions to simulate application work load
// The process data function simply reads the
// memory.
//------------------------------------------------------

BOOL AllocateData()
{
   DWORD t1 = GetTickCount();
   _cprintf("- Allocating Data:ram .... ");
#idef USE_STD_VECTOR
    data = new std::vector<KIND>(size);
#else
    data = new KIND[size];
#endif
   _cprintf("%d\n",GetTickCount()-t1);
   return TRUE;
}

void DeallocateData()
{
   if (bUseFileMap) {
      fmdata.Close();
   } else {
      delete data;
   }
}

#pragma optimize("",off)
void ProcessData()
{
   KIND num;
   for(DWORD r = 0; r < nRepeat; r++) {
      for (DWORD i=0; i < size; i++) {
         DWORD j = i;
#idef USE_STD_VECTOR
         num = (*data)[j];
#else
         num = data[j];
#endif
      }
   }
}
#pragma optimize("",on)

And run it with no switches and then /t:2 and /t:4.

WATCH it performs for better!

I would also explore it with USE_STD_VECTOR commented out.

--

Peter Olcott wrote:

The code below apparently proves that you were right all
along.
I ran it as two separate processes and it took a like
16.5 seconds for one instance and 16.55 seconds for two
instances.

"Hector Santos" <sant9442@nospam.gmail.com> wrote in
message news:uwC5O9jyKHA.3884@TK2MSFTNGP06.phx.gbl...

Peter Olcott wrote:

Try running your process again using a
std::vector<unsigned int>
Make sure that you initialize all of this to the
subscript of the init loop.
Make sure that the process monitor shows that the
amount of memory you are allocating is the same amount
that total memory is reduced by.
Make sure that you only use 1/2 of total memory or
less.
Make a not of the page fault behavior.
I will try the same thing.

You better! :)

I'll BE BACK!

--
HLS

#include <stdio.h>
#include <stdlib.h>
#include <vector>
#include <time.h>

#define uint32 unsigned int

const uint32 repeat = 100;
const uint32 size = 524288000 / 4;
std::vector<uint32> Data;

void Process() {
  clock_t finish;
  clock_t start = clock();
  double duration;
  uint32 num;
  for (uint32 r = 0; r < repeat; r++)
    for (uint32 i = 0; i < size; i++)
      num = Data[num];
   finish = clock();
   duration = (double)(finish - start) / CLOCKS_PER_SEC;
   printf("%4.2f Seconds\n", duration);
}

int main() {
  printf("Size in bytes--->%d\n", size * 4);
  Data.reserve(size);
  for (int N = 0; N < size; N++)
    Data.push_back(rand() % size);

  char N;
  printf("Hit any key to Continue:");
  scanf("%c", &N);

  Process();

return 0;
}

--
HLS