Re: Can extra processing threads help in this case?

Peter Olcott wrote:


IMO, it just that its an odd approach to load balancing. You are
integrating software components, like a web server with an
multi-thread ready listening server and you are hampering it with a
single thread only FIFO queuing. It introduces other design
considerations. Namely, you will need to consider a store and forward
concept for your request and delayed responses. But if your request
processing is very fast, maybe you don't need to worry about it.

In practice the "FIFO" would be at the socket level or listening level
with concepts dealing with load balancing by restricting and balancing
your connection with worker pools or simply letting it to wait knowing
that processing won't be too long. Some servers have guidelines for
waiting limits. For the WEB, I am not recall coming across any
specific guideline other than a practical one per implementation. The
point is you don't want the customers waiting too long - but what is
"too long."


Any good collection class will do as long as you wrap it with
synchronization. Example:

typedef struct _tagTSlaveData {
    ... data per request.........
} TSlaveData;

class CBucket : public std::list<TSlaveData>
{
public:
     CBucket() { InitializeCriticalSection(&cs); }
     ~CBucket() { DeleteCriticalSection(&cs); }

     void Add( const TSlaveData &o )
       {
          EnterCriticalSection(&cs);
          insert(end(), o );
          LeaveCriticalSection(&cs);
       }

     BOOL Fetch(TSlaveData &o)
       {
          EnterCriticalSection(&cs);
          BOOL res = !empty();
          if (res) {
             o = front();
             pop_front();
          }
          LeaveCriticalSection(&cs);
          return res;
       }
private:
    CRITICAL_SECTION cs;
} Bucket;

--
HLS