Re: Can extra processing threads help in this case?
Joseph M. Newcomer wrote:
Although it may be temporary ignorance on my part that
suggested such a thing, I was thinking that it might be
simpler to do it this way because every client request will
be associated with a financial transaction. Each financial
transaction depends upon its corresponding client request
and each client request depends upon its corresponding
financial transaction. With such mutual dependency it only
seemed natural for the underlying representation to be
singular.
****
There are several possible approaches here, with different tradeoffs.
(a) get PayPal acknowledgement before starting the transaction
(b) Because the amount is so small, extend credit, and do the PayPal processing "offline"
out of the main processsing thread; in fact, don't even request the PayPal debiting until
the transaction has completed, and if it is refused, put the PayPal processing FIRST for
their next request (thus penalizing those who have had a refused transaction); you might
lost a dime here and there, but you have high performance for other than those who aren't
cleared.
(c) if the transaction fails, and you have already debited the account, have a background
process credit the account for the failed transaction.
You are confusing IPC with robustness mechanism. IPC is a pure and simply a transport
mechanism; anything about robustness has to be implemented external to the IPC.
joe
Joe, why is his application any different or special than any other
client/server framework?
He stated:
100 ms RTPT (Response Time per Transaction).
That is a 1 thread/process rate. That means within 1 sec, 1 thread
can handle 10 transactions per second.
Since he also stated:
100 TPS (Transactions per Sec)
That means he will need at least 10 threads in his worker pool -
period to handle the 100 TPS at 100 ms RTPT.
His HTTP posted input REQUEST data will have at a minimum:
- HTTP or COOKIE based authentication data, UserAuthData:
- MIME based form fields meta data information, UserMetaData
- MIME based form upload file data, UserFileData
So when a HTTP POST arrives, he needs a "TRequestMetaData" structure
with the above plus session related state information. He can start
with a enum state point type:
enum TRequestState {
rsNone = 0, // just to make it understood
rsPosted,
rsProcessing,
rsProcessingComplete,
rsSendingResult,
rsSendingComplete,
rsSuccess
};
Altogether, TRequestMetaData structure will contain:
struct TRequestMetaData {
TUserAuthdata UserAuthdata;
TUserMetaData UserMetaData;
TUserFileData UserFileData;
TUserRespData UserRespData;
FILETIME PostTime;
FILETIME ProcessTime;
FILETIME CompleteTime;
TRequestState RequestState
};
As long as he keeps this fixed typedef structure, he won't have to
worry about class serialization and OLE/VARIANT. However, he can use
a CRecordSet with a data exchange handler for an ODBC interface.
Either way, the fundamental state machine needs to be established.
TRequestMetaData needs to immediately flushed to disk using whatever
storage method he wishes to use at each update point:
1 - Request.Add()
Initialize new TRequestMetaData record
set RequestState = rsPosted
set some HASH for the HTTP data request
RESTART CHECK
- Lookup by HASH to see if request
already exist. Go to Proper
state accordindly.
For new, Append to data file/base
Get new row index
2 - Request.StartProcessing()
set RequestState = rsProcessing
set initialize other processing information
Update Record at index location
call the DFA/OCR processor (in threaded fashion)
wait for completion, thread efficient BLOCKED
set RequestState = rsProcessingComplete
set other processing complete information
Update Record at index location
3 - Request.SendResult
Prepare HTTP response
set RequestState = rsSendingResult
set UserReponseData = HTTP response
Update Record at index location
Send HTTP result, it is blocked, the
only reason for failure of the user disappeared
set RequestState = rsSendingComplete
set other sending complete information
Update Record at index location
4 - Request.Complete()
set RequestState = rsComplete
set other processing complete information
Update Record at index location
The above is the basic idea. Of course there are parts that could be
folded or reduced or done better.
But as Peter D. told Peter O, now he doesn't have to worry about much
about faults at any of the above states. Each request state will be
known and he can restart or rehandle accordingly. And as you and I
told him, he has to negate his idea about no pages, no caching, not
because thats bad - its insignificant.
Anyway, the only thing really different from any other common
application framework like this, is state #2 and the plug and play
DFA/OCR processor.
Just consider the odds are good the BUGS and FAULTS will be at this
point. :)
--
HLS