Re: Can extra processing threads help in this case?
"Joseph M. Newcomer" <newcomer@flounder.com> wrote in
message news:r7jkr55trmr5langp6gukohsb6d7dq9lu3@4ax.com...
In the case where pread() and pwrite() must work together
to
provide a single atomic operation, such as the case of
deducting ten cents from a customer's account balance you
are correct. A simpler way to provide this atomicity for
my
purposes might be to only have a single thread that
updates
the client's balances. In this case there is no need of
contriving a record lock on a system that lacks this
inherent capability.
****
Actually, they do not provide a single atomic operation,
and this should be obvious. The
CORRECT way to provide the atomicity is use a transacted
database. Nothing else is going
to be reliable, unless, of course, you re-implement
transacted databases on your own.
When one is referring to transactions one is referring to at
least one read operation combined with at least one write
operation forming the single atomic transaction unit. This
is useful when one may be deducting a dime from the
customer's current account balance.
There are other less inclusive forms of atomicity that can
also be relevant. If one thread needs to write to a record
in a file (specific byte offset within the file), and
another thread needs to append a record (fixed length set of
bytes) to a file it is useful to know that these operations
can complete without interfering with each other. This level
of atomicity is sufficient for updating my transaction log,
yet insufficient for updating the customer's account
balance.
If you think otherwise, you do not have a clue about what
a transacted database does, why
they are important, and why there are no alternatives!
****
In other less complex cases such as updating a status flag
from [InProccesing] to [Completed] it would seem that the
single guarantee that the pwrite() operation itself is
atomic would provide sufficient atomicity for the current
design (only one writer per record).
****
pwrite() does not guarantee an atomic transaction. Your
failure to comprehend this after
It doesn't have to, there are no transactions involved.
I have explained it several times is why I keep being
forced to apply the term "stupid".
Because you are not paying enough attention to what I am
saying.
THERE IS NO GUARANTEE! You did not find one in the
documentation, and you will not find
There is a guarantee that two pwrites() will not collide
with each other resulting in garbage. This is sufficient for
the transaction log, yet insufficient for updating the
customer's account balance.
one in the implementation. You have confused a concept
(atomic modification to a file)
with a much more important conept (a transacted database
that DOES NOT LOSE INFORMATION AS
A CONSEQUENCE OF ANY ABNORMAL TERMINATION OF THE APP OR
SYSTEM!)
Which as I have already said, is easily prevented by using
the design pattern in SQLite.
Please explain how pwrite guarantees transactional
integrity. You obviously have some
information which is not in the documentation and cannot
be derived by reading the linux
source code.
The transaction log does not need transactional integrity.
It does need crash fault tolerance, which might be construed
as a form of transactional integrity. I could make all of
this moot by simply having a single process with a single
thread that handles all disk writes to a specific resource.
This would inherently have transaction integrity, and crash
proofing would be as easy as using the SQLite design
pattern.
Consider the following
Let f be a file handle
Let f be an empty file when this code starts
char buf = '*';
for(int i = 0; i < SOME_SIZE; i++)
pwrite(f,&buf, 1, i);
Now, if the systyem fails for ANY REASON in the loop, you
will, upon restart, find in the
file represented by handle f:
(1) SOME_SIZE instances of '*'
(2) 0 <= i <= SOME_SIZE instances of '*', depending on the
exact value of i when the crash
occurred
(3) 0 instanceas of '*'
(4) Some indeterminate number of instances, m, of '*', 0
<= m <= i where i is the value
achieved when the system crashed, where m is unrelated to
the value of i when the system
crashed exxept that it must be less than or equal to it
Hint: the correct answer is 4.
Which is very easy to handle using the SQLite design
pattern.
Question: suppose the loop completes, and you are well
into some other, much later,
computation when the crash occurs.
Hint: the correct answer is 4.
Now, consider the following, where I hypotehsize the
existence of two primitives
BEGIN_TRANSACTION starts a transaction of the file handle
END_TRANSACTiON ends the transaction and commits the
changes to the file
BEGIN_TRANSACTION(f); // [1]
for(int i = 0; i < SOME_SIZE; i++) // [2]
prwrite(...as above...); // [3]
END_TRANSACTION(f) // [4]
If there is an abnormal temrination before line [4]
executes completely, the nuimber of
instances of * found will be
(choose one of the above answers)
Hint: the correct answer is 3
I already know what transaction processing is.
In fact, there are points in the implementaiton of line 4
where either the transaction has
not been committed, and the correct answer is 3, or the
transaction is committed and the
correct answer is 1, but there is NEVER any point in the
code where any other answer is
possible! And that's why transacted systems are HARD to
write!
Now consider the following:
for(int i = 0; i < SOMES_SIZE; i++) // [1]
{
BEGIN_TRANSACTION(f); // [2]
pwrite(...as above...); // [3]
END_TRANSACTION(f); // [4]
}
// [5]
Choose one of the answers.
Hint: the correct answer is 2, and the exact value depends
on whether or not the crash
took place on lines1, 2, 3, 4 or 5. If it took place on
any line but 5, then the number
of *s will be i - 1. If it took place on line 5, the
number of *s will be i..
If you do not understand the reasons why wrapping the file
operation in a hyopthetical
transaction makes a difference, go back and undertand what
a transacted database does. In
the case of a transacted database, the "add new record"
operation or "update record"
operation replaces pwrite.
****
That is exactly what the SQLite design pattern does, and it
allows clean restarts even after power failures.