Re: How to flush a Java Socket?

Hello,
 
I've run into an interesting phenomenon with sockets in Java.
 
Before I go through the long-winded explanation, here is the short
question for which I want an answer: How do I guarantee that a write
operation on a socket output stream has completed in Java? Things
that
don't work: calling flush on the stream, and setting SO_TCP_NODELAY.
Things that do: closing and reopening the socket, but this is a long
lived
connection and I can't afford to do that on every send.
 
So, here is the problem. I have a server process that reads byte
arrays
off the wire, processes them and sends data back over the same socket
connection.
The client may have multiple threads submitting to the server, but
uses a single socket for all of them. This is handled by queueing the
byte arrays to send and
having a single thread dequeue and write the arrays out to the socket.
 
Essentially, the send thread looks like this:
 
DataOutputStream sockOutStream=new
DataOutputStream(socket.getOutputStream());
 
while(true) {
 
  byte []sendData=outputQueue.dequeue();
 
  sockOutStream.write(sendData);
  sockOutStream.flush();
}
 
When testing some code that enqueues one byte array, waits for
notification that the dequeue and send took place, and then calls
System.exit(0) I
found that over 50% of the time the byte array didn't get to the
server. When I added a sleep(1000) to the end of the program, the
byte array
would get to the server every time.
 
Then, I removed the sleep() and the System.exit(0), and again the
event
got to the server every time. Eventually I figured out that it is
because
I have a finalizer which closes the socket, and it is called when the
program just runs until the end, but not when System.exit() is called.
 If
the socket isn't closed by socket.close(), the JVM may terminate
without
having sent all the data out the socket (regardless of the fact that I
called flush() on the socket output stream). So, the reason why the
byte array
isn't getting through is that the socket wasn't getting properly
flushed,
because finalizers don't get run by default on a call to
System.exit().
 
All fine and good. I added a shutdown hook which gets called during
both
a normal exit and a call to System.exit() so that now the socket gets
closed and flushed properly either way. Now the Java program works
100%
of the time with both a run to exit and a System.exit().
 
Unfortunately this isn't the end of the story. We also allow C++
clients to use our library, and provide this functionality by
embedding a JVM running our code inside a C++ application, and then
using a JNI based wrapper to let the C++ app use our client code. For
the C++ applications the bug is now back, because the C++ program
exits catastrophically without notifying the embedded JVM that it
needs to tie things up. Unfortunately now I am stuck because C++
doesn't have shutdown hooks, so I can't add some nice last minute code
to tidy things up any
more.
 
The root of the problem is that when the byte array is written, I want
to
guarantee that the data has been written out the socket to the server
when
control returns to the calling application (which actually happens
because the sending thread calls notify() waking up the actual sender,
but I think that is a technicality). That way the application can
crash, exit or do whatever it wants, but it will know that if it
received the notification from the send thread, the event was at least
written out the socket. If I could do this then I wouldn't have to
worry about all this shutdown hook mumbo-jumbo for pure Java clients
either. The problem is I can't see anyway to do that (given that
flush() doesn't work) other than closing and reopening a socket to the
server on every single call, which seems like incredible overkill.
 
So, has anybody run into this, or do they have any ideas? I've done a
search through the newsgroups on Google and haven't come up with
anything.
Also, I already have SO_TCP_NODELAY set, so that also doesn't help in
this
situation.
 
Answers and suggestions appreciated,
 
-Brad