Re: Process big Object in java
Colin Song <song6295@gmail.com> writes:
in my program,I use junit testcase call a webservice,and this
webservice call another webservice(I call it service provider).
code:
in junit testcase(web service client):
StringBuffer request = new StringBuffer();
One optimization is to make the StringBuffer big enough.
It won't cause memory leak to make it too small, but it will
cost extra time and allocation each time it's extended. So:
StringBuffer request = new StringBuffer(10000000);
String str = "!@#$%^ *<>";//10Byte
This is actually 10 chars. Chars in Strings are represented internally
as UTF16, so it's 20 bytes.
for(int j=0;j<1000000;j++)
{
request.append(str); //10MB
.... and 20 MB.
}
CommonServicePortClient proxy=new CommonServicePortClient();//
webservice client
proxy.sendRequest(request.toString());
This is not the entire test case. Roedy Green asked for a
SSCCE (http://mindprod.com/jgloss/sscce.html). The problem
might be in the parts you didn't show.
Is this problem occuring when you only run one test case once?
in server side(webservice deployed on OC4J):
It's a J2EE container from Oracle, right?
this webservice only pass parameter to another webservice(service
provicer),it is a common service,give a single interface to
customer,the implement :
public XXXXResponseDTO sendRequest(String request) throws
RemoteException,
FWServiceException {
XXXXServicePortClient proxy = null;
try {
proxy = new XXXXServicePortClient();
} catch (Exception e) {
e.printStackTrace();
}
// cast ,set url,so I can call many different service provider
XXXXServiceBinding_Stub stub = (XXXXServiceBinding_Stub) proxy
.getPort();
stub._setProperty(Stub.ENDPOINT_ADDRESS_PROPERTY,
this.getRequestType()
.getUrl());
// call service
return proxy.sendRequest(request);
This is a web service, so I guess it's using HTTP. I don't know much
about web services otherwise, nor do I recognize whatever framework
you are using.
You are sending a large string of characters that needs to be escaped
to occur in an URL. If the request goes into the URL, or if the
framework decides to escape the characters, then it'll about triple
the size. It might also keep the original around for
reference. That'll get you to 80 MB.
Now the service provider just return the request(big String ....).
and my webservice and service provider is on the same oc4j.
So each of these will have their own versions of this very big string.
It does add up. Not 500 MB, but getting there.
My primary concern is that HTTP might not be the best protocol for
transferring very large files, and specially not relaying them.
At least it has to be done carefully, so you won't get more than one
copy of it at any time, and preferably less. The service provider
should see if it's possible to build the request directly to a stream,
while reading its own input, instead of first building the string
internally, and then writing it all out again.
/L
--
Lasse Reichstein Nielsen - lrn@hotpop.com
DHTML Death Colors: <URL:http://www.infimum.dk/HTML/rasterTriangleDOM.html>
'Faith without judgement merely degrades the spirit divine.'