Re: Get "java.lang.OutOfMemoryError" when Parsing an XML useing DOM
On Mar 24, 3:00 am, "John W. Kennedy" <jwke...@attglobal.net> wrote:
NeoGeoSNK wrote:
On Mar 23, 3:23 pm, "Andrew Thompson" <andrewtho...@gmail.com> wrote:
On Mar 23, 5:53 pm, "NeoGeoSNK" <ny1...@gmail.com> wrote:
On Mar 23, 1:52 pm, "Mike Schilling" <mscottschill...@hotmail.com>
..
..SAX or a pull
parser would be far better choices.
*
..
I just use the java -Xmx1024m option to allocated 1GB memory to JVM,
but 40 minutes from now, it haven't work out the XML file
Another 20 minutes and it becomes an
'incomputable problem' according to
the definition as I vaguely recall..
* Sounds as though the task might be better
achieved using the optimal tools for the
job, rather than try to 'work around' the
problems of parsing the entire document
using DOM.
Andrew T.
Thanks,
I can't wait any more time, the job is take nearly 2 hours but haven't
finished yet.I think I'll try the SAX api, is there more fast api to
parsing XML in java?
SAX won't necessarily be /faster/ -- it could be a lot slower. It
depends on what you're doing.
Are you page-thrashing? If so, than SAX is definitely a good idea.
--
John W. Kennedy
"...if you had to fall in love with someone who was evil, I can see why
it was her."
-- "Alias"
* TagZilla 0.066 *http://tagzilla.mozdev.org- Hide quoted text -
- Show quoted text -
what "...if you had to fall in love with someone who was evil, I can
see why
it was her." means?
I don't know how DOM works when it parsing a XML, I use DOM that is
because the XPath can quciky location some particular elements. I
think if the SAX only reports events but not store the whole structure
of XML like DOM does, It must be more efficient. What does "page-
thrashing" means ?
I paste the source of the code:)
FYI
public Set parsing(String filename) throws Exception{
Set subset = new LinkedHashSet();
File f = new File(filename);
DocumentBuilderFactory factory =
DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(f);
Element root = doc.getDocumentElement();
XPathFactory xpfactory = XPathFactory.newInstance();
XPath path = xpfactory.newXPath();
NodeList recoredlist = (NodeList)path.evaluate("/journal/record",
doc, XPathConstants.NODESET);
// System.out.println("frameIdlist.getLength()= " +
recoredlist.getLength());
//enumerate all record in a log
for(int i = 0; i < recoredlist.getLength(); i ++){
// System.out.println("recoredlist = " + recoredlist.item(i));
Node record = recoredlist.item(i);
Element recordelement = (Element)record;
//System.out.println(recordelement.getTagName());
//get operat type
String BEtype = (String)path.evaluate("header/header_generic/domain/
@value", recordelement);
// System.out.println("operation type = " + BEtype);
if(!BEtype.equals("SHLR::Subscription"))
continue;
SubInfo subscriber = new SubInfo();
NodeList framelist = (NodeList)path.evaluate("body/frame",
recordelement, XPathConstants.NODESET);
// System.out.println("framelist = " + framelist.getLength());
//enumerate frame list in a record
for(int j = 0; j < framelist.getLength(); j++){
// System.out.println("frame = " + framelist.item(j));
NodeList attriblist = (NodeList)path.evaluate("attribute/
attribute_value/string/@value", framelist.item(j),
XPathConstants.NODESET);
for(int k = 0; k < attriblist.getLength(); k++){
//System.out.println(attriblist.item(k));
//System.out.println(attriblist.item(k).getClass());
Node attribute = attriblist.item(k);
String value = attribute.getNodeValue();
//String value = att.getAttribute("Value");
// System.out.println("Value = " + value);
if(value.equals("create")){
subscriber.setModifier("create");
}else{
if(value.equals("modify")){
subscriber.setModifier("modify");
}else{
if(value.equals("delete")){
subscriber.setModifier("delete");
}else{
if(value.trim().matches("dirNumberId.*")){
//System.out.println("dirNumberId = " +
value);
String dirnumber =
value.substring(value.indexOf("dirNumberId=") + 12,
value.indexOf(",sHLRSubsOrganizationId"));
String ndc =
value.substring(value.indexOf("nDCId=") + 6,
value.indexOf(",managedElementId=SHLR"));
// System.out.println("dirnumber=" +
dirnumber + ndc);
subscriber.setNDCId(ndc);
subscriber.setdirNumberId(dirnumber);
}else{
if(value.equals("calledList")){
Node calledattr = attriblist.item(k + 1);
String calledvalue =
calledattr.getNodeValue();
// System.out.println("calledList = " +
calledvalue);
if(calledvalue.equals("NULL"))
subscriber.removeCalledList();
else
subscriber.addCalledList(calledvalue);
}else{
if(value.equals("callingList")){
Node callingattr = attriblist.item(k + 1);
String callingvalue =
callingattr.getNodeValue();
// System.out.println("callingList = " +
callingvalue);
if(callingvalue.equals("NULL"))
subscriber.removeCallingList();
else
subscriber.addCallingList(callingvalue);
}else{
if(value.equals("lRNumberId")){
Node lrnattr = attriblist.item(k + 1);
String lrnvalue = lrnattr.getNodeValue();
subscriber.setlrnNumberId(lrnvalue);
}
}
}
}
}
}
}
}
}
if(subscriber != null)
subset.add(subscriber);
}
return subset;
}