Re: Detect XML document encodings with SAX
On 11/24/2012 4:18 PM, Sebastian wrote:
Am 24.11.2012 11:14, schrieb Lew:
[snip]
Obviously I don't know the answer, but he's asking for suggestions
to investigate, AIUI. He's having encoding problems. His XML is
apparently
encoded in Windows-1252, a notoriously funky encoding especially for
the variety of characters with which one might wish to deal. So why not
investigate obtaining material that isn't in such a notoriously funky
encoding, like, oh, say, the old reliable standard UTF-8?
Perhaps that isn't feasible, for reasons as yet unstated, but that's
the nature of brainstorming.
Here's the background to my question:
I am dealing with other people's code that processes XML files.
Unfortunately, that code, which I have no control over, seems to use
some home-grown parsing algorithm, which DOES NOT always detect
encodings correctly, but expects to be told them.
The XML files come from several sources in different encodings, and I
cannot dictate anything there either.
I would consider it tempting to rewrite that app to use a standard
XML parser.
It would solve this problem and possibly also some future problems.
So I thought, well, why don't I add a little preprocessor to discover
the encoding to give to that terrible file processor I'm stuck with.
Shouldn't be that hard, because, as Arne said:
> Am 24.11.2012 03:11, schrieb Arne Vajh?j:
> Obviously the parsers
> need to internally detect correct. Otherwise they
> could not parse correct.
The only approach that seems to work (at least for Arne), namely
W3C DOM, is out of the question for me, because the files are
potentially huge and I cannot keep a complete document model in memory.
I need something along the lines of SAX. I'll have to look around some
more.
What about just reading the first few lines until you have the
XML declaration.
Parsing the encoding out of that should be simple.
private static final Pattern encpat =
Pattern.compile("encoding\\s*=\\s*['\"]([^'\"]+)['\"]");
private static String detectSimple(String fnm) throws IOException {
BufferedReader br = new BufferedReader(new FileReader(fnm));
String firstpart = "";
while(!firstpart.contains(">")) firstpart += br.readLine();
br.close();
Matcher m = encpat.matcher(firstpart);
if(m.find()) {
return m.group(1);
} else {
return "Unknown";
}
}
I do not like the solution, but given the restrictions in the
context, then maybe it is what you need.
PS: The author of that article from which I took the code isn't just
anyone. Elliotte Rusty Harold hosts the XML web site
http://www.cafeconleche.org/ and is affiliated with the University of
North Carolina. Perhaps I could try to get in touch with him.
Teaching at a university is no guarantee of good practical
programming skills.
Arne