Re: speeding up URLConnection reading

From:

"Daniel Pitts" <googlegroupie@coloraura.com>

Newsgroups:

comp.lang.java.programmer

Date:

4 Nov 2006 15:12:11 -0800

Message-ID:

<1162681931.230706.174480@b28g2000cwb.googlegroups.com>

mark wrote:

Hello,

I want to read the content of some webpages and make some string
comparisons with them (i.e. check if there is some text in it, use some
regular expressions, etc.).

StringBuilder htmlCode = new StringBuilder();
URL url = new URL(fileName);
URLConnection conn = url.openConnection();
conn.connect();
BufferedReader dis = new BufferedReader(new
InputStreamReader(conn.getInputStream()));
String inputLine = "";
for(;;) {
      inputLine = dis.readLine();
        if (inputLine == null) break;
        htmlCode.append(inputLine);
}

It works, but it is very, very slow comparing to browser. Do you know
any ways to speed it up??

Regards, mark

Don't use a buffered reader, as you don't need to read it one line at a
time.

        final URL url = new URL(adjustUrl(page));
        final HttpURLConnection connection = (HttpURLConnection)
url.openConnection();

        connection.setRequestMethod(method);
        connection.connect();
        try {
            final InputStream is = connection.getInputStream();
            final Reader reader = new InputStreamReader(is);
            final char[] buf = new char[1024];
            int read;
            final StringBuffer sb = new StringBuffer();
            while((read = reader.read(buf)) > 0) {
                sb.append(buf, 0, read);
            }
        } finally {
            connection.disconnect();
        }