Re: Java URL

From:

Roland de Ruiter <roland.de.ruiter@example.invalid>

Newsgroups:

comp.lang.java.help

Date:

Tue, 06 Jun 2006 23:00:17 +0200

Message-ID:

<4485ece0$0$31638$e4fe514c@news.xs4all.nl>

Oliver Wong wrote:

<oceanb1114@gmail.com> wrote in message
news:1149559702.448495.21060@f6g2000cwb.googlegroups.com...

Exactly. What I did was this:

public void download(OutputStream os) throws IOException {
   byte[] buffer = new byte[2048]; //2K Buffer

   try {
     int pos = 0;

     URL targetUrl = new URL(this.url);
     URLConnection uc = targetUrl.openConnection();
     InputStream is = uc.getInputStream();

     while ((pos = is.read(buffer)) > 0)
       os.write(buffer, 0, pos);

     os.flush();
     os.close();
     is.close();
   } catch (Exception ex) {
     throw new IOException(ex.toString());
   }
}

But it won't work.

I've heard that Google blocks Java from connecting to it. Did you
test your program with URLs that don't point to one of Google's servers?
[...]

- Oliver

That's right, Google blocks Java clients. But it can be 'circumvented'
quite easily by setting the User-Agent request header with a value used
by well known browsers. In OP's code:

   ...
   URL targetUrl = new URL(this.url);
   URLConnection uc = targetUrl.openConnection();
   uc.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows
NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4");
   InputStream is = uc.getInputStream();
   ...

Note, that the string denoting the User-Agent value ("Mozilla/5.0 ...")
should be on one line (i.e. shouldn't contain line breaks).

Java's default value of the User-Agent header is "Java/1.5.0_06" or
similar: the version part in this string depends on which version of JRE
you have installed (or rather: which JRE version is executing the code).
--
Regards,

Roland