Re: Wonky HTTP behavior?
Twisted wrote:
The code generating the initial connection is:
uc = (HttpURLConnection)u.url.openConnection();
uc.setInstanceFollowRedirects(true); // Auto follow what redirects we
can.
uc.setRequestProperty("User-Agent", Main.USER_AGENT);
uc.setRequestProperty("Accept-Language", "en-us,en;q=0.5");
uc.setRequestProperty("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7");
uc.setRequestProperty("Keep-Alive", "300");
if (u.prevURL != null) {
uc.setRequestProperty("Referer", u.prevURL);
}
if (Main.COOKIE_ENABLE) {
String k = getCookiesFor(u.url.getHost());
if (k != null) uc.setRequestProperty("Cookie", k);
}
uc.connect();
So, three questions:
1. Is the "Content-type" header what's screwing things up and causing
400 errors or worse?
2. How do I get rid of it?
3. Is the header rearrangement (relative to what Firefox outputs) a
likely source of problems?
I tried your code as below and got
Response code: 200 'OK'.
------------------------------------------------------------------------
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
public class WebGet {
public static void main(String[] args) throws IOException {
/*
* Contrived class so Twisted's code snippet
* can be used unchanged.
*/
class Foo {
URL url;
String prevURL;
Foo() {
try {
url = new URL("http://www.movingtofreedom.org/" +
"2006/11/12/" +
"a-round-of-gnu-linux-" +
"heading-in-to-the-back-nine-part-2/");
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
}
Foo u = new Foo();
HttpURLConnection uc;
// Twisted's code starts here
// ..........................................................
uc = (HttpURLConnection)u.url.openConnection();
uc.setInstanceFollowRedirects(true);
uc.setRequestProperty("User-Agent", Main.USER_AGENT);
uc.setRequestProperty("Accept-Language", "en-us,en;q=0.5");
uc.setRequestProperty("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7");
uc.setRequestProperty("Keep-Alive", "300");
if (u.prevURL != null) {
uc.setRequestProperty("Referer", u.prevURL);
}
if (Main.COOKIE_ENABLE) {
String k = getCookiesFor(u.url.getHost());
if (k != null) uc.setRequestProperty("Cookie", k);
}
uc.connect();
// ..........................................................
// Twisted's code ends here
System.out.println("Response code: "+uc.getResponseCode() +
" '"+uc.getResponseMessage()+"'.");
} // main()
private static String getCookiesFor(String host) {
return null;
}
} // WebGet
class Main {
static String USER_AGENT = "Foo";
static Boolean COOKIE_ENABLE = false;
}
------------------------------------------------------------------------