Re: HTTPUrlConnection does not download the whole page
On 3 Feb., 19:30, Lothar Kimmeringer <news200...@kimmeringer.de>
wrote:
The87Boy wrote:
public String getPage(String link) {
String pageEscaped = "";
try {
URL url = new URL(link);
// Open the Connection
HttpURLConnection conn = (HttpURLConnection)
url.openConnection();
// Set the information
conn.setRequestProperty("user_agent", =
"Mozilla/5.0
(Windows; U; Windows NT 6.0; da-DK; rv:1.9.1.4) Gecko/20091016 Firefox/
3.5.4 (.NET CLR 3.5.30729)");
conn.setRequestProperty("max_redirects", =
"0");
There is a set-Method to disable redirects, no need to set that
property directly.
Oh, I have not seen that ;)
conn.setRequestProperty("timeout", =
"300");
There are two methods allowing you to set the timeout for
connect and read, no need to set that property. Also it might
have no effect on the behavior of the connection-class, because
it most likely will not parse the data you set to the header.
Okay ;) I have already seen that, but I thought it was the same
conn.setRequestMethod("GET");
This is the default-method and only changes (also autoamtically)
if you set doInput to true.
Okay ;) Then there are no reason to set it
conn.setDoOutput(true);
// Connect
conn.connect();
You don't need call that, it happens already when calling
getInputStream.
Yes, but I have to get the status-code first, that's why I call the
connect
// Get the Status-Code and add it to the HashMa=
p
int statusCode = conn.getResponseCode();
What is the value of statusCode?
200
String page = this.getPage(conn.getInputStrea=
m());
[...]
} catch (IOException e) {System.err.println(e.getCause
());System.err.println(e.getMessage());}
A simple e.printStackTrace() should give out all the informations
you print here and more that are most likely valuable to find the
reason for problems.
Oh, I have always printed the exception using getCause and getMessage,
but now I know the easist way :D
public String getPage(InputStream is) throws IOException {
BufferedReader br = new BufferedReader(new InputStrea=
mReader
(is));
This uses the encoding of the system, not the encoding being
used by the server when sending the data, so you most likely
will corrupt your data.
How can I convert the encodings?
String line = "";
StringBuilder sb = new StringBuilder();
while ((line = br.readLine()) != null) {
sb.append(line+'\n');
System.out.println(line);
Any lines being given out while reading in data?
Yes, all the data
I thought it was the easist way to debug