Java program that reads websites to download from a file
Sometime back I wrote a simple Java program that reads a list of URLs
from a file and stores their contents on the local file system. I
have no problems with normal (i.e. html) pages, but I am not able to
download asp files. They are stored as zero length files.
I would greatly appreciate if someone could suggest a way out.
My source code follows:
/////////////////// Source code begin /////////////////////
// This program reads a list of URLs to access and store on the local
// file system from a file. The name of the file is passed as the
// first command line argument. Each URL is on a separate line.
// Lines beginning with the '#' character are treated as blanks and
// are skipped.
public static char replaceChar = '~';
public static void main(String argv) throws IOException
// The following two lines were suggested by the following
// They help in suppressing the java.net.MalformedURLException
if(argv.length != 0)
br = new BufferedReader(new FileReader(argv));
// Read URLs from the file. Skip blank lines and lines
// with the '#' character.
origName = br.readLine();
if(origName == null)
origName = origName.trim();
if(origName.length() == 0)
if(origName.charAt(0) == '#')
URL url = new URL(origName);
if(url == null)
BufferedReader bufRdr = new BufferedReader(new
// The name of the file to which the website contents are
// is derived from the URL by substituting the following
// with some "non-offending" character:
String modName = origName;
modName = modName.replace('\\', replaceChar);
modName = modName.replace('/', replaceChar);
modName = modName.replace(':', replaceChar);
modName = modName.replace('*', replaceChar);
modName = modName.replace('?', replaceChar);
modName = modName.replace('"', replaceChar);
modName = modName.replace('<', replaceChar);
modName = modName.replace('>', replaceChar);
modName = modName.replace('|', replaceChar);
FileWriter fWriter = new FileWriter(modName);
System.out.println("Writing contents of " + origName + " to "
"the following file: " + modName);
String thisLine = bufRdr.readLine();
if(thisLine == null)