Re: Text File vs Database reading performance

Thanks for all the input, I will try to implement some of the tests
suggested.

1. processing = each line is basically a parameter/value pair,
depending on the parameter type I will use the value in some specific
computation.

2. some time = around 5 seconds per file (around 40,000 lines per
file), I have more than 200 files, so I'm looking at more than 15
minutes per "run"

3. I am using BufferedReader in the following way:

    private void readMarketDataFile(String file) throws
FileNotFoundException, IOException {
        String line;
        int numberLines = 0;
        BufferedReader in = new BufferedReader(new FileReader(file));
        while ((line = in.readLine()) != null) {
            numberLines++;
            try {
                analyseLine(line);
            }
            catch(Exception e) { e.printStackTrace(System.out); }
        }
        System.out.println("number of lines read: " + numberLines);
        in.close();
    }

my problem is that I feel I'm reading the file "in pieces" and
alternating between reading & processing. I'm wandering if there's a
faster way, to read the file at once and store elements in a data
structure, then analyse each entry of that data structure...

any take on that ?

thanks again

-Antoine