Re: How to search in a file efficiently (=fast !) for a certain hex
value ?
Arne VajhHj wrote:
Christian wrote:
John W. Kennedy schrieb:
Ulf Meinhardt wrote:
How can I efficiently (!) and fast search a file for a given hex
value (e.g. x'6A')
resp. how can I count the number of occurencies in a given hex value ?
java.nio.MappedByteBuffer
probably this will be even slower than just using an input Stream..
MappedByteBuffer may have some possibilitys for use.. but not for sth
like this.. secially if the file is too big... the os might have to
cache the file on another place of the disc..
It may be difficult to guarantee anything for something that is
so platform specific.
But common platforms should:
- map all the file into virtual memory
- only have some of it in physical memory
- read directly from the file with no use of pagefile or
other temp disk IO
to the op:
what is wrong with
BufferedInputStream bis = new BufferedInputStream(new
FileInputStream(file));
int read;
int[] timesRead = new int[256];
while (-1 != (read = bis.read()) {
timesRead[read & 0xff]++;
}
to read in the file..
it is simple, you won't get much faster and afterwards you can look up
the occurences easily ...
Even with BufferedInputStream I would go for multi byte read instead
of single byte read.
Arne
Just for curiosity I tried writing some simple programs to test relative
speed. Using a BufferedInputStream with the default buffer size and
reading 64k at a whack is about twice as fast as reading a single byte
at a time. I noticed on my computer that that method was only using one
core of the processor. So I thought I would write another program
that ran two threads reading from a FileChannel. The docs imply that
concurrent I/O will block but I thought that the non-reading thread
could be searching the buffer for the target byte. Only I can't get it
two work correctly. I've never used FileChannel I/O before and I don't
think I have something right. Please take a look at the example below
and tell me where I've gone wrong. Thanks,
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
public class test3 {
public static void main(String[] args) throws Exception {
String fname = "Fedora-8-x86_64-DVD.iso";
int count = 0;
FileInputStream fis = new FileInputStream(fname);
FileChannel fc = fis.getChannel();
ByteBuffer bybuf = ByteBuffer.allocate(65536);
long start = System.currentTimeMillis();
int n;
do {
bybuf.clear();
n = fc.read(bybuf);
bybuf.flip();
if (n > 0) {
try {
while (true)
if (bybuf.get() == (byte)0xfc)
++count;
} catch (BufferUnderflowException bue) { }
}
} while (n != -1) ;
long end = System.currentTimeMillis();
System.out.println(end - start);
System.out.println(count);
}
}
--
Knute Johnson
email s/nospam/knute/
--
Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
------->>>>>>http://www.NewsDemon.com<<<<<<------
Unlimited Access, Anonymous Accounts, Uncensored Broadband Access