Re: java thread for core 2 due processors
On Mon, 17 Aug 2009, Motaz K. Saad wrote:
I am working on file processing (large number of files) which is totally
independent task for each file. I want to speedup processing using java
thread but I do not want overwhelm my processor (core 2 due processor)
with all threads at the same time.
You'd have to have an awful lot of threads to 'overwhelm' the processor -
hundreds, or perhaps even thousands. There is a chance that it might
overwhelm the disk, though, depending on things like how big your files
are and how the threads access them.
However, running your jobs in 1000 threads won't be any faster than
running them in 10 (although that might be faster than running them in 2,
because it lets you saturate the disk - two threads on two CPUs will leave
either the disk or CPU underutilised at some point, unless readahead and
GC keep them busy), so your conclusion is right - you want to use fewer
threads than you have tasks.
I want to run 2 parallel threads simultaneously and wait until they
finish then run the next 2 threads.
No, you want to run two parallel tasks simultaneously and wait until they
finish then run the next two tasks. You don't need to have one thread per
task.
furthermore, each thread allocate large amount of memory so I need to
de-allocate the thread after collecting the result from it.
As long as there are no pointers to the allocated objects after the file
is processed, the garbage collector will reclaim it.
I would appreciate if anyone direct my to similar set example I would
appreciate guideline and help
You want:
http://java.sun.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
Your code looks like:
public class FileProcessingTask implements Runnable {
private final File file;
public FileProcessingTask(File file) {
this.file = file;
}
public void run() {
try {
// process file
}
catch (Exception e) {
// log the exception
// close any open files
}
}
}
public class FileProcessingApp {
public static void main(String... args) {
Collection<File> filesToProcess; // initialise this however you like
int numThreads = Runtime.getRuntime().availableProcessors() * 2;
ExecutorService executor = Executors.newFixedThreadPool(numThreads);
for (File file: filesToProcess) {
executor.execute(new FileProcessingTask(file));
}
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
}
}
tom
--
Brace yourself for an engulfing, cowardly autotroph! I want your
photosynthetic apparatii!