Re: Searching a disk-backed Map
Arne Vajh?j wrote:
Patricia Shanahan wrote:
Stefan Ram wrote:
This should be a common need. Yet I am not aware of anything
like it in Java SE. What is the most common (pure Java)
solution to it?
I would like to have an implementation of java.util.Map,
which is constructed with an int ?m? and a java.io.File ?f?.
It will use no more than ?m? bytes of memory, but ?swap? out
(the least often used) entries to the file ?f?, when they do
not fit into the given memory size anymore.
Have you considered putting the data in a database instead, and using
java.sql to access it? The data structures and algorithms that Java uses
for in-memory maps are not very suitable for disk-based maps. Database
managers use structures and algorithms designed for the job.
Map is an interface and does not really specify algorithms or
data structures. 15 seconds looking at Map indicates that most
of the stuff could easily be implemented on top of a database.
Only entrySet, keySet and values would require non-trivial code.
But that is probably an unimportant question. The important question
is: would having a database backed map provide any benefits over
just using the database directly. I can not see any. A Map is
more general but having memory access and disk access being
transparent is a double edged sword.
Given this problem, I would probably write an interface with exactly the
needed methods and parameter types for the job. One implementation, for
small scale testing, would use a Map to implement the interface. Another
would use a disk based implementation, possibly based on a database.
There is no sense in making the disk implementation more general or
powerful than it needs to be for the actual task.
Note that I am not thinking in terms of the full power of a database,
but of a cheap way of getting an indexed disk file, by having a single
table with each row containing a key-value pair, indexed on the key.
Patricia