Re: concurrent DBM for Java?
On 04/03/2010 12:23 AM, Jim Janney wrote:
Ken <ken@kenmcwilliams.com> writes:
On Apr 1, 3:01 pm, Jim Janney <jjan...@shell.xmission.com> wrote:
Arne Vajh=F8j <a...@vajhoej.dk> writes:
On 01-04-2010 16:08, Jim Janney wrote:
Arne Vajh=F8j<a...@vajhoej.dk> writes:
On 01-04-2010 15:11, Jim Janney wrote:
I need to maintain a data base of small text snippets keyed by
arbitrary strings, without the overhead of a full SQL relational
database. We will have several people putting data into it so it=
needs to support concurrent access over a network.
JDBM catches my eye (http://jdbm.sourceforge.net/) but I can't fi=
nd
any place that says whether it supports concurrent access or not.=
Berkeley DB would be perfect but would need a license from Oracle=
since this is a commercial product. Gdbm apparently does not sup=
port
concurrent access. TDB (http://sourceforge.net/projects/tdb/) lo=
oks
good but would require a JNI interface and probably some work to =
get
it to compile under Windows.
Any recommendations? I can do JNI if neccessary but would be jus=
t as
happy not to need to. I need something I can get going quickly -=
-
management wants it yesterday, as usual...
Why are you so sure that a full SQL relational database has
too much overhead?
Five years of experience using a relational database in the
application for which this is intended. They're great for some
things, and we use it for those things. This is not one of them.
And the simple solution of buying some more powerful
database hardware is not an option?
Tell all of our customers they have to upgrade their network and buy =
a
faster AS/400? No, that's not going to fly.
You have an AS/400... I don't understand what the issue is... are you
trying to replace the AS/400?
If the AS/400 is fast enough and for some reason think that SQL is too=
slow, if it's an old dieing 400 I could see that. But if you only
have a few hundred users and it's a half way modern iSeries I couldn't=
imagine them being able to stress it with SQL, unless they're day
traders!!!
Anyways if the AS/400 is fast enough what is wrong with its record
level access? I've never tried to use Java for record level access
(but I very much bet it's possible via JTOpen) I've only it in RPG (if=
you don't program in RPG see SETLL and CHAIN) and even then I'd rather=
use sql than doing record level. The problem with it is if you
support more than one client. Different clients are going to have
different data needs and so an optimal set of CHAINs for one customer
is not going to be the same for another and if this simple group of
strings ever has a dependency with data elsewhere... it's just so much=
harder to manage than simply using transactions.
I haven't done much research on Express-C (isn't it free?), it might
have the same functionality and you'd probably be more comfortable
with a flavor of DB2.
Our customers have AS/400s, some of them dating back to 2001. Either
SQL or record level access would be more than fast enough if the code
ran directly on the 400, but when you connect over a network the
latency goes way up, and I wanted to be able to do a bunch of
unrelated queries (roughly 10 to 200) with no perceptible delay to the
user (this is part of displaying a screen).
I'm beginning to think that the same problem would apply to anything I
do over a network, so I'm currently rethinking my whole approach to
the problem. The data is small enough I can cache it all locally, but
I need to reload it when it changes.
Network latencies are pretty low these days. And even if you are
spending a few milliseconds the user won't probably notice.
In situations like these this is what I typically do: I create a toy
example that models important aspects of the original problem and
benchmark it. In your case I would create a simplified schema that
covers what you need and create a console only application that accesses =
it in meaningful ways.
One solution to make JDBC with multiple queries fast is using the JDBC
driver's batch mode. That way you may reduce network roundtrips. The
efficiency of this of course depends on the driver and DB at hand.
Given that DB2 is a mature product and IBM is heavily involved in Java
for years I expect it to work pretty well.
Another solution would be to write stored procedures that smartly return =
the data you need. Whether that is feasible depends of course on the
application logic.
In any case, if customers do have an AS/400 already and if they have DB2 =
on that machine already I'd try to use that. Advantage is that they do
not need to buy new licenses, install additional software, train admins
etc. and you can be sure that the DB is capable of handling the load
(and probably much higher load).
Depending on your data and access logic it may even be simple to hack
some client side caching together on top of JDBC.
Even if that approach fails or you have extreme requirements for
performance I would look into "non standard" solutions.
Kind regards
robert
--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/