Re: Class dependency problem
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Fri, 28 Jul 2006 09:28:49 +0100, Chris Uppal wrote:
Kenneth P. Turvey wrote:
The problem is that I would like the "Master" from the compute server to
be able to connect to the space and put a bunch of tasks in that it
would like the "Worker" nodes to compute. Then the "Master" should be
able to disconnect and be turned off.
But this is a completely different problem from the one you originally
described, where the worker could ask for classes it needed but didn't
have. The new version, fire-and-forget, is perfectly reasonable, but
requires a different architecture.
I looked at the previous articles in the thread. I think what I said is
pretty clear, but you know how that goes.. I do appreciate your effort to
decipher what I'm writing.
For a start, the job fired off to the worker must have /all/ the necessary
information for computation to complete (configuration, input data,
classfiles, ...). One natural medium for delivery of all that data would
be a JAR file containing all the needed. Note that in a fire-and-forget
environment it is better to be pessimistic about providing data, in
particular the effort of identifying a truly minimal set of required
classes would be a waste of time.
You have a point about the complexity of making sure all files are
available. Some of this can be handled by using a custom security model
(which is what I was planning on doing anyway) to disallow the kinds of
programming constructs that would cause problems.
So I would just fire off a JAR file containing all the calculation classes
with every request. There would be a lot of duplication, but so what ?
It would make your life much easier.
That might be the way to go. It just isn't as clean as calling a method
doMyWork().
If you do want to try to cut down the number of classes sent, then you'll
have to do it on the master machine, or in advance (perhaps by hand). If
you want to generate such a list automatically (and I repeat that I don't
think it would be worthwhile), then a good way to do it would be to scan
the bytecodes of the relevant classes, forming a transitive closure of the
class references starting at the "entry-point". The easy, but coarse, way
to track inter-class references is to look at each classfile's constant
pool for references to other classes. If you are up for a bit of extra
effort you could do the same kind of thing, but at method granularity. In
either case, a classfile library like ASM or BCEL would do the bulk of the
work for you. Note that you would have to find some way of handling
reflective references to class (Class.forName()) -- the simplest way of
doing that would be to forbid them. Alternatively look for literal
strings which are valid class names (in JNI syntax).
I'll think about this a bit.
There are tools available which will do this kind of analysis in order to
reduce the size of JAR files. You might be able to apply such a tool
directly, or "borrow" some of its code.
I used to use a tree-shaker under Lisp. I hadn't seen anything similar
under Java. I guess it doesn't surprise me that such a beast lives.
- --
Kenneth P. Turvey <kt-usenet@squeakydolphin.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
iD8DBQFEypfoi2ZgbrTULjoRAngEAJ40Nr4q54LJwX9S9FAplGSU/if5WQCfVqRU
tE/NKS3RUhq7LmURqiKocLg=
=RWs6
-----END PGP SIGNATURE-----