Re: Smartest way of compressing numbers over a webservice?
Casper wrote On 04/13/07 17:21,:
other things you will need to know;
what is the distribution of these numbers? (random, smooth &
contiguous, can you describe most of the numbers in a stream using a
function)
How much do they change (variance)? Standard deviation?
Is there a rule (function) or process on how these are generated?
Distribution is unknown, hence not mentioned.
How many variables are involved.
As I mentioned, potentially more then 100K.
Does a typical number series have a period? Do they repeat?
Does the number stream have some finite period where there is a
pattern and how often does it occur?
As I mentioned, from -50.0 and 50.0 meaning two digits with a separator
and a decimal. A span that would fit within 10bit if the decimal is
factored away by multiplying with 10.
Okay, let's take a step back. Suppose you do something
really straightforward and uncomplicated, like sending each
number as a five-byte string (sign, two digits, decimal
point, fraction digit). How much data must you transmit to
send 100K of such numbers?
Half a megabyte.
Even with a dial-up connection, less than two minutes.
Get a little fancier, and encode each of the 1001 possible
values as a two-character pair (A-Z and 0-9 gives 36 "digits,"
and 36*36 > 1001). Wow! You've saved sixty percent!
Which is 300KB.
Not quite one dial-up minute.
What I'm getting at here is that the potential gain from
exotic compression schemes seems fairly limited, unless your
circumstances are more exigent than you've let on so far.
It simply may not be worth your while to indulge in trickiness.
--
Eric.Sosman@sun.com