com.clearspring.analytics.stream.quantile

Class QDigest

java.lang.Object

com.clearspring.analytics.stream.quantile.QDigest

All Implemented Interfaces:

IQuantileEstimator

public class QDigest
extends Object
implements IQuantileEstimator

Q-Digest datastructure.

Answers approximate quantile queries: actual rank of the result of query(q) is in q-eps .. q+eps, where eps = log(sigma)/compressionFactor and log(sigma) is ceiling of binary log of the largest value inserted, i.e. height of the tree.

Two Q-Digests can be joined (see unionOf(QDigest, QDigest)).

Source: N.Shrivastava, C.Buragohain, D.Agrawal Medians and Beyond: New Aggregation Techniques for Sensor Networks http://www.cs.virginia.edu/~son/cs851/papers/ucsb.sensys04.pdf

This is a slightly modified version. There is a small problem with the compression algorithm in the paper, see https://plus.google.com/u/0/109909935680879695595/posts/768ZZ9Euqz6

So we use a different algorithm here:

• When an item is inserted, we compress along the path to root from the item's leaf
• When the structure becomes too large (above the theoretical bound), or at "too destructive" operations (e.g. union or rebuild) we compress fully

Note that the accuracy of the structure does NOT suffer if "property 2" from the paper is violated (in fact, restoring property 2 at any node decreases accuracy).

So we can say that we preserve the paper's accuracy and memory consumption claims.

Constructor Summary

Constructors

Constructor and Description
QDigest(double compressionFactor)

Method Summary

Methods

Modifier and Type	Method and Description
long	computeActualSize()
static QDigest	deserialize(byte[] b)
long	getQuantile(double q)
void	offer(long value)
static byte[]	serialize(QDigest d)
List<long[]>	toAscRanges()
String	toString()
static QDigest	unionOf(QDigest a, QDigest b)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

QDigest

public QDigest(double compressionFactor)

Method Detail

offer

public void offer(long value)

Specified by:

offer in interface IQuantileEstimator

unionOf

public static QDigest unionOf(QDigest a,
              QDigest b)

getQuantile

public long getQuantile(double q)

Specified by:

getQuantile in interface IQuantileEstimator

toAscRanges

public List<long[]> toAscRanges()

toString

public String toString()

Overrides:

toString in class Object

serialize

public static byte[] serialize(QDigest d)

deserialize

public static QDigest deserialize(byte[] b)

computeActualSize

public long computeActualSize()