com.clearspring.analytics.stream.cardinality

Class CountThenEstimate

java.lang.Object

com.clearspring.analytics.stream.cardinality.CountThenEstimate

All Implemented Interfaces:

ICardinality, Externalizable, Serializable

public class CountThenEstimate
extends Object
implements ICardinality, Externalizable

Exact -> Estimator cardinality counting

Avoids allocating a large block of memory for cardinality estimation until a specified "tipping point" cardinality is reached.

Currently supports serialization with LinearCounting or AdaptiveCounting

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	CountThenEstimate.CountThenEstimateMergeException

Field Summary

Fields

Modifier and Type	Field and Description
protected static byte	AC
protected IBuilder<ICardinality>	builder Factory for instantiating estimator after the tipping point is reached
protected Set<Object>	counter Cardinality counter Null after tipping point is reached
protected ICardinality	estimator Cardinality estimator Null until tipping point is reached
protected static byte	HLC
protected static byte	HLPC
protected static byte	LC
protected static byte	LLC
protected boolean	tipped True after switching to estimation
protected int	tippingPoint Cardinality after which exact counting gives way to estimation

Constructor Summary

Constructors

Constructor and Description
CountThenEstimate() Default constructor Exact counts up to 1000, estimation done with default Builder
CountThenEstimate(byte[] bytes) Deserialization constructor
CountThenEstimate(int tippingPoint, IBuilder<ICardinality> builder)

Method Summary

Methods

Modifier and Type	Method and Description
long	cardinality()
byte[]	getBytes()
ICardinality	merge(ICardinality... estimators) Merges estimators to produce a new estimator for the combined streams of this estimator and those passed as arguments.
static CountThenEstimate	mergeEstimators(CountThenEstimate... estimators) Merges estimators to produce an estimator for their combined streams
boolean	offer(Object o)
boolean	offerHashed(int hashedInt) Offer the value as a hashed long value
boolean	offerHashed(long hashedLong) Offer the value as a hashed long value
void	readExternal(ObjectInput in)
int	sizeof()
boolean	tipped()
void	writeExternal(ObjectOutput out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LC

protected static final byte LC

See Also:

Constant Field Values

AC

protected static final byte AC

See Also:

Constant Field Values

HLC

protected static final byte HLC

See Also:

Constant Field Values

LLC

protected static final byte LLC

See Also:

Constant Field Values

HLPC

protected static final byte HLPC

See Also:

Constant Field Values

tippingPoint

protected int tippingPoint

Cardinality after which exact counting gives way to estimation

tipped

protected boolean tipped

True after switching to estimation

builder

protected IBuilder<ICardinality> builder

Factory for instantiating estimator after the tipping point is reached

estimator

protected ICardinality estimator

Cardinality estimator Null until tipping point is reached

counter

protected Set<Object> counter

Cardinality counter Null after tipping point is reached

Constructor Detail

CountThenEstimate

public CountThenEstimate()

Default constructor Exact counts up to 1000, estimation done with default Builder

CountThenEstimate

public CountThenEstimate(int tippingPoint,
                 IBuilder<ICardinality> builder)

Parameters:

tippingPoint - Cardinality at which exact counting gives way to estimation

builder - Factory for instantiating estimator after the tipping point is reached

CountThenEstimate

public CountThenEstimate(byte[] bytes)
                  throws IOException,
                         ClassNotFoundException

Deserialization constructor

Parameters:

bytes -

Throws:

IOException

ClassNotFoundException

Method Detail

cardinality

public long cardinality()

Specified by:

cardinality in interface ICardinality

Returns:

the number of unique elements in the stream or an estimate thereof

offerHashed

public boolean offerHashed(long hashedLong)

Description copied from interface: ICardinality

Offer the value as a hashed long value

Specified by:

offerHashed in interface ICardinality

Parameters:

hashedLong - - the hash of the item to offer to the estimator

Returns:

false if the value returned by cardinality() is unaffected by the appearance of hashedLong in the stream

offerHashed

public boolean offerHashed(int hashedInt)

Description copied from interface: ICardinality

Offer the value as a hashed long value

Specified by:

offerHashed in interface ICardinality

Parameters:

hashedInt - - the hash of the item to offer to the estimator

Returns:

false if the value returned by cardinality() is unaffected by the appearance of hashedInt in the stream

offer

public boolean offer(Object o)

Specified by:

offer in interface ICardinality

Parameters:

o - stream element

Returns:

false if the value returned by cardinality() is unaffected by the appearance of o in the stream.

sizeof

public int sizeof()

Specified by:

sizeof in interface ICardinality

Returns:

size in bytes needed for serialization

tipped

public boolean tipped()

getBytes

public byte[] getBytes()
                throws IOException

Specified by:

getBytes in interface ICardinality

Returns:

Throws:

IOException

readExternal

public void readExternal(ObjectInput in)
                  throws IOException,
                         ClassNotFoundException

Specified by:

readExternal in interface Externalizable

Throws:

IOException

ClassNotFoundException

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Specified by:

writeExternal in interface Externalizable

Throws:

IOException

merge

public ICardinality merge(ICardinality... estimators)
                   throws CardinalityMergeException

Description copied from interface: ICardinality

Merges estimators to produce a new estimator for the combined streams of this estimator and those passed as arguments. Nor this estimator nor the one passed as parameters are modified.

Specified by:

merge in interface ICardinality

Parameters:

estimators - Zero or more compatible estimators

Throws:

CardinalityMergeException - If at least one of the estimators is not compatible with this one

mergeEstimators

public static CountThenEstimate mergeEstimators(CountThenEstimate... estimators)
                                         throws CardinalityMergeException

Merges estimators to produce an estimator for their combined streams

Parameters:

estimators -

Returns:

merged estimator or null if no estimators were provided

Throws:

CountThenEstimate.CountThenEstimateMergeException - if estimators are not mergeable (all must be CountThenEstimate made with the same builder)

CardinalityMergeException