org.kitesdk.data
Class PartitionStrategy

java.lang.Object
  org.kitesdk.data.PartitionStrategy

@Immutable
public class PartitionStrategy
extends Object

The strategy used to determine how a dataset is partitioned.

A PartitionStrategy is configured with one or more FieldPartitioners upon creation. When a Dataset is configured with a partition strategy, that data is considered partitioned. Any entities written to a partitioned dataset are evaluated with its PartitionStrategy which, in turn, produces a PartitionKey that is used by the dataset implementation to select the proper partition.

You should use the inner PartitionStrategy.Builder to create new instances.

See Also:

FieldPartitioner, PartitionKey, DatasetDescriptor, Dataset

Nested Class Summary

static class

PartitionStrategy.Builder A fluent builder to aid in the construction of PartitionStrategys.

Method Summary

boolean	equals(Object o)
int	getCardinality() Return the cardinality produced by the contained field partitioners.
List<org.kitesdk.data.spi.FieldPartitioner>	getFieldPartitioners() Get the list of field partitioners used for partitioning.
org.kitesdk.data.spi.FieldPartitioner	getPartitioner(String name) Get a partitioner by partition name.
int	hashCode()
boolean	hasPartitioner(String name) Check if a partitioner for the partition name exists.
PartitionKey	partitionKey(Object... values) Deprecated. will be removed in 0.16.0; use RefinableView methods instead
PartitionKey	partitionKeyForEntity(Object entity) Deprecated. will be removed in 0.16.0; use RefinableView methods instead
PartitionKey	partitionKeyForEntity(Object entity, PartitionKey reuseKey) Deprecated. will be removed in 0.16.0; use RefinableView methods instead
String	toString()
String	toString(boolean pretty)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Method Detail

getFieldPartitioners

public List<org.kitesdk.data.spi.FieldPartitioner> getFieldPartitioners()

Get the list of field partitioners used for partitioning.

FieldPartitioners are returned in the same order they are used during partition selection.

getPartitioner

public org.kitesdk.data.spi.FieldPartitioner getPartitioner(String name)

Get a partitioner by partition name.

Returns:

a FieldPartitioner with the given partition name

Since:

0.15.0

hasPartitioner

public boolean hasPartitioner(String name)

Check if a partitioner for the partition name exists.

Returns:

true if this strategy has a partitioner for the name

Since:

0.15.0

getCardinality

public int getCardinality()

Return the cardinality produced by the contained field partitioners.

This can be used to aid in calculating resource usage during certain operations. For example, when writing data to a partitioned dataset, you can use this method to estimate (or discover exactly, depending on the partition functions) how many leaf partitions exist.

Warning: This method is allowed to lie and should be treated only as a hint. Some partition functions are fixed (for example, hash modulo number of buckets), while others are open-ended (for example, discrete value) and depend on the input data.

Returns:

The estimated (or possibly concrete) number of leaf partitions.

partitionKey

@Deprecated
public PartitionKey partitionKey(Object... values)

Deprecated. will be removed in 0.16.0; use RefinableView methods instead

Construct a partition key with a variadic array of values corresponding to the field partitioners in this partition strategy.

It is permitted to have fewer values than field partitioners, in which case all subpartititions in the unspecified parts of the key are matched by the key.

Null values are not permitted.

partitionKeyForEntity

@Deprecated
public PartitionKey partitionKeyForEntity(Object entity)

Deprecated. will be removed in 0.16.0; use RefinableView methods instead

Construct a partition key for the given entity.

This is a convenient way to find the partition that a given entity is written to, or to find a partition using objects from the entity domain.

partitionKeyForEntity

@Deprecated
public PartitionKey partitionKeyForEntity(Object entity,
                                                     @Nullable
                                                     PartitionKey reuseKey)

Deprecated. will be removed in 0.16.0; use RefinableView methods instead

Construct a partition key for the given entity, reusing the supplied key if not null.

This is a convenient way to find the partition that a given entity is written to, or to find a partition using objects from the entity domain.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object

toString

public String toString(boolean pretty)

Parameters:

pretty - true to indent and format JSON

Returns:

this PartitionStrategy as its JSON representation