Package | Description |
---|---|
org.kitesdk.data |
Provides interfaces and classes for reading and writing datasets.
|
org.kitesdk.data.crunch |
Provides helper classes for working with datasets using Crunch.
|
org.kitesdk.data.mapreduce |
Provides helper classes for working with datasets using MapReduce.
|
org.kitesdk.tools |
Modifier and Type | Interface and Description |
---|---|
interface |
Dataset<E>
A logical representation of a set of data entities.
|
interface |
RandomAccessDataset<E>
A
Dataset that supports random access operations on entities. |
interface |
RefinableView<E>
A
RefinableView specifies a subset of a Dataset by one or
more logical constraints. |
Modifier and Type | Method and Description |
---|---|
static <V extends View<GenericRecord>> |
Datasets.create(String uri,
DatasetDescriptor descriptor)
Create a
Dataset for the given dataset or view URI string. |
static <E,V extends View<E>> |
Datasets.create(String uri,
DatasetDescriptor descriptor,
Class<E> type)
Create a
Dataset for the given dataset or view URI string. |
static <V extends View<GenericRecord>> |
Datasets.create(URI uri,
DatasetDescriptor descriptor)
Create a
Dataset for the given dataset or view URI. |
static <E,V extends View<E>> |
Datasets.create(URI uri,
DatasetDescriptor descriptor,
Class<E> type)
Create a
Dataset for the given dataset or view URI. |
static <V extends View<GenericRecord>> |
Datasets.load(String uriString)
|
static <E,V extends View<E>> |
Datasets.load(String uriString,
Class<E> type)
|
static <V extends View<GenericRecord>> |
Datasets.load(URI uri)
|
static <E,V extends View<E>> |
Datasets.load(URI uri,
Class<E> type)
|
Modifier and Type | Method and Description |
---|---|
static <E> ReadableSource<E> |
CrunchDatasets.asSource(View<E> view)
Expose the given
View as a Crunch ReadableSource . |
static <E> Target |
CrunchDatasets.asTarget(View<E> view)
|
static <E> PCollection<E> |
CrunchDatasets.partition(PCollection<E> collection,
View<E> view)
Partitions
collection to be stored efficiently in View . |
static <E> PCollection<E> |
CrunchDatasets.partition(PCollection<E> collection,
View<E> view,
int numWriters)
Partitions
collection to be stored efficiently in View . |
Modifier and Type | Method and Description |
---|---|
DatasetKeyOutputFormat.ConfigBuilder |
DatasetKeyOutputFormat.ConfigBuilder.appendTo(View<?> view)
Adds configuration for
DatasetKeyOutputFormat to append to the
given dataset or view URI, leaving any existing data intact. |
DatasetKeyOutputFormat.ConfigBuilder |
DatasetKeyOutputFormat.ConfigBuilder.overwrite(View<?> view)
|
DatasetKeyInputFormat.ConfigBuilder |
DatasetKeyInputFormat.ConfigBuilder.readFrom(View<?> view)
|
DatasetKeyOutputFormat.ConfigBuilder |
DatasetKeyOutputFormat.ConfigBuilder.writeTo(View<?> view)
|
Constructor and Description |
---|
CopyTask(View<E> from,
View<E> to) |
CopyTask(View<E> from,
View<E> to) |
TransformTask(View<S> from,
View<T> to,
DoFn<S,T> transform) |
TransformTask(View<S> from,
View<T> to,
DoFn<S,T> transform) |
Copyright © 2013–2014. All rights reserved.