org.kitesdk.data

Class DatasetDescriptor.Builder

java.lang.Object

org.kitesdk.data.DatasetDescriptor.Builder

Enclosing class:

DatasetDescriptor

public static class DatasetDescriptor.Builder
extends Object

A fluent builder to aid in the construction of DatasetDescriptors.

Constructor Summary

Constructors

Constructor and Description
DatasetDescriptor.Builder()
DatasetDescriptor.Builder(DatasetDescriptor descriptor) Creates a Builder configured to copy descriptor, if it is not modified.

Method Summary

Methods

Modifier and Type	Method and Description
DatasetDescriptor	build() Build an instance of the configured dataset descriptor.
DatasetDescriptor.Builder	columnMapping(ColumnMapping columnMappings) Configure the dataset's column mapping descriptor (optional)
DatasetDescriptor.Builder	columnMapping(File file) Configure the dataset's column mapping descriptor from a File.
DatasetDescriptor.Builder	columnMapping(InputStream in) Configure the dataset's column mapping descriptor from an InputStream.
DatasetDescriptor.Builder	columnMappingLiteral(String literal) Configure the dataset's column mappings from a String literal.
DatasetDescriptor.Builder	columnMappingUri(String uri) Configure the dataset's column mappings from a String URI.
DatasetDescriptor.Builder	columnMappingUri(URI uri) Configure the dataset's column mappings from a URI.
DatasetDescriptor.Builder	compressionType(CompressionType compressionType) Configure the dataset's compression format (optional).
DatasetDescriptor.Builder	compressionType(String compressionTypeName) Configure the dataset's compression format (optional).
DatasetDescriptor.Builder	format(Format format) Configure the dataset's format (optional).
DatasetDescriptor.Builder	format(String formatName) Configure the dataset's format from a format name String (optional).
DatasetDescriptor.Builder	location(Path uri) Configure the dataset's location (optional).
DatasetDescriptor.Builder	location(String uri) Configure the dataset's location (optional).
DatasetDescriptor.Builder	location(URI uri) Configure the dataset's location (optional).
DatasetDescriptor.Builder	partitionStrategy(File file) Configure the dataset's partition strategy from a File.
DatasetDescriptor.Builder	partitionStrategy(InputStream in) Configure the dataset's partition strategy from an InputStream.
DatasetDescriptor.Builder	partitionStrategy(PartitionStrategy partitionStrategy) Configure the dataset's partitioning strategy (optional).
DatasetDescriptor.Builder	partitionStrategyLiteral(String literal) Configure the dataset's partition strategy from a String literal.
DatasetDescriptor.Builder	partitionStrategyUri(String uri) Configure the dataset's partition strategy from a String URI.
DatasetDescriptor.Builder	partitionStrategyUri(URI uri) Configure the dataset's partition strategy from a URI.
DatasetDescriptor.Builder	property(String name, String value) Add a key-value property to the descriptor.
<T> DatasetDescriptor.Builder	schema(Class<T> type) Configure the dataset's schema via a Java class type.
DatasetDescriptor.Builder	schema(File file) Configure the dataset's schema from a File.
DatasetDescriptor.Builder	schema(InputStream in) Configure the dataset's schema from an InputStream.
DatasetDescriptor.Builder	schema(Schema schema) Configure the dataset's schema.
DatasetDescriptor.Builder	schemaFromAvroDataFile(File file) Configure the dataset's schema by using the schema from an existing Avro data file.
DatasetDescriptor.Builder	schemaFromAvroDataFile(InputStream in) Configure the dataset's schema by using the schema from an existing Avro data file.
DatasetDescriptor.Builder	schemaFromAvroDataFile(URI uri) Configure the dataset's schema by using the schema from an existing Avro data file.
DatasetDescriptor.Builder	schemaLiteral(String s) Configure the dataset's schema from a String.
DatasetDescriptor.Builder	schemaUri(String uri) Configure the Dataset's schema from a String URI.
DatasetDescriptor.Builder	schemaUri(URI uri) Configure the Dataset's schema from a URI.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DatasetDescriptor.Builder

public DatasetDescriptor.Builder()

DatasetDescriptor.Builder

public DatasetDescriptor.Builder(DatasetDescriptor descriptor)

Creates a Builder configured to copy descriptor, if it is not modified. This is intended to help callers copy and update descriptors even though they are Immutable.

Parameters:

descriptor - A DatasetDescriptor to copy settings from

Since:

0.7.0

Method Detail

schema

public DatasetDescriptor.Builder schema(Schema schema)

Configure the dataset's schema. A schema is required, and can be set using one of the methods: schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

schema

public DatasetDescriptor.Builder schema(File file)
                                 throws IOException

Configure the dataset's schema from a File. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

schema

public DatasetDescriptor.Builder schema(InputStream in)
                                 throws IOException

Configure the dataset's schema from an InputStream. It is the caller's responsibility to close the InputStream. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

schemaUri

public DatasetDescriptor.Builder schemaUri(URI uri)
                                    throws IOException

Configure the Dataset's schema from a URI. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Parameters:

uri - a URI object for the schema's location.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

Since:

0.8.0

schemaUri

public DatasetDescriptor.Builder schemaUri(String uri)
                                    throws IOException

Configure the Dataset's schema from a String URI. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Parameters:

uri - a String URI

Returns:

An instance of the builder for method chaining.

Throws:

IOException

Since:

0.8.0

schemaLiteral

public DatasetDescriptor.Builder schemaLiteral(String s)

Configure the dataset's schema from a String. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Since:

0.8.0

schema

public <T> DatasetDescriptor.Builder schema(Class<T> type)

Configure the dataset's schema via a Java class type. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Since:

0.2.0

schemaFromAvroDataFile

public DatasetDescriptor.Builder schemaFromAvroDataFile(File file)
                                                 throws IOException

Configure the dataset's schema by using the schema from an existing Avro data file. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

schemaFromAvroDataFile

public DatasetDescriptor.Builder schemaFromAvroDataFile(InputStream in)
                                                 throws IOException

Configure the dataset's schema by using the schema from an existing Avro data file. It is the caller's responsibility to close the InputStream. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

schemaFromAvroDataFile

public DatasetDescriptor.Builder schemaFromAvroDataFile(URI uri)
                                                 throws IOException

Configure the dataset's schema by using the schema from an existing Avro data file. A schema is required, and can be set using one of the methods schema, schemaLiteral, schemaUri, or schemaFromAvroDataFile.

Returns:

An instance of the builder for method chaining.

Throws:

IOException

format

public DatasetDescriptor.Builder format(Format format)

Configure the dataset's format (optional). If not specified Formats.AVRO is used by default.

Returns:

An instance of the builder for method chaining.

Since:

0.2.0

format

public DatasetDescriptor.Builder format(String formatName)

Configure the dataset's format from a format name String (optional). If not specified, Formats.AVRO is used by default.

Parameters:

formatName - a String format name

Returns:

An instance of the builder for method chaining.

Throws:

UnknownFormatException - if the format name is not recognized.

Since:

0.8.0

location

public DatasetDescriptor.Builder location(@Nullable
                                 URI uri)

Configure the dataset's location (optional).

Parameters:

uri - A URI location

Returns:

An instance of the builder for method chaining.

Since:

0.8.0

location

public DatasetDescriptor.Builder location(Path uri)

Configure the dataset's location (optional).

Parameters:

uri - A location Path

Returns:

An instance of the builder for method chaining.

Since:

0.8.0

location

public DatasetDescriptor.Builder location(String uri)

Configure the dataset's location (optional).

Parameters:

uri - A location String URI

Returns:

An instance of the builder for method chaining.

Since:

0.8.0

property

public DatasetDescriptor.Builder property(String name,
                                 String value)

Add a key-value property to the descriptor.

Parameters:

name - the property name

value - the property value

Returns:

An instance of the builder for method chaining.

Since:

0.8.0

partitionStrategy

public DatasetDescriptor.Builder partitionStrategy(@Nullable
                                          PartitionStrategy partitionStrategy)

Configure the dataset's partitioning strategy (optional).

Returns:

An instance of the builder for method chaining.

partitionStrategy

public DatasetDescriptor.Builder partitionStrategy(File file)

Configure the dataset's partition strategy from a File. The File contents must be a JSON-formatted partition strategy that is produced by PartitionStrategy.toString().

Parameters:

file - The File

Returns:

An instance of the builder for method chaining.

Throws:

ValidationException - If the file does not contain a valid JSON-encoded partition strategy

DatasetIOException - If there is an IOException accessing the file contents

Since:

0.14.0

partitionStrategy

public DatasetDescriptor.Builder partitionStrategy(InputStream in)

Configure the dataset's partition strategy from an InputStream. The InputStream contents must be a JSON-formatted partition strategy that is produced by PartitionStrategy.toString().

Parameters:

in - The input stream

Returns:

An instance of the builder for method chaining.

Throws:

ValidationException - If the stream does not contain a valid JSON-encoded partition strategy

DatasetIOException - If there is an IOException accessing the InputStream contents

Since:

0.14.0

partitionStrategyLiteral

public DatasetDescriptor.Builder partitionStrategyLiteral(String literal)

Configure the dataset's partition strategy from a String literal. The String literal is a JSON-formatted partition strategy that can be produced by PartitionStrategy.toString().

Parameters:

literal - A partition strategy String literal

Returns:

This builder for method chaining.

Throws:

ValidationException - If the literal is not a valid JSON-encoded partition strategy

Since:

0.14.0

partitionStrategyUri

public DatasetDescriptor.Builder partitionStrategyUri(URI uri)
                                               throws IOException

Configure the dataset's partition strategy from a URI.

Parameters:

uri - A URI to a partition strategy JSON file.

Returns:

This builder for method chaining.

Throws:

ValidationException - If the literal is not a valid JSON-encoded partition strategy

IOException

Since:

0.14.0

partitionStrategyUri

public DatasetDescriptor.Builder partitionStrategyUri(String uri)
                                               throws IOException

Configure the dataset's partition strategy from a String URI.

Parameters:

uri - A String URI to a partition strategy JSON file.

Returns:

This builder for method chaining.

Throws:

ValidationException - If the literal is not a valid JSON-encoded partition strategy

IOException

Since:

0.14.0

columnMapping

public DatasetDescriptor.Builder columnMapping(@Nullable
                                      ColumnMapping columnMappings)

Configure the dataset's column mapping descriptor (optional)

Parameters:

columnMappings - A ColumnMapping

Returns:

This builder for method chaining

Since:

0.14.0

columnMapping

public DatasetDescriptor.Builder columnMapping(File file)

Configure the dataset's column mapping descriptor from a File. The File contents must be a JSON-formatted column mapping. This format can produced by ColumnMapping.toString().

Parameters:

file - The file

Returns:

This builder for method chaining

Throws:

ValidationException - If the literal is not valid JSON-encoded column mappings

DatasetIOException - If there is an IOException accessing the file contents

Since:

0.14.0

columnMapping

public DatasetDescriptor.Builder columnMapping(InputStream in)

Configure the dataset's column mapping descriptor from an InputStream. The InputStream contents must be a JSON-formatted column mapping. This format can produced by ColumnMapping.toString().

Parameters:

in - The input stream

Returns:

This builder for method chaining

Throws:

ValidationException - If the literal is not valid JSON-encoded column mappings

DatasetIOException - If there is an IOException accessing the InputStream contents

Since:

0.14.0

columnMappingLiteral

public DatasetDescriptor.Builder columnMappingLiteral(String literal)

Configure the dataset's column mappings from a String literal. The String literal is a JSON-formatted representation that can be produced by ColumnMapping.toString().

Parameters:

literal - A column mapping String literal

Returns:

This builder for method chaining

Throws:

ValidationException - If the literal is not valid JSON-encoded column mappings

Since:

0.14.0

columnMappingUri

public DatasetDescriptor.Builder columnMappingUri(URI uri)
                                           throws IOException

Configure the dataset's column mappings from a URI.

Parameters:

uri - A URI to a column mapping JSON file

Returns:

This builder for method chaining

Throws:

ValidationException - If the literal is not valid JSON-encoded column mappings

IOException - If accessing the URI results in an IOException

Since:

0.14.0

columnMappingUri

public DatasetDescriptor.Builder columnMappingUri(String uri)
                                           throws IOException

Configure the dataset's column mappings from a String URI.

Parameters:

uri - A String URI to a column mapping JSON file

Returns:

This builder for method chaining

Throws:

ValidationException - If the literal is not valid JSON-encoded column mappings

IOException - If accessing the URI results in an IOException

Since:

0.14.0

compressionType

public DatasetDescriptor.Builder compressionType(CompressionType compressionType)

Configure the dataset's compression format (optional). If not set, default to CompressionType.Snappy.

Parameters:

compressionType - the compression format

Returns:

This builder for method chaining

Since:

0.17.0

compressionType

public DatasetDescriptor.Builder compressionType(String compressionTypeName)

Configure the dataset's compression format (optional). If not set, default to CompressionType.Snappy.

Parameters:

compressionTypeName - the name of the compression format

Returns:

This builder for method chaining

Since:

0.17.0

build

public DatasetDescriptor build()

Build an instance of the configured dataset descriptor. Subsequent calls produce new instances that are similarly configured.

Since:

0.9.0