org.kitesdk.tools

Class TransformTask<S,T>

java.lang.Object

org.apache.hadoop.conf.Configured

org.kitesdk.tools.TransformTask<S,T>

All Implemented Interfaces:

Configurable

Direct Known Subclasses:

CopyTask

public class TransformTask<S,T>
extends Configured

Since:

0.16.0

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TransformTask.CheckEntityClass<E>

Constructor Summary

Constructors

Constructor and Description
TransformTask(View<S> from, View<T> to, DoFn<S,T> transform)

Method Summary

Methods

Modifier and Type	Method and Description
long	getCount()
TransformTask	noCompaction() Do not shuffle data to writers to minimize the number of files.
PipelineResult	run()
TransformTask	setFilesPerPartition(int filesPerPartition) Set the number of files to create in each output partition.
TransformTask	setNumWriters(int numWriters) Set the number of concurrent writer processes.
TransformTask	setWriteMode(Target.WriteMode mode) Set the output write mode: default, overwrite, or append.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TransformTask

public TransformTask(View<S> from,
             View<T> to,
             DoFn<S,T> transform)

Method Detail

getCount

public long getCount()

noCompaction

public TransformTask noCompaction()

Do not shuffle data to writers to minimize the number of files. Write from each read task.

Returns:

this for method chaining

setNumWriters

public TransformTask setNumWriters(int numWriters)

Set the number of concurrent writer processes.

If set to 0, this disables compaction.

Parameters:

numWriters - the number of concurrent writer processes

Returns:

this for method chaining

setFilesPerPartition

public TransformTask setFilesPerPartition(int filesPerPartition)

Set the number of files to create in each output partition.

Parameters:

filesPerPartition - a target number of files per output partition

Returns:

this for method chaining

Since:

1.1.0

setWriteMode

public TransformTask setWriteMode(Target.WriteMode mode)

Set the output write mode: default, overwrite, or append.

Parameters:

mode - the output write mode

Returns:

this for method chaining

run

public PipelineResult run()
                   throws IOException

Throws:

IOException