org.kitesdk.data.mapreduce

Class DatasetKeyInputFormat<E>

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<E,Void>

org.kitesdk.data.mapreduce.DatasetKeyInputFormat<E>

Type Parameters:

E - The type of entities in the Dataset.

All Implemented Interfaces:

Configurable

public class DatasetKeyInputFormat<E>
extends InputFormat<E,Void>
implements Configurable

A MapReduce InputFormat for reading from a Dataset. Since a Dataset only contains entities (not key/value pairs), this output format ignores the value.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DatasetKeyInputFormat.ConfigBuilder

Field Summary

Fields

Modifier and Type	Field and Description
static String	KITE_INPUT_URI
static String	KITE_PARTITION_DIR
static String	KITE_TYPE

Constructor Summary

Constructors

Constructor and Description
DatasetKeyInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
static DatasetKeyInputFormat.ConfigBuilder	configure(Configuration conf) Returns a helper to add input options to the given Configuration.
static DatasetKeyInputFormat.ConfigBuilder	configure(Job job) Configures the Job to use the DatasetKeyInputFormat and returns a helper to add further configuration.
RecordReader<E,Void>	createRecordReader(InputSplit inputSplit, TaskAttemptContext taskAttemptContext)
Configuration	getConf()
List<InputSplit>	getSplits(JobContext jobContext)
void	setConf(Configuration configuration)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KITE_INPUT_URI

public static final String KITE_INPUT_URI

See Also:

Constant Field Values

KITE_PARTITION_DIR

public static final String KITE_PARTITION_DIR

See Also:

Constant Field Values

KITE_TYPE

public static final String KITE_TYPE

See Also:

Constant Field Values

Constructor Detail

DatasetKeyInputFormat

public DatasetKeyInputFormat()

Method Detail

configure

public static DatasetKeyInputFormat.ConfigBuilder configure(Job job)

Configures the Job to use the DatasetKeyInputFormat and returns a helper to add further configuration.

Parameters:

job - the Job to configure

Since:

0.15.0

configure

public static DatasetKeyInputFormat.ConfigBuilder configure(Configuration conf)

Returns a helper to add input options to the given Configuration.

Parameters:

conf - a Configuration

Since:

0.15.0

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

setConf

public void setConf(Configuration configuration)

Specified by:

setConf in interface Configurable

getSplits

public List<InputSplit> getSplits(JobContext jobContext)
                           throws IOException,
                                  InterruptedException

Specified by:

getSplits in class InputFormat<E,Void>

Throws:

IOException

InterruptedException

createRecordReader

public RecordReader<E,Void> createRecordReader(InputSplit inputSplit,
                                      TaskAttemptContext taskAttemptContext)
                                        throws IOException,
                                               InterruptedException

Specified by:

createRecordReader in class InputFormat<E,Void>

Throws:

IOException

InterruptedException