org.kitesdk.data.filesystem
Class FileSystemMetadataProvider

java.lang.Object
  org.kitesdk.data.spi.AbstractMetadataProvider
      org.kitesdk.data.filesystem.FileSystemMetadataProvider

All Implemented Interfaces:

MetadataProvider

public class FileSystemMetadataProvider
extends org.kitesdk.data.spi.AbstractMetadataProvider

A MetadataProvider that stores dataset metadata in a Hadoop FileSystem.

When configured with a root directory, this implementation serializes the information within a DatasetDescriptor on the provided FileSystem. The descriptor is serialized as an Avro object and stored in a directory named after the dataset name. For example, if the dataset name is logs, the directory rootDirectory/logs/ will be created, if it doesn't exist, and the serialized descriptor will be stored in the file descriptor.avro.

Nested Class Summary

static class

FileSystemMetadataProvider.Builder A fluent builder to aid in the construction of FileSystemMetadataProvider instances.

Constructor Summary

FileSystemMetadataProvider(Configuration conf, Path rootDirectory)

Method Summary

DatasetDescriptor	create(String name, DatasetDescriptor descriptor) Create a DatasetDescriptor for the dataset named name.
boolean	delete(String name) Delete all metadata associated with the dataset named name.
boolean	exists(String name) Checks if there is a DatasetDescriptor for the dataset named name.
List<String>	list() List the names of the Datasets managed by this MetadataProvider.
DatasetDescriptor	load(String name) Load the dataset descriptor for the dataset name.
String	toString()
DatasetDescriptor	update(String name, DatasetDescriptor descriptor) Update a DatasetDescriptor for the dataset named name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FileSystemMetadataProvider

public FileSystemMetadataProvider(Configuration conf,
                                  Path rootDirectory)

Method Detail

load

public DatasetDescriptor load(String name)

Description copied from interface: MetadataProvider

Load the dataset descriptor for the dataset name.

Parameters:

name - The fully qualified name of an existing dataset.

Returns:

A dataset descriptor.

create

public DatasetDescriptor create(String name,
                                DatasetDescriptor descriptor)

Description copied from interface: MetadataProvider

Create a DatasetDescriptor for the dataset named name. The descriptor will be stored for the named data set in this provider's metadata store. It is illegal to create more than one descriptor for a named data set, and an exception will be thrown.

Parameters:

name - The fully qualified name of a dataset.

descriptor - A dataset descriptor.

Returns:

The descriptor as persisted to the Metadata store.

update

public DatasetDescriptor update(String name,
                                DatasetDescriptor descriptor)

Description copied from interface: MetadataProvider

Update a DatasetDescriptor for the dataset named name. The new descriptor will be stored for the named data set in this provider's metadata store, replacing the previous descriptor. The named dataset must already exist or an error will be thrown. This method is optional.

Parameters:

name - The fully qualified name of a dataset.

descriptor - A dataset descriptor.

Returns:

The descriptor as persisted to the Metadata store.

delete

public boolean delete(String name)

Description copied from interface: MetadataProvider

Delete all metadata associated with the dataset named name.

Parameters:

name - The fully qualified name of a dataset.

Returns:

True if the dataset is successfully deleted, false if the dataset doesn't exist.

exists

public boolean exists(String name)

Description copied from interface: MetadataProvider

Checks if there is a DatasetDescriptor for the dataset named name.

Parameters:

name - a Dataset name to check the existence of

Returns:

true if name exists, false otherwise

list

public List<String> list()

Description copied from interface: MetadataProvider

List the names of the Datasets managed by this MetadataProvider. If there is not at least one Dataset, an empty list will be returned.

Returns:

a Collection of Dataset names (Strings)

toString

public String toString()

Overrides:

toString in class Object