| Modifier and Type | Method and Description |
|---|---|
Class<CommandBuilder> |
MorphlineContext.getCommandBuilder(String builderName) |
| Constructor and Description |
|---|
MorphlineCompilationException(String msg,
com.typesafe.config.Config config,
Throwable th,
CommandBuilder builder) |
| Modifier and Type | Class and Description |
|---|---|
class |
ExtractAvroPathsBuilder
Command that uses zero or more avro path expressions to extract values from an Avro object.
|
class |
ExtractAvroTreeBuilder
Command that converts an attached Avro datum to a morphline record by recursively walking the
Avro tree and extracting all data into a single morphline record, with fields named by their path
in the Avro tree.
|
class |
ReadAvroBuilder
Command that parses an InputStream that contains Avro data; for each Avro datum, the command
emits a morphline record containing the datum as an attachment in
Fields.ATTACHMENT_BODY. |
class |
ReadAvroContainerBuilder
Command that parses an InputStream that contains Avro binary container file data; for each Avro
datum, the command emits a morphline record containing the datum as an attachment in
Fields.ATTACHMENT_BODY. |
class |
ToAvroBuilder
Command that converts a morphline record to an Avro record.
|
class |
ToAvroMapBuilder
Command that converts a morphline record to an Avro record that contains a Map with string keys
and array values where the array values can be null, boolean, int, long, float, double, string,
bytes.
|
class |
WriteAvroToByteArrayBuilder
Command that serializes the Avro records contained in the _attachment_body field into a byte
array and replaces the _attachment_body field with that byte array.
|
| Constructor and Description |
|---|
AbstractCommand(CommandBuilder builder,
com.typesafe.config.Config config,
Command parent,
Command child,
MorphlineContext context)
Using the given
builder, constructs a command rooted at the given morphline JSON
config. |
| Modifier and Type | Class and Description |
|---|---|
class |
DownloadHdfsFileBuilder
Command for transferring HDFS files, for example to help with centralized configuration file
management.
|
class |
OpenHdfsFileBuilder
Opens an HDFS file for read and return a corresponding InputStream.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ReadAvroParquetFileBuilder
Command that parses a Hadoop File that contains Parquet data; for each Parquet Avro
datum, the command emits a morphline record containing the record as an attachment in
Fields.ATTACHMENT_BODY. |
| Modifier and Type | Class and Description |
|---|---|
class |
ReadRCFileBuilder
Command for reading Record Columnar (RC) Files.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ReadSequenceFileBuilder
Command that emits one record per sequence file entry in the input stream of the first attachment.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ExtractJsonPathsBuilder
Command that uses zero or more JSON path expressions to extract values from a JSON object.
|
class |
ReadJsonBuilder
Command that parses an InputStream that contains JSON data; for each JSON object in the stream,
the command emits a morphline record containing the object as an attachment in
Fields.ATTACHMENT_BODY. |
| Modifier and Type | Class and Description |
|---|---|
class |
GeoIPBuilder
Command that returns Geolocation information for a given IP address, using an efficient in-memory
Maxmind database lookup.
|
| Modifier and Type | Class and Description |
|---|---|
class |
RegisterJVMMetricsBuilder
Command that registers metrics that are related to the Java Virtual Machine with the
MorphlineContext of the morphline.
|
class |
StartReportingMetricsToHTTPBuilder
Command that exposes liveness status, health check status, metrics state and thread dumps via a
set of HTTP URLs served by Jetty, using the AdminServlet.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ExtractProtobufPathsBuilder
Command that uses zero or more protocol buffer path expressions to extract
values from a protocol buffer object.
|
class |
ReadProtobufBuilder
Command that parses an InputStream that contains Protocol buffer data; the
command emits a morphline record containing the object as an attachment in
Fields.ATTACHMENT_BODY. |
| Modifier and Type | Class and Description |
|---|---|
class |
ConvertHTMLBuilder
Command that converts HTML to XHTML using the TagSoup library.
|
class |
XQueryBuilder
Command that parses an InputStream that contains an XML document and runs the given XQuery over
the XML document.
|
class |
XSLTBuilder
Command that parses an InputStream that contains an XML document and runs the given XSL Transform
over the XML document.
|
| Modifier and Type | Class and Description |
|---|---|
class |
GenerateSolrSequenceKeyBuilder
A command that assigns a record unique key that is the concatenation of the given
baseIdField record field, followed by a running count of the record number within
the current session. |
class |
LoadSolrBuilder
A command that loads (or deletes) a record into a SolrServer or MapReduce SolrOutputFormat.
|
class |
SanitizeUnknownSolrFieldsBuilder
Command that sanitizes record fields that are unknown to Solr schema.xml by either deleting them
(renameToPrefix is absent or a zero length string), or by moving them to a field prefixed with
the given renameToPrefix (e.g.
|
class |
TokenizeTextBuilder
A command that uses the embedded Solr/Lucene Analyzer library to generate tokens from a text
string, without sending data to a Solr server.
|
| Modifier and Type | Class and Description |
|---|---|
class |
SolrCellBuilder
Command that pipes the first attachment of a record into one of the given Tika parsers, then maps
the Tika output back to a record using SolrCell.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ReadBlobBuilder
Command that emits one record for the entire input stream of the first attachment, interpreting
the stream as a Binary Large Object (BLOB), i.e.
|
class |
ReadClobBuilder
Command that emits one record for the entire input stream of the first attachment, interpreting
the stream as a Character Large Object (CLOB).
|
class |
ReadCSVBuilder
Command that extracts zero or more records from the input stream of the first attachment,
representing a Comma Separated Values (CSV) file.
|
class |
ReadLineBuilder
Command that emits one record per line in the input stream of the first attachment.
|
class |
ReadMultiLineBuilder
Multiline log parser that collapse multiline messages into a single record; supports "regex",
"what" and "negate" configuration parameters similar to logstash.
|
| Constructor and Description |
|---|
AbstractParser(CommandBuilder builder,
com.typesafe.config.Config config,
Command parent,
Command child,
MorphlineContext context) |
| Modifier and Type | Class and Description |
|---|---|
class |
AddCurrentTimeBuilder
A command that adds the result of
System.currentTimeMillis() to a given output field. |
class |
AddLocalHostBuilder
A command that adds the name or IP of the local host to a given output field.
|
class |
AddValuesBuilder
For each input field value, add the value to the given record output field.
|
class |
AddValuesIfAbsentBuilder
For each input field value, add the value to the given record output field if the value isn't
already contained in that field.
|
class |
CallParentPipeBuilder
Command that routes records to the enclosing pipe morphline object.
|
class |
ContainsBuilder
Command that succeeds if one of the field values of the given named field is equal to one of the
the given values, and fails otherwise; Multiple fields can be named, in which case the results
are ANDed.
|
class |
ConvertTimestampBuilder
Command that converts the timestamps in a given field from one of a set of input date formats (in
an input timezone) to an output date format (in an output timezone), while respecting daylight
savings time rules.
|
class |
DecodeBase64Builder
Command that converts a Base64 encoded String to a byte[] per Section 6.8.
|
class |
DropRecordBuilder
Command that silently consumes records without ever emitting any record - think /dev/null.
|
class |
EqualsBuilder
Command that succeeds if all field values of the given named fields are equal to the the given
values, and fails otherwise.
|
class |
ExtractURIComponentBuilder
A command that extracts a subcomponent from the URIs contained in the given input field and
appends it to the given output fields.
|
class |
ExtractURIComponentsBuilder
A command that extracts subcomponents from the URIs contained in the given input field and
appends them to output fields with the given prefix, namely scheme, authority, host, port, path,
query, fragment, schemeSpecificPart, userInfo.
|
class |
ExtractURIQueryParametersBuilder
A command that extracts the query parameters with a given name from the URIs contained in the
given input field and appends them to the given output field.
|
class |
FindReplaceBuilder
Command that examines each string value in a given field and replaces each substring of the string value
that matches the given string literal or grok pattern with the given replacement.
|
class |
GenerateUUIDBuilder
A command that sets a universally unique identifier on all records that are intercepted.
|
class |
GrokBuilder
The Grok command uses regular expression pattern matching to extract structured fields from
unstructured log data.
|
class |
HeadBuilder
Command that ignores all input records beyond the N-th record, thus emitting at most N records,
akin to the Unix
head command. |
class |
IfThenElseBuilder
An If-Then-Else command consists of a chain of zero or more conditions commands, as well as a
chain of zero or or more commands that are processed if all conditions succeed ("then commands"),
as well as a chain of zero or more commands that are processed if one of the conditions fails
("else commands").
|
class |
JavaBuilder
The "java" command compiles the given Java code block, wrapped into a Java method with a boolean
return type and several parameters, along with a Java class definition that contains the given
import statements.
|
class |
LogDebugBuilder
Command that logs to slf4j at DEBUG level.
|
class |
LogErrorBuilder
Command that logs to slf4j at ERROR level.
|
class |
LogInfoBuilder
Command that logs to slf4j at INFO level.
|
class |
LogTraceBuilder
Command that logs to slf4j at TRACE level.
|
class |
LogWarnBuilder
Command that logs to slf4j at WARN level.
|
class |
NotBuilder
A Not command consists of one nested command, the boolean return value of which is inverted.
|
class |
PipeBuilder
Factory to create morphline pipe instances.
|
class |
RemoveFieldsBuilder
Removes all record fields for which the field name matches at least one of the given blacklist
predicates but none of the given whitelist predicates.
|
class |
RemoveValuesBuilder
Removes all record field values for which all of the following conditions hold:
1) the field name matches at least one of the given nameBlacklist predicates but none of the
given nameWhitelist predicates.
|
class |
ReplaceValuesBuilder
Replaces all record field values for which all of the following conditions hold:
1) the field name matches at least one of the given nameBlacklist predicates but none of the
given nameWhitelist predicates.
|
class |
SampleBuilder
Command that forwards each input record with a given probability to its child command, and
silently ignores all other input records.
|
class |
SeparateAttachmentsBuilder
Command that emits one output record for each attachment in the input record's list of
attachments; The result is a list of records, each of which has at most one attachment.
|
class |
SetValuesBuilder
For each input field, set the given record output field to the same values.
|
class |
SplitBuilder
Command that divides strings into substrings, by recognizing a separator (a.k.a.
|
class |
SplitKeyValueBuilder
Command that iterates over the items in a given record input field, interprets each item as a key-value
pair where the key and value are separated by the given separator character, and adds the pair's
value to the record field named after the pair's key.
|
class |
StartReportingMetricsToCSVBuilder
Command that starts periodically appending the metrics of the
com.codahale.metrics.MetricRegistry of the
morphline context to a set of CSV files, configured via a
com.codahale.metrics.CsvReporter.Builder. |
class |
StartReportingMetricsToJMXBuilder
Command that starts publishing the metrics of the
com.codahale.metrics.MetricRegistry of the morphline context
to JMX, configured via a com.codahale.metrics.JmxReporter.Builder. |
class |
StartReportingMetricsToSLF4JBuilder
Command that starts periodically logging the metrics of the
com.codahale.metrics.MetricRegistry of the
morphline context to SLF4j, configured via a
com.codahale.metrics.Slf4jReporter.Builder. |
class |
ToByteArrayBuilder
Command that converts the Java objects in a given field via
Object.toString() to
their string representation, and then via String.getBytes(Charset) to their byte
array representation. |
class |
ToStringBuilder
Command that converts the Java objects in a given field via
Object.toString() to
their string representation, and optionally also applies String.trim(). |
class |
TranslateBuilder
Command that examines each string value in a given field and replaces it with the replacement
value defined in a given dictionary aka hash table.
|
class |
TryRulesBuilder
A tryRules command consists of zero or more rules.
|
| Constructor and Description |
|---|
AbstractFieldTransformCommand(CommandBuilder builder,
com.typesafe.config.Config config,
Command parent,
Command child,
MorphlineContext context) |
| Modifier and Type | Class and Description |
|---|---|
class |
DetectMimeTypeBuilder
Command that auto-detects the MIME type of the first attachment, if no MIME type is defined yet.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DecompressBuilder
Command that decompresses the first attachment.
|
class |
UnpackBuilder
Command that unpacks the first attachment.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ReadJsonTestTweetsBuilder
JSON parser that extracts search documents from twitter tweets obtained from the twitter 1% sample firehose with the delimited=length option.
|
| Modifier and Type | Class and Description |
|---|---|
class |
UserAgentBuilder
Command that parses user agent strings and returns structured higher level data like user agent
family, operating system, version, and device type, using the underlying API and regexes.yaml
BrowserScope database from https://github.com/tobie/ua-parser.
|
Copyright © 2013–2015. All rights reserved.