A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _

A

AbstractSource - Class in gratum.source

AbstractSource(String) - Constructor in AbstractSource

add(Source) - Method in ConcatSource

add(long) - Method in Duration

addAll(Collection<Source>) - Method in ConcatSource

addDefaultRejections() - Method in Pipeline

addField(String, Closure) - Method in Pipeline

Adds a new field to each row with the value returned by the given closure.

addRejection(RejectionCategory, String, Integer) - Method in LoadStatistic

addStep(GString, Closure<Map<String, Object>>) - Method in Pipeline

addTiming(String, long) - Method in LoadStatistic

addTo(long) - Method in Duration

addToCurrentRow(Object, int, int) - Method in XlsProcessor

after(Closure<Void>) - Method in Pipeline

Adds a closure to the end of the Pipeline.

AfterStep - Class in gratum.etl

AfterStep(Closure<Void>) - Constructor in AfterStep

apply(Closure<Pipeline>) - Method in Pipeline

Passed a closure that is called with this Pipeline.

ArchivedSource - Class in gratum.source

ArchivedSource supports other archive formats that the JDK doesn't support.

ArchivedSource(File) - Constructor in ArchivedSource

asBoolean(String) - Method in Pipeline

Parses the string value at given column into a java.lang.Boolean value.

asDate(String, String) - Method in Pipeline

Parses the string at the given column name into a Date object using the given format.

asDouble(String) - Method in Pipeline

Return a Pipeline where the given column is converted from a string to a java.lang.Double.

asInt(String) - Method in Pipeline

Parses the string value at given column into a java.lang.Integer value.

asType(Class) - Method in FileOpenable

asType(Class) - Method in Openable

attach(Pipeline) - Method in CsvSink

attach(Pipeline) - Method in JsonSink

attach(Pipeline) - Method in Sink

attach(Pipeline) - Method in XlsxSink

authBasic(String, String) - Method in OkHttpSource.OkHttpBuilder

Helper method to add Basic authentication header to the request.

authPass(String, String) - Method in SshSource

Authenticate with a username and password.

authToken(String) - Method in OkHttpSource.OkHttpBuilder

Helper method to add Bearer authentication header to the request.

avg(String) - Method in LoadStatistic

B

branch(Map<String, ?>, Closure<Pipeline>) - Method in Pipeline

Copies all rows on this Pipeline to another Pipeline where the given condition returns is true.

build() - Method in OkHttpSource.OkHttpBuilder

C

cell(String, String, XSSFComment) - Method in XlsxSource.XslxSheetHandler

ChainedSource - Class in gratum.source

ChainedSource(Pipeline) - Constructor in ChainedSource

client - Field in OkHttpSource.OkHttpBuilder

clip(String) - Method in Pipeline

Remove all columns from each row so that only the fields given will be returned.

close() - Method in CsvSink

close() - Method in FileOpenable

close() - Method in JsonSink

close() - Method in XlsxSink

ClosureSource - Class in gratum.source

ClosureSource(Closure) - Constructor in ClosureSource

collect(Closure<Pipeline>) - Method in ConcurrentContext

collect(Closure<Pipeline>) - Method in LocalConcurrentContext

CollectionSource - Class in gratum.source

A source that uses a Collection<Map> as its source for rows.

CollectionSource(Iterable<Map>) - Constructor in CollectionSource

concat(Source) - Method in ConcatSource

concat(Pipeline) - Method in Pipeline

Concatenates the rows from this pipeline and the given pipeline.

ConcatSource - Class in gratum.source

ConcurrentContext - Interface in gratum.concurrency

Condition - Class in gratum.etl

Condition(Map<String, Object>) - Constructor in Condition

connect() - Method in ConcurrentContext

connect() - Method in LocalConcurrentContext

create(String, Closure) - Method in Pipeline

Creates a pipeline where startClosure is the source.

createClosureCallback(String, Closure) - Method in Condition

createEqualsCallback(String, Object) - Method in Condition

createHttpConfigClosure() - Method in HttpSource

createPatternCallback(String, Pattern) - Method in Condition

createResultProcessor() - Method in LocalConcurrentContext

csv(String, InputStream, String, List<String>) - Method in CsvSource

CsvSink - Class in gratum.sink

CsvSink(String, String, List<String>) - Constructor in CsvSink

CsvSource - Class in gratum.source

Reads a delimited separated text file into a series of rows.

CsvSource(Reader, String, List<String>) - Constructor in CsvSource

CustomDateDataFormatter - Class in gratum.source

CustomDateDataFormatter(String) - Constructor in CustomDateDataFormatter

D

database(String, String, String) - Method in JdbcSource

dateFormat(String) - Method in XlsxSource

Sets the default format for dates (default format is yyyy-MM-dd).

decryptPgp(String, Closure) - Method in Pipeline

Decrypts using PGP a stream on the Pipeline and rewrites the stream back onto the Pipeline.

defaultsBy(Map<String, String>) - Method in Pipeline

Sets the destination column to the value of the source column if it is empty or null.

defaultValues(Map<String, Object>) - Method in Pipeline

Sets the destination column to the value within the given Map when the destination column is empty/null.

delete(Closure) - Method in HttpSource

DO_NOT_TRACK - Field in Pipeline

doRejections(Map<String, Object>, String, int) - Method in Pipeline

doStart(Pipeline) - Method in AbstractSource

doStart(Pipeline) - Method in ArchivedSource

doStart(Pipeline) - Method in ChainedSource

doStart(Pipeline) - Method in ClosureSource

doStart(Pipeline) - Method in CollectionSource

doStart(Pipeline) - Method in ConcatSource

doStart(Pipeline) - Method in CsvSource

doStart(Pipeline) - Method in FileSystemSource

doStart(Pipeline) - Method in HttpSource

doStart(Pipeline) - Method in JdbcSource

doStart(Pipeline) - Method in JsonSource

doStart(Pipeline) - Method in OkHttpSource

doStart(Pipeline) - Method in SshSource

doStart(Pipeline) - Method in XlsSource

doStart(Pipeline) - Method in XlsxSource

doStart(Pipeline) - Method in ZipSource

download(String) - Method in SshSource

One or more remote paths you wish to download from the SFTP server.

downloadPath(Pipeline, ChannelSftp, String) - Method in SshSource

Duration - Class in gratum.etl

Created by charliehubbard on 7/11/18.

Duration(long) - Constructor in Duration

E

enableStrictHostCheck(boolean) - Method in SshSource

Can be used to disable the known host checking, but this has serious security implications which need to be carefully considered before doing this.

encryptPgp(String, Closure) - Method in Pipeline

Encrypts using PGP a stream on the pipeline and rewrite that stream back to the pipeline.

endRow(int) - Method in XlsxSource.XslxSheetHandler

escaping(boolean) - Method in CsvSource

Turn on/off CSV escaping rules.

exchange(Closure<Pipeline>) - Method in Pipeline

This takes a closure that returns a Pipeline which is used to feed the Pipeline returned by exchange().

execute() - Method in AfterStep

execute(Pipeline, Map<String, Object>, int) - Method in Step

F

FileOpenable - Class in gratum.etl

Represents a file that can be cast to an InputStream or OutputStream giving it the ability to masquerade as an InputStream/OutputStream.

FileOpenable(File) - Constructor in FileOpenable

files(Iterable<File>) - Method in FileSystemSource

Pass an Iterable of files to this method and this source will visit each file.

FileSystemSource - Class in gratum.source

Creates a source that enumerates files within a directory, the file itself, and returns a Pipeline that produces a Map object with keys file and stream with types java.io.File and java.io.InputStream respectively.

FileSystemSource(Iterable<File>) - Constructor in FileSystemSource

fillDownBy(Closure<Boolean>) - Method in Pipeline

This returns a Pipeline where the rows with empty columns are filled in using the values in the previous row depending on what the given closure returns.

filter(Pattern) - Method in FileSystemSource

Add a filename filter using a regular expression to match filenames against.

filter(Map) - Method in Pipeline

This adds a step to the Pipeline that passes all rows where the values of the columns on the given Map are equal to the columns in the row.

finished() - Method in Pipeline

flattenWindow(String, Closure<List<Map<String, Object>>>) - Method in Pipeline

Collects or flattens a set of rows that share the same value in succession within the stream into a List of those matching rows.

format(String) - Method in ArchivedSource

formatRawCellContents(double, int, String, boolean) - Method in CustomDateDataFormatter

from(Iterable<Map>) - Method in CollectionSource

G

get(Closure) - Method in HttpSource

getDuration() - Method in LoadStatistic

getDuration() - Method in Step

getElapsed() - Method in LoadStatistic

getEnd() - Method in LoadStatistic

getHours() - Method in Duration

getLoaded() - Method in LoadStatistic

getLoaded() - Method in Step

getName() - Method in CsvSink

getName() - Method in JsonSink

getName() - Method in Pipeline

The name of the pipeline.

getName() - Method in Sink

getName() - Method in XlsxSink

getRejections(RejectionCategory, String) - Method in LoadStatistic

getRejections() - Method in Step

getRejectionsByCategory() - Method in LoadStatistic

getRejectionsFor(RejectionCategory) - Method in LoadStatistic

getResult() - Method in CsvSink

getResult() - Method in JsonSink

getResult() - Method in Sink

getResult() - Method in XlsxSink

getStart() - Method in LoadStatistic

go() - Method in Pipeline

Starts processing the rows returned from the Source into the Pipeline.

groupBy(String) - Method in Pipeline

Return a Pipeline where the row is grouped by the given columns.

H

head(Closure) - Method in HttpSource

header(Closure<Void>) - Method in CsvSource

Register a closure to receive the header before processing starts

header(Closure<Void>) - Method in XlsxSource

Attaches a closure that will be called back when the headers are available.

http(String, HttpBuilder) - Method in HttpSource

http(String, OkHttpClient, Closure) - Method in OkHttpSource

Best practice is to share OkHttpClients when performing multiple http calls or multiple OkHttpSources.

https(String, HttpBuilder) - Method in HttpSource

https(String, OkHttpClient, Closure) - Method in OkHttpSource

Best practice is to share OkHttpClients when performing multiple http calls or multiple OkHttpSources.

HttpSource - Class in gratum.source

@deprecated This class will be removed in the future in favor of {@see gratum.source.OkHttpSource}

HttpSource.HttpVerb - Enum in gratum.source

HttpSource(HttpBuilder) - Constructor in HttpSource

I

identity(String, File, String) - Method in SshSource

This uses a SSH key pair to authenticate the given username.

includeRoot(boolean, String) - Method in JsonSource

incrementRejections(RejectionCategory) - Method in Step

inject(Closure) - Method in Pipeline

Delegates to inject(java.lang.String, groovy.lang.Closure) with a default name.

intersect(Pipeline, def) - Method in Pipeline

Return a Pipeline where all of the rows from this Pipeline and adds a single column "included" with a true/false value depending on whether the current row is occurs in the given Pipeline and the values of the specified columns are equal in both Pipelines.

into() - Method in AbstractSource

This converts the source into a Pipeline to attach steps to.

into() - Method in Source

J

JdbcSource - Class in gratum.source

A source that uses a database query for the source of the rows it feeds through the pipeline.

JdbcSource(String, String, String) - Constructor in JdbcSource

join(Pipeline, def, boolean) - Method in Pipeline

Returns Pipeline that joins the columns from this Pipeline with the given Pipeline where the columns are equal.

json(String, String) - Method in JsonSource

json(String, List<String>) - Method in Pipeline

Write out the rows produced to JSON file.

jsonl(File) - Method in JsonSource

jsonl(File, List<String>) - Method in Pipeline

Write out the rows produced into a given File as JSON Lines format.

jsonObjectPerLine(boolean) - Method in JsonSink

JsonSink - Class in gratum.sink

JsonSink(String, Writer, Collection<String>) - Constructor in JsonSink

JsonSource - Class in gratum.source

JsonSource(String, Reader) - Constructor in JsonSource

K

knownHosts(File) - Method in SshSource

A file containing the list of known hosts the client will use to authorize the connection to the server.

L

limit(long, boolean) - Method in Pipeline

Limit the number of rows you take from a source to an upper bound.

LoadStatistic - Class in gratum.etl

This object contains the statistics on how many items were processed by the Pipeline.

LocalConcurrentContext - Class in gratum.concurrency

LocalConcurrentContext(int, int) - Constructor in LocalConcurrentContext

logger - Field in OkHttpSource

logger - Field in Pipeline

M

mark() - Method in LoadStatistic

matches(Map) - Method in Condition

MAX_ERROR_THRESHOLD - Field in Step

MAX_RETRIES - Field in OkHttpSource

merge(LoadStatistic, boolean) - Method in LoadStatistic

mergeRejections(LoadStatistic) - Method in LoadStatistic

mergeTimings(LoadStatistic) - Method in LoadStatistic

N

name(String) - Method in AbstractSource

Overrides the name of the {@see gratum.etl.Pipeline}.

name - Field in Step

O

of(Closure<Void>) - Method in ClosureSource

of(Iterable<Map>) - Method in CollectionSource

of(String, InputStream, String, List<String>) - Method in CsvSource

OkHttpSource - Class in gratum.source

OkHttpSource.OkHttpBuilder - Class in gratum.source

A wrapper class around OKHttp's Request.Builder for building the request being sent in an Http call.

OkHttpSource.OkHttpBuilder(OkHttpClient, Builder, HttpUrl) - Constructor in OkHttpSource.OkHttpBuilder

OkHttpSource(String, OkHttpClient, Closure) - Constructor in OkHttpSource

The constructor of OkHttpSource used internally.

onRejection(Closure<Void>) - Method in Pipeline

Takes a closure that is passed the rejection Pipeline.

Openable - Interface in gratum.etl

outputFormulaValues(boolean) - Method in XlsSource

Sets the values to output formula values vs the formula itself.

P

parse(POIFSFileSystem) - Method in XlsProcessor

parseRecordPerLine(Reader, Pipeline) - Method in JsonSource

password(String) - Method in XlsSource

Sets the password if the xls file is protected.

password(String) - Method in XlsxSource

The password to use for encrypted spreadsheets

path(List<String>) - Method in JsonSource

Pipeline - Class in gratum.etl

A Pipeline represents a series of steps that will be performed on 1 or more rows.

Pipeline(String, Pipeline) - Constructor in Pipeline

PipelineWorker - Class in gratum.concurrency

PipelineWorker(String, Closure<LoadStatistic>) - Constructor in PipelineWorker

post(Closure) - Method in HttpSource

prependStep(String, Closure<Map<String, Object>>) - Method in Pipeline

Prepend a step to the pipeline.

printRow(String) - Method in Pipeline

Prints the values of the given columns for each row to the console, or all columns if no columns are given.

process(Collection<Map>) - Method in ChainedSource

process(File, Pipeline) - Method in FileSystemSource

process(Map, int) - Method in Pipeline

This method is used to send rows to the Pipeline for processing.

processRecord(Record) - Method in XlsProcessor

progress(int) - Method in Pipeline

put(Closure) - Method in HttpSource

Q

query(GString) - Method in JdbcSource

query(Map<String, Object>) - Method in OkHttpSource.OkHttpBuilder

Helper method to add query parameters to the existing url.

R

recordPerLine(boolean) - Method in JsonSource

recursive(boolean) - Method in FileSystemSource

Configures this source to be visit directories recursively or not.

reduce(String, Map<String, Object>, Closure<Map<String, Object>>) - Method in Pipeline

Reduces all upstream rows into a value that is passed into the given closure (similar to Groovy inject method).

reject(Rejection) - Method in LoadStatistic

reject(Map<String, Object>, int) - Method in Pipeline

reject(Map<String, Object>, String, RejectionCategory) - Method in Step

REJECTED_KEY - Field in Pipeline

Rejection - Class in gratum.etl

Created by charliehubbard on 7/11/18.

Rejection(String, RejectionCategory, String, Throwable) - Constructor in Rejection

RejectionCategory - Enum in gratum.etl

Created by charliehubbard on 7/11/18.

rejectNon200(boolean) - Method in OkHttpSource

removeField(String, Closure) - Method in Pipeline

Removes a field based on whether the given closure returns true or false.

renameFields(Map) - Method in Pipeline

Rename a row's columns in the given map to the value of the corresponding key.

replaceAll(String, Pattern, String) - Method in Pipeline

Replaces all occurrences of the regular expression with given withClause.

replaceValues(String, Map<String, String>) - Method in Pipeline

Given a column replace all values at that column that match the given key with the value of the values Map.

run() - Method in PipelineWorker

S

save(Sink<Map<String, Object>>) - Method in Pipeline

Write the rows to a provided Sink.

setDateFormat(String) - Method in XlsxSource

setEnd(Long) - Method in LoadStatistic

setEscaped(boolean) - Method in CsvSource

setField(String, Object) - Method in Pipeline

Sets a fieldName in each row to the given value.

setHeaders(List<String>) - Method in CsvSource

setName(String) - Method in AbstractSource

setPassword(String) - Method in XlsxSource

setSeparator(String) - Method in CsvSource

setSheet(String) - Method in XlsxSource

setStart(Long) - Method in LoadStatistic

sevenZ(File) - Method in ArchivedSource

Sink - Interface in gratum.sink

sort(String) - Method in Pipeline

Return a Pipeline where the rows are ordered by the given columns.

SortOrder - Enum in gratum.etl

Source - Interface in gratum.source

Created by charliehubbard on 7/11/18.

source(Source) - Method in Pipeline

Assigned a new source to a Pipeline and returns the pipeline.

spread(Closure<Pipeline>) - Method in ConcurrentContext

spread(Closure<Pipeline>) - Method in LocalConcurrentContext

ssh(String, int) - Method in SshSource

Returns a SshSource for file transfer over sftp connecting the given remote host over the given TCP port.

SshSource - Class in gratum.source

SshSource allows you to connect to a remote SSH server and download one or more paths.

SshSource(String, int) - Constructor in SshSource

start(Pipeline) - Method in AbstractSource

start() - Method in Pipeline

Start processing rows from the source of the pipeline.

start() - Method in PipelineWorker

start(Pipeline) - Method in Source

startOnRow(int) - Method in XlsSource

Sets the row to start reading the table data.

startRow(int) - Method in XlsxSource.XslxSheetHandler

Step - Class in gratum.etl

step - Field in Step

sub(long) - Method in Duration

subFrom(long) - Method in Duration

T

timed(String, Closure<R>) - Method in LoadStatistic

toLoadStatistic(long, long) - Method in Pipeline

toMillis() - Method in Duration

toString() - Method in Condition

toString() - Method in Duration

toString(boolean) - Method in LoadStatistic

trim() - Method in Pipeline

Returns a Pipeline where all white space is removed from all columns contained within the rows.

U

unarchive(File) - Method in ArchivedSource

unique(String) - Method in Pipeline

Only allows rows that are unique per the given column.

unzip(File) - Method in ArchivedSource

unzip(File) - Method in ZipSource

url - Field in OkHttpSource.OkHttpBuilder

V

W

withInputStream(Closure<V>) - Method in FileOpenable

withOutputStream(Closure<V>) - Method in FileOpenable

X

xls(File, String) - Method in XlsSource

Reads the given excelFile and pulls out all of the tabular data on the given sheet.

XlsProcessor - Class in gratum.source

XlsProcessor(Pipeline, Integer, Boolean, String) - Constructor in XlsProcessor

XlsSource - Class in gratum.source

A AbstractSource that implements reading excel workbooks in xls format also known as HSSF.

XlsSource(String, POIFSFileSystem, String) - Constructor in XlsSource

xlsx(File, String) - Method in XlsxSource

Reads the given excelFile and pulls out all of the tabular data on the given sheet.

XlsxSink - Class in gratum.sink

XlsxSink(File, String) - Constructor in XlsxSink

XlsxSource - Class in gratum.source

A Source that implements reading excel workbooks in xlsx format.

XlsxSource.XslxSheetHandler - Class in gratum.source

XlsxSource.XslxSheetHandler(Pipeline) - Constructor in XlsxSource.XslxSheetHandler

XlsxSource(File, String) - Constructor in XlsxSource

Reads the given excelFile and pulls out all of the tabular data on the given sheet.

Y

Z

ZipSource - Class in gratum.source

ZipSource(File) - Constructor in ZipSource

_