CSV source : a step further

The CSVSource task has been designed to receive a CSV file as input.

Basic usage

With little to no configuration, each line represents one document with different values matching the column header. From a Fast2 standpoint, a CSV with following content :

header1,header2
value1_A,value2_A
value1_B,value2_B

will generate 2 documents :

• The first document will have 2 data, header1: value1_A and header2: value2_A
• The second document will have 2 data, header1: value1_B and header2: value2_B

Although the default before resolved the data names from the 1st row (column headers), these names can be overwritten by the user, or even enriched.

Change data names

As for the first option (overwriting data names), the configuration needs to focus on the “New column names to set”.

Enter each new header on a new line, making sure your input covers all the columns found in the CSV file.

With such a setting, Fast2 will map the data retrieved from the CSV directly under those new data names.

Example

Let’s consider processing a CSV file with the following content:

header1,header2
value1 ,value2 

With the default settings, the document in Fast2 would have such dataset:

{
    "header1": "value1",
    "header2": "value2"
}

If the CSVSource task is configured as shown below,

the created document will only have a dataset looking like:

{
    "new header A": "value1",
    "new header B": "value2"
}

Fast2 will keep no trace of the old header names, generating a document with a dataset populated from the CSV file alongside new data names.

Of course this data name mapping could have been handled by an additional task, such as Drools or JSTransform (just to name a few).

But this CSV task here combines these 2 steps (of parsing and mapping) into a single one, lowering room for error and freeing the document of unnecessary information you’d not even have used.

Create extra columns based on existing data

This feature requires the configuration of the extracolumns option.

Syntaxe

Enter one line per new data you intend to create.

Syntax goes as follows :

[variable]=[function]:[param1]:[param2]:[param3]:[param4]...

Rules

1. The separator is the character : (semi-colon).
2. Parameters have to striclty match the format $<data_name>. A data with the name “key” will be accessed under $key.
3. Parameters can use other params

param1=stringLength:$keyA
param2=substring:$param1:3:5

Supported functions

Example

We consider the following CSV content as input :

header1,header2,header3,header4
value1 ,value2 ,value3 ,this-is-the-value4

In the following examples, a new data with the name ‘var_name’ will be created with the value depending on the chosen option.