Loading tasks

The loading tasks are the end of the process. They load your transformed data to a new data warehouse.

Two ways to perform the loading phase : the one-shot way with full load, or incremental load. Check their functional purposes here.


Perform a bulk import on Alfresco

Load documents and metadata into Alfresco without changing the current tree structure of those same documents. The good performances of such injection are restrained with the complexity of the tree-view setup

Mandatory settings

Key Type Description
Source directory String Path to the folder to migrate into Alfresco
Target path String Path where the folder will be stored into Alfresco
Alfresco connection provider AlfrescoConnectionProvider This modules is responsible of the two-way communication between Fast2 and the designated Alfresco instance.
Target NodeRef String NodeRef of the parent where the folder will be stored into Alfresco
Key Type Description Default value
Clean destination Boolean Replace all existing content or metadata present in the destination folder before injection false
Disable rules Boolean Disable rules for injection to prevent Alfresco to run checks on each document false
Add metadata Boolean Load document content and its metadata. All metadata might not me compatible with Alfresco standards false
Number of threads Integer Number of threads to allocate for the bulk import 1
Timeout Integer Time to wait before closing the session. Of not set, the value will be set to 300'000
Copy files Boolean Leave all documents in the source folder false
Batch size Integer Size of the batch to build for upload 0
Injection into Alfresco ECM using CMIS protocol

This task can be used to inject documents into Alfresco, using the CMIS protocol on top of HTTP. We rely on v1.0 of the opencmis module made available by Alfresco.

Mandatory settings

Key Type Description
Alfresco connection provider AlfrescoCMISConnectionProvider This modules is responsible of the two-way communication between Fast2 and the designated Alfresco instance
Key Type Description Default value
Alfresco ID key String Document metadata key with the Alfresco ID of the injected document. alfDocumentId
Property Helper PropertyHelper
Properties regex String Regex pattern to filter the properties to inject with the document. (cmis:.*)
Hash content column name String Hash content column name to version a document only when the content is different (but same index)
Destination folder String Folder where the documents will be loaded into
SQL update query String CMIS SQL update query to select the document to update.
Overwrite ‘can create’ Boolean Ask Fast2 to create destination folder(s) if they do not already exist true
Hash index column name String Hash index column name to version a document only when the content is different (but same index)
Prevent duplicate Boolean Fast2 will throw an error if the document has already been injected
Force update Boolean Throw an error if the document did not exist prior to the loading call
Alfresco injector using Alfresco REST protocol

This task relies on the Alfresco public REST API (with v1.0.4 of the Alfresco REST client) to load documents and metadata into a given Alfresco instance.

Mandatory settings

Key Type Description
Alfresco connection provider AlfrescoRESTConnectionProvider
Key Type Description Default value
Root folder name to inject in a specific repository String ‘-my-', ‘-shared-', ‘-root-’ are equivalent -my-
Alfresco destination path String The path of the folder where the documents will be saved in Alfresco. This field supports patterns (based on punnet, document and campaign metadata).
Regex pattern filter for document properties String (cm:.*)
Auto rename feature Boolean Triggers the Alfresco auto-rename feature, to prevent Alfresco to throw a ‘duplicate document’ error. false
Overwrite documents when they already exist Boolean Triggers the Alfresco overwrite feature, where the incoming document will replace an existing document having the same key. false
Injector into AWS S3 buckets

Fast2 proposes this task to load your documents, metadata and more within designated S3 buckets. Both client- and server-side encryption are supported (v1.6 of AWS encyption SDK). This loader relies on v1.11.848 of AWS Java SDK. The uploaded file will be title according to the `name` metadata of the processed document.

Mandatory settings

Key Type Description Default value
Destination bucket String The name of the bucket where the documents will be migrated to. fast2-default
AWS credentials AWSConnectionProvider Must have granted AmazonS3FullAccess permission
Key Type Description Default value
Encryption key String Key used for server-side encryption.
Ex/ 01234567-abcd-efgh-ijkl-0123456789ab
Destination folder String The parent folder of the documents to inject. This field supports pattern (using punnet, document or campaign metadata). Leave empty for storing at the root of the bucket.
Destination file name String Key of the metadata whose value will be set as the file name once injected into S3 bucket name
Dry run Boolean Simulates an injection, performs document integrity controls, but does not load the document into AWS S3 false
Encryption context String Context used for server-side encryption. This context is a JSON map.
Ex/ {"testKey":"testValue"}
Create destination bucket Boolean Create destination bucket if it does not already exist. Throws an error otherwise.
ARN key String Key used for client-side encryption, before loading the document into S3.
Ex/ arn:aws:kms::111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab
Update only Boolean Only changing metadata, content is left as is false
CSV file writer

Use this task to write punnet and document related data into a CSV. You can specify the name of such file as well as the path where your want Fast2 to create and populate it.

Mandatory settings

Key Type Description
Path for (local) output CSV files String Using a pattern for this field will trigger a resolution by Fast2 at runtime
Key Type Description Default value
Close output file at each line Boolean Close CSV file after each punnet processed. This option can come useful to prevent too many files opened errors when each punnet created a dedicated CSV file.
New column headers String list The new column names to use for the document metadata. If no set, the column headers will be populated from the names of the document metadata.
Multivalue column name String The name of the column having multiple values. Only one such column is supported as of now. When set, a line will be generated for all values holding this column (no patternResolver is used to resolve this value).
CSV separator String The separator used between columns in the resulting CSV file. ;
Write all in the same CSV file Boolean Merge metadata of all punnets in a single output CSV file. The missing columns headers will be added on the fly, although it is wiser to list them all in the ‘New column headers’ field. If set to false, any existing CSV file with the same name will be overwritten. true
Protect with double quote Boolean This option will surround every value with double quotes. Such use will be mostly relevant when dealing with multivalued metadata.
Injection into Documentum

Use this task to inject into Documentum ECM system. Fast2 embeds v6.7 of Documentum modules to take the most out of of this injection phase.

Mandatory settings

Key Type Description
Credentials DctmConnectionProvider Connection module establishing the communication with a given Documentum instance.
Key Type Description
Documentum configuration DctmConfiguration Customize here the Documentum details related to the instance you are planning to inject documents into. For more, refer to the appropriate section.
Injector for FileNet P8 3•5

Use this task to inject documents and data into a FileNet P8 3.5

Mandatory settings

Key Type Description
The FileNet connection provider FileNet35ConnectionProvider Module to establish the connection with the destination FileNet infrastructure
ObjectStore name String Name of the destination object-store where the documents and metadata will be injected
Key Type Description Default value
Try using ‘DocumentTitle’ property Boolean When filing a document in a folder, try to use the FileNet DocumentTitle property for the RelationShip name
Dry run Boolean Needs to set ‘UpdatingBatch on documents’ to true
Process multi-pages content as multi-content Boolean Treat the content of multi-pages document as multi-content document in FileNet.
WHERE clause for folder case String where clause to define for case research
Restrain search results to documents Boolean Force Document search to limit to the class name provided on the document
Only process 1st content Boolean Relevant for multi-content documents
Restrain search results to folders Boolean Force folder search to limit to the class name provided on the folder
Skip content injection Boolean Skip document content injection, only load the metadata and/or annotations of the processed document
Skip document unfiling Boolean Force to skip Document unfiling to existing folders before linking it to the provided folders ; new linkages will be added to the existing ones
Restrain search results to case Boolean Force case search to limit to the class name provided on the case
Clear in-place annotations Boolean If source document contains annotations, cleanup existing ones in P8
Synchronous folder creation Boolean Enforce synchronous folder creation, to make them more thread-safe true
Metadata carrying document UUID String Leave empty to disable updating fileNetDocumentId
WHERE clause for folder research String Fast2 will update all folders matching the following WHERE statement
Force deletion Boolean If no matching document can be found, an error is thrown
Variable name of annotation ID String Variable name of annotation id used to replace it by generated FileNet annotation id ${annotationId}
Associate annotation FileNet ID to its content Boolean Update annotation content with its generated FileNet id according to annotation id variable
Delete in-place version Boolean Delete the last document version after checkin a new one
Update system properties Boolean It can only be used for either document creation or update (when a new version is created)
Default MimeType String Mime-type to set when none has been found
Fields to update String Default query to select fields to update
Post-commit delta Integer Add a post-commit time, may be usefull to let FileNet perform asynchronous handling of document injection 0
Limit CE connection life-time Long Limit CE Session life-time : at end of TTL, the Session will be replaced by a brand new one Long.MAX_VALUE
Force to perform update Boolean In case the document did not exist, an error is thrown
WHERE clause for update String Fast2 will update all documents matching the following WHERE statement
Ex/ [Id]=${myFileNetDocumentId})
Injector for FileNet

Use this task to inject documents and data into a FileNet. To inject a document with multiple attached versions, Fast2 will look for the VersionSeriesId' metadata in each document dataset. If all documents have the same UUID for this data, then the FileNet versionable object will be created based on the 'MajorVersionNumber' and 'MinorVersionNumber' properties.

Mandatory settings

Key Type Description
The FileNet connection provider FileNetConnectionProvider The module establishing the connection to the remote FileNet instance. For more configuration about this object, refer to appropriate section.
ObjectStore name String Name of the destination object-store where the documents and metadata will be injected
Key Type Description Default value
Property Helper PropertyHelper
Try using ‘DocumentTitle’ property Boolean When filing a document into a folder, try to use the FileNet ‘DocumentTitle’ for the RelationShip name
Dry run Boolean Do not perform anything, just prepare UpdatingBatch and drop when finished. It implies to activate ‘Use UpdatingBatch on documents’.
Process multi-pages content as multi-content Boolean Treat the content of multi-pages document as multi-content document in FileNet
Restrain search results to documents Boolean Force Document search to limit to the class name provided on the document
Metadata carrying parent folder UUID String Name of the metadata where Fast2 will store the UUID of the parent folder of the document injected into FileNet P8. Leave empty to disable updating
Only process 1st content Boolean When a document has multiple contents, its forces to process the first one only. The others are then skipped
Accept unset properties Boolean Allow registration of blank metadata in FileNet
Skip content injection Boolean Skip document content injection, only load the metadata and/or annotations of the processed document
Name of ID property String Name of the document property, found in the document dataset, which will be used to force Id at document creation. Leave blank to disable this feature
Throw exception if document already exists Boolean An exception is thrown in case an older document has been found. To properly use this options, ‘Prevent document overwriting’ requires to be true
Skip document unfiling Boolean Force to skip Document unfiling to existing folders before linking it to the provided folders. New linkages will be added to the existing ones
Auto-classiify at checking Boolean Enable the FileNet Auto-Classify feature when the document is at checking stage
Clear in-place annotations Boolean If source document contains annotations, clean up existing ones in P8
Synchronous folder creation Boolean Enforce synchronous folder creation, to make them more thread-safe true
Metadata carrying document UUID String Name of the metadata where Fast2 will store the UUID of the document injected into FileNet P8. Leave empty to disable updating fileNetDocumentId
Safe update of document Boolean Try updating a document. If no older version of the document can be found, create it
Force deletion Boolean Force document delete action. If no matching document can be found, an error is thrown
Variable name of annotation ID String Variable name of annotation id used to replace it by generated FileNet annotation id ${annotationId}
Use UpdatingBatch of folders Boolean Use FileNet UpdatingBatch also for folders creation, which may not be thread-safe
Associate annotation FileNet ID to its content Boolean Update annotation content with its generated FileNet id according to annotation id variable
Delete in-place version Boolean Delete the last document version after checkin a new one
Update system properties Boolean It can only be used for either document creation or update (when a new version is created)
Use UpdatingBatch on documents Boolean Run the ‘UpdatingBatch’ feature of FileNet, at each punnet being processed.
Default MimeType String The mime-type to set when no MimeType has been provided neither in document nor its content
Limit CE connection life-time Long At end of TTL, the connection will be replaced by a brand new one Long.MAX_VALUE
Fields to update String Default query to select fields to update
Force folder creation Boolean Overwrite folder canCreate property : create folders when they do not exist
Post-commit delta Integer Time to wait after a commit instruction, may be useful to let FileNet perform asynchronous handling of document injection 0
Prevent document overwriting Boolean Check if the document already exists before creating it using WHERE clause. You can throw an exception in case an older document can be found (see Throw exception if document already exists). If false, create all documents without control
Force to perform update Boolean Force document Update action. In case the document did not exist, an error is thrown
WHERE clause for update String The criteria which the documents to update will have to match
Ex/ [Id]=${myFileNetDocumentId}
Metadata to use for VersionSeriesId String Fast2 will create versioned documents based on the value matching the key entered here. If provided, the FileNet document will keep this VersionSeriesId; if not, the new value created by FileNet will be stored as metadata in the Fast2 document under ‘New_VersionSeriesId’. Leave empty to let FileNet create the VersionSeriesId.
Email sender task

This task will send custom built emails to specific people or mailing list of your choice.

Mandatory settings

Key Type Description
Email provider MailSenderProvider The Fast2 module establishing the connection to the email server, from the account of a given user. For more about the configuration of the object, please refer to the appropriate section
Key Type Description
Email subject String The subject/object of the email to send.
Sender address String The value of this field can be resolved by Fast2 (ie you can use dynamic values)
Recipient address(es) String list The value of this field can be resolved by Fast2 (ie you can use dynamic values)
Multi-line content String list The content of the email. It can be composed with different paragraphs, please note Fast2 is not responsible for the text formatting though.
Inject documents into your ASG Mobius system

This task will upload documents and metadata onto Mobius, from version 8 up to version 11. Based on the `className` and `section` properties, specify exactly where you'd like your documents to be stored.

Mandatory settings

Key Type Description
Mobius connection provider MobiusConnectionProvider The Fast2 module required to establish the communication with the destination Mobius instance
Key Type Description
Properties to inject into Mobius String list List of names of the properties which will be added to the topic of the document. These properties have to be attached to the document
Update a database with several document data

Mandatory settings

Key Type Description
WHERE clause String All matching rows will be updated. This field will be resolved by Fast2 before the task is executed
Query caller SQLQueryGenericCaller This modules is responsible of establishing the connection between Fast2 and the designated database
Table name String The table of the row to update
Key Type Description Default value
Skip exception Boolean Fast2 will either throw an error if no update action has been executed, or proceed to next document true
Data to update String list The list of all key-values pairs to update the given rows with
Ex/ targetColumnName/documentData
Add multiple data to documents in database

Complex task to perform a SQL database query and fill each Document data with results

Mandatory settings

Key Type Description
Query caller SQLQueryColumnCaller This modules is responsible of establishing the connection between Fast2 and the designated database
Source attributes String/Pattern map Key: Desired column for where clause; Value: Source Document data to use for query
Key Type Description Default value
Use PreparedStatements Boolean Use PreparedStatements instead of plain SQL statements
Target attributes String/Pattern map Key: Desired column for select clause; Value: Target Document data to fill result into
Reset target data Boolean Clean content when target already exists true
SQL column types String/String map SQL column types mapping (String, Integer), keep blank for default behavior (String)
Add data to documents in database

Simple task to query a SQL database and fill each Document data with results

Mandatory settings

Key Type Description
Key of target data String The name of the data where the value must be added
Query caller SQLQueryColumnCaller This modules is responsible of establishing the connection between Fast2 and the designated database
Key of source data String The name of the data to update the documents with. If the data is not retrieved from the document, Fast2 will skip this document
Key Type Description Default value
Reset target data Boolean Clean content when target already exists true
Insert or updated database

With this task, you will be able to perform any SQL instruction (such as insertions and updates) on any given table of the specified database

Mandatory settings

Key Type Description
SQL statement String The statement you want Fast2 to run on the database. The syntax needs to match SQL standards
Query caller SQLQueryGenericCaller This modules is responsible of establishing the connection between Fast2 and the designated database
Key Type Description Default value
Skip exception Boolean Fast2 will either throw an error if the statement has not properly been executed, or fail silently false
Update SQL database

This task will perform update instructions base on document data onto a given SQL database

Mandatory settings

Key Type Description
SQL connection provider SQLQueryGenericCaller The module establishing the communication between Fast2 and the designated database
Name of the new column String The name of the column which will be added to the row with the value to update
Table name String The name of the SQL table on which all update statements will be performed
Key Type Description Default value
WHERE clause String All matching rows will be updated. This field will be resolved by Fast2. Leave empty to target all rows.
Ignore when no row updated Boolean Skip exception when no database row has been updated true
Value to update String Name (= key) of the document metadata whose value will be inserted into the row. If none is found in the document, this latter is skipped