The installation of Fast2 requires a few environment specifications to run properly :
- JRE11 : you can either get it from Oracle or OpenJDK. If you have multiple JDK/JRE already installed, point out the correct one in the
config/env.propertiesfile of the Fast2 installation folder.
- RAM : we highly recommend having 8GB or more, although 4GB is the strict minimum.
- Processor : 8 CPUs.
Make sure to confirm the compatibility between Elasticsearch and your environment at Elasticsearch Support Matrix.
The Fast2 distribution you need depends on your target environment. It exists three way to deploy a Fast2 :
- locally : Classic package, its an all-in-one zip files
- AWS : Standard AMIs
- K8S : Docker Images
Each distribution ships the following
- A broker with an embedded worker and a user interface
- An additional worker with all tasks catalog
- A template to create workers with custom tasks
Root folder anatomy
Start Fast2 Broker
Whether Fast2 is launched from the batch file or as a service on your environment, the UI will be available at http://localhost:1789/
By default, Fast2 Broker starts an embedded Elastic Search and an embedded Fast2 Worker.
All commands below are to be run under the Fast2 install path (where the Zip has been unzipped).
From command line
Go into the Fast2 install folder, and run :
Administrator rights might be required since Fast2 will handle some port communications.
To end the Fast2 process, just hit
Ctrl+C in the command line the startup file opened.
The following Linux installation steps work for most of Unix-based sytems. Elasticsearch cannot be started from root user, you will need to create a secondary user to start the database binary alongside your Fast2 process.
Once connected as the latter user, start Elasticsearch via its binaries.
Then, since you started Elasticsearch manually, disable the command triggering Fast2 to start the embedded Elasticsearch, from the
config/application.properties file :
You can now properly execute the following script:
To quit the Fast2 process, just hit
Ctrl+C in the command line the startup file opened.
Go into the Fast2 installation folder, and open the Windows Command Prompt.
To install the service :
C:\path-to-fast2\service\windows> Fast2_broker_service.exe install
Your machine may prompt a message asking to download .NET components. Please click [OK] and proceed.
Once this command is complete, you should see in your services registory a newly installed Fast2 service. You can start/stop/restart it as any other service, or via the Command Prompt (just replace
install in the previous command by start/stop/uninstall/restart/status).
The logs of the service will be available from the
There are several ways to create a service under linux distribution. We will do it through systemd. Its major benefit is that it has been the default init system for the majority of linux distributions (Ubuntu, Red Hat, Fedora…).
ExectStart field from the file
service/linux/fast2-broker.service by changing the
PATH/TO/FAST2 portion: set it to Fast2 install path.
[Unit] Description=Fast2 Broker [Service] ExecStart=/home/userName/fast2-complete-package-2.0.0/startup-broker.sh [Install] WantedBy=default.target
Now link it to the
/etc/systemd/system directory through a symbolic link.
$ sudo ln -s SOURCE TARGET $ sudo ln -s service/linux/fast2-broker.service /etc/systemd/system
If the links are broken once they’re created, you probably need to put an absolute path for the target as follow ;
$ sudo ln -s /home/userName/fast2-complete-package-2.0.0/service/linux/fast2-broker.service /etc/systemd/system
Next, reload systemd services unit and enable them :
$ systemctl daemon-reload $ systemctl enable fast2-broker.service
The terminal should prompt the following message :
Created symlink from /etc/systemd/system/default.target.wants/fast2-broker.service to /etc/systemd/system/fast2-broker.service.
Test your script by starting it and then checking the status :
$ service fast2-broker start $ service fast2-broker status OR $ systemctl start fast2-broker.service $ systemctl status fast2-broker.service
You can restart or stop the service at anytime with the commands :
$ service fast2 start | restart | stop | status
Start Fast2 Worker
The Broker starts an embedded worker by default. If you wish to start multiple workers, just hit :
If the worker and broker are not booted up on the same machine, you need to setup the Broker host name in the worker configuration file.
Edit the file
config/application.properties and modify
You can setup Fast2 Worker as a service the same way you did for the Fast2 Broker.
Every object passing through Fast2 is stored into Elasticsearch. Metadata carried by the document are also kept in the warehouse. The major benefit is the opportunity to check whether everything is going well by making counters about documents/data processed during your migrations. In addition, we can easily come back in case of server crashes. Nothing will be lost and Fast2 will precisely know where it all stopped. There is the logic behind real-time backups in ES.
Each Elasticsearch index referenced by Fast2 will be written with a
f2_ prefix. An index is always written in lower case even if the campaign name in Fast2 contains characters in upper cases.
At the first broker run, some indexes will be automatically created :
- f2_campaigns: list of existing campaigns with processing dates and status
- f2_campaigns_sources: links between campaigns and workers having performed this campaign
- f2_queue_settings: reference information about source and task threads
- f2_jobs_settings: gather the configuration of save jobs
- f2_jobs_info: information about jobs past execution details
For each new campaign of Fast2, an index will be created: if you decided to run a new campaign named
EcmInjection, the new index will be
Elasticsearch without Kibana
To visualize data stored in Elasticsearch without using Kibana, we recommand to use web navigator plugins. These GUI are focused on table visualisation of your documents, more easily than with Kibana. You can browse your data as classic SQL tables. The downside is ElasticVue does not offer any metrics, graphs or anaytics options. Two different plugins exist, depending on your web navigator:
- ElasticVue under Mozilla Firefox, MS Edge and Google Chrome
- ElasticSearch Head only for Google Chrome
The elasticsearch and kibana binaries files are not allowed to being executed by default. Open a terminal, go to your Fast2 root folder and execute the following command:
sudo chmod -R 777