What is Fast2 ?
Fast2 is an ETL (Extract-Transform-Load). It embeds a workflow engine specially designed for mass migrations.
Collect data from multiple sources (Data Warehouse, ECM Engine, Excel…). Transform them to give them even more values. Whether functional or business rules, Fast2 can be adapted to your needs. Last but not least, inject your data into a newer ECM system for instance. The list of connectors is constantly growing, offering new features each time.
Data traceability is an important aspect in any type of migration. That’s why we decided to couple an instance of Elasticsearch to Fast2. All data that has passed through Fast2 will be stored in Elasticsearch (data, logs…). As long as a Kibana is set up on your environment, you will have in hand a complete solution to monitor your migrations and check that everything is compliant, as you expected.
Fast2 is based on Java. To turn it into a web application we use the famous framework GWT. Even if it’s not longer supported by Google it offers many benefits :
- The code generated by GWT supports all major browsers
- Rich and complete libraries to build complexe user interfaces
- Easy debugging mode designed for developers
- Communication with servers through asynchronous calls
- A history management system on the browser
Precisely, we used the MVP (Mode-View-Presenter) pattern to build the Fast2 v2. Besides, the Google implementation of this pattern is named Activity & Places (A&P). As you can guess, the Activity takes the role of the Presenter. The Place will be the View. A place is considered as a web page and is therefore in the history of your navigator.
For the design of widgets we used the uiBinder tool to map XML objects to java objects. The scheme below is here to show the different place which a user can visit.
- Design Place : Workflow management (add tasks, configure them…)
- Run Place : Run your map and see performance statistics
- Explorer Place : Look into a specific task to see what happend inside
- Configuration Place : Manage your maps, campaigns, shared objects and workers
- Monitor Place : Redirect you to Kibana URL linked to the Fast2’s ElasticSearch instance
Fast2 is accessible from the port 1789. It can be easily modify from the configuration files.
Actions performed by users are transmitted to the broker through a REST API. A dedicated worker is always present for the broker. However, you can add many workers as you wish depending on your needs. New workers will communicate to the broker through another API to be synchronised between them.
When workers have done their job, the rendering is shared with the broker who hastens to store everything in ElasticSearch to maintain maximum traceability.
A Kibana instance (accessible from the port 5601 by default) is connected to ElasticSearch to build reports of migration.
Known as the first product of the suite ELK, ElasticSearch is a powerfull noSQL database. It uses directly document metadata rather than schema or tables. One the many benefits, its an open source system allowing user real-time searching and analyzing of complex date system. Having noSQL allows you to have a scalable product and even more faster.
With a documented REST API coupled to our broker, we built a whole system able to store, trace and analyze your data.
Kibana is like the fifth wheel in the cart, to be precise the third since is the last product of the ELK suite. It will fill the missing ElasticSearch features like visualization and data management. You will have the ability to build histograms and many others types of graphs. This is the perfect tool for creating relevant reports of a migration. Your data will no longer hold any secrets from you.