Tuesday, November 1, 2016

CloverETL Milestone Review

I couldn't resist updating the blog with the latest features that CloverETL is releasing in the coming months for the production release. We'll call this a review of the CloverETL 4.4.M1 that is available to all existing customers in a test release only. It is not advised to upgrade your production systems until the full release of 4.4.0 is out.

Remote file event listener (out of the box functionality added)!! Yes, it was possible to handle remote file event listening prior to this update, but now, you can setup a file event listener for remote servers simply by configuring the server configuration (and reacting to the event in any way necessary).

Listener for failed listener - This is quite interesting and pretty easy to use. You can now configure 1 failed listener to listen for any failures in event handling/scheduling and react to the scenario. They are allowing for wildcards and matching of all events setup, or you can setup failed listeners for individual events configured on your CloverETL server. For example, if you have an event setup to grab a file from a remote FTP site, but your network becomes unresponsive. The failed listener can notify you that you have a failure contacting your event source when attempted to retrieve file changes when attempting to execute a file event listener and will allow the user to have a quicker response time to the outage.

Salesforce Updates:
If you are not aware, CloverETL has created a connector to Salesforce and is available in the production release of 4.3, however, the latest milestone release has seen a number of improvements using the Salesforce Connector.

1. Included a new Salesforce Writer using the SOAP API. This will allow you to write data into Salesforce which are considered micro batches. However, if you are working with large data sets being migrated into Salesforce, it is recommended that you use the Bulk API writer to limit the number of API calls required from Salesforce.

2. Updated - Milestone M2 will include the Salesforce SOAP API Reader. This will allow for subquery's as well as function calls directly in your SOQL query.

3. A more user friendly SOQL editor which will sort all objects and fields for the objects. This makes it much easier to search and find the appropriate object.

AWS Updates:
-Redshift driver bundled with CloverETL Designer. This will now ship with the product so you no longer have to setup your own driver in Designer.
-ParallelReader support has been added for S3 so you can now read data from an S3 bucket in parallel and improve performance.