Monday, May 2, 2016

CloverETL Products

When talking about CloverETL with various business, sales, and technical resources it is often hard to communicate which CloverETL product your project requires. Before I dive into the reasoning that I use behind each product, I want to briefly describe each of the products.

The CloverETL Designer is a visual tool that's used for development and manual execution of graphs/jobflows that the developer builds. This is a great tool for debugging your logic in a graphical environment.

The CloverETL Corporate Server allows the developer and/or data analyst full workflow management. The user can setup schedules for jobs to execute, event listeners, publish web services, and create an orchestration framework for your project.

The CloverETL Cluster in addition to all of the CloverETL Corporate Server functionality, allows the users to setup job parallelism with load balancing and failover redundancy.

Now that you have a quick understanding of the products that CloverETL offers, I am going to give you a few scenarios and let's see if you can determine which CloverETL product you would recommend.

Scenario 1: I (along with tens of thousands) play fantasy baseball so I like to process all historical statistics on a yearly basis to track player trends prior to the start of my fantasy baseball draft. This process will ingest all statistical category and can offer a value for a particular player for their estimated value the next year.

Scenario 2: I am a baseball organization who tracks statistics for every baseball game that is happening on a given day. I execute the process at the conclusion of the final game of each night that will update all statistical categories, hitting trends, pitching trends, and fielding trends.

Scenario 3: I am a broadcasting organization which tracks statistical trends as they happen in real-time. If a player strikes out in the fourth inning, I want all statistics and trends updated when they hit again in the sixth inning.

Which CloverETL products would you recommend and why?

Scenario 1: I would recommend the CloverETL Designer. This is more of a manual process that is being run on a yearly basis. You can save the logic that you have created and change the input for another year's worth of data.

Scenario 2: I would recommend the CloverETL Corporate Server. This should be an automated process because it's happening every night. You can schedule the job to run at a certain time of the evening, automate all logging, and create a generic workflow for handling a number of scenarios programmatically.

Scenario 3: Personally, I would recommend the CloverETL Cluster, but you could use the CloverETL Corporate Server. The reason why I would recommend the Cluster in this instance is because we are talking about a business critical function since the broadcasting company relies on this data being always updated. I would setup the Cluster for parallel processing with load balancing to execute the job faster as well as setting up the nodes such that the job could still execute if one of the nodes was unresponsive.

I hope you have learned a bit about which CloverETL product is best for your project. If you have any questions, please don't hesitate to comment below and I would be happy to respond with any recommendations.

No comments:

Post a Comment