The explosive growth of information in the last few years has led to a proliferation of databases in most organisations. The goal most strive for is to harness this data for better decision making; but this is only possible where a single, coherent view is achieved. To enable this, organisations have been using a process called Extract Transform and Load (ETL).
ETL enables organsations to move data from multiple sources, reformat, cleanse it, eliminate duplicates, test for quality and consistency and then load it into another database for analysis, or to an operational system to support a business process.
But there is a big problem because data lies in disparate systems in many different formats. ‘Customer’ may be defined in the CRM system in but called ‘Account Holder’ in the accounting system.
In the past, organisations developed their own hand-coded approach to this problem; but this has proven to be an inefficient use of resources. As the source data file formats and the validation rules applying to the data evolved, the ETL code had to be modified and maintained. Companies encountered problems as they added systems and the amount of data in them grew. Lack of scalability has been a serious issue with homegrown ETL software.
ETL software tools will solve this problem; some are more effective than others. All require a learning curve of some degree which impacts their time-to-benefit.
ETL as a service
We deliver ETL as a service. Whether it is a one-time activity or on-going we use our expertise coupled with our Transformation Manager (TM) Data Integration platform to create a personalised solution.
The key to the longevity of our solution is the ability to easily and quickly update the source data mappings and new systems inputs. With a hand-coded approach this is just not possible and often requires a rewrite each time. This is time consuming and very expensive which is why our approach has a strong appeal.
Benefits
The biggest benefit of our solution is that it allows teams of business users to operate from a single version of the truth. Our clients are using ETL not only for data warehousing and business intelligence activities but also for moving data from one operational system to another; such as from an ERP system to a CRM application.
One client who manufacturers telecommunications equipment, collects information from 20 different procurement systems then sends it to its global supply chain management data warehouse to analyse aggregate component spending; enabling significant economies to be achieved throughout the procurement process.