Etl Solutions - RSS Updates Feed

Software requirements for Solvency II

ej@etlsolutions.com (Editor) — Wed, 25 Jan 2012 00:00:00 GMT

Accurate transfer of data is critical in the highly complex and fast-moving Financial Services sector. After its introduction in 2014, the requirements of the Solvency II directive will increase the need for the timely transfer of data to the regulatory authorities. This guide outlines how data quality and data integration software can be optimised to meet the Solvency II requirements.

A quick overview of Solvency II

Solvency II redefines the required capital for insurance companies to operate in the EU. The aim is to reduce the risk of insolvency and increase consumer protection. The framework has three main pillars:

Pillar 1 defines the capital requirements for an insurance company, such as the amount of capital the insurer should hold;
Pillar 2 outlines the requirements for the governance and risk management of insurance companies;
Pillar 3 focuses on disclosure and transparency requirements.

For a company to adhere to the Solvency II requirements, its data strategy for the three pillars must be detailed and robust. Low data quality and poor transfer strategies increase the potential of inaccurate risk capital calculations.

Pillar I directives

There are two possible approaches to collating the information available for calculation of Pillar 1’s Technical Provisions (TP) and Solvency Capital Requirements (SCR):

Wrap the data around the directives. Centralising information into a single data store allows calculations and audits to be performed against the store;
Wrap the directives around the data. This involves ad hoc consolidation of information, as required, to perform the necessary calculations and audits.

Both approaches have similar software requirements. The main difference is that the ad hoc option requires the data to be migrated and consolidated on a frequent basis to allow the calculation of the TP and SCR.

Data migration requirements

The optimal data migration software capabilities include:

Data connections to the various silos, with clean, transparent mapping logic to define business rules and system level rules;
Access to all data source types including databases, XML, flat files, Excel, and custom structures with generic adapter capability;
Manipulation of data and structures including:

De-duplicating records based on surrogate or natural identification
Addition and manipulation of relationships within the data models for access to data and performance gains;

Data cleansing abilities.

Data quality requirements

Data quality checks should be carried out prior to migrations to assess the current quality of the data and highlight any issues. Software should be able to perform data verification during the migration, validating the data against business and system-level rules. Ideally, the software should also use these rules to validate data as it transfers between internal data silos or external third parties.

Example rules:

The value date of a policy must be after the birth date of a customer;
The value must match an enumerated list, such as gender of a customer being male, female, or a company;
An insurance policy book code does not match any entry from the deal book table;
The type of policy must be coherent with the value of the premium periodicity.

Data requires continuous verification: information can contain a multitude of anomalies when entered by users, or when transferred between systems or from third parties. Accuracy, range, and structural verification are a few of the checks required to maintain fit-for-purpose data.

Data reconciliation requirements

Ideally, the software should provide reconciliation of information by consolidating information directly from the data stores and comparing that against official data records.

Logging and error reporting requirements

During data migration and data verification, logging and error reporting is essential. It should be defined to provide the user with detailed information about data that has failed to adhere to the business and system rules.

Pillar II directives

Using specialist software to implement parts of the enterprise management system demonstrates that dedicated tools are being used to handle the data. This shows the authorities that the company has a stable platform for data migration, data verification and data cleansing.

For maximum efficiency, it is important that the software provides future proofing by:

Updating model structures as new versions of central stores or the various data silos are created;
Making change requests to the business logic in accordance with future directives;
Connecting to new systems when new data silos are brought online, or when existing silos need to be joined to access more data as the directives progress.

Using a specialised tool to provide data management services creates a high level of confidence about the process being created and implemented.

Pillar III directives

The data management software should provide a clear, stable platform for providing reports to the regulatory body. For instance, our Transformation Manager software uses its migration functionality to generate XBRL files composed of data from the centralised store or workflow data. This involves:

Consolidating data from data stores to populate the reports and automatically formulate it into XBRL based on business rules; or
Formulating data into XBRL directly from a single source based on business rules.

Using a specialised tool like Transformation Manager reduces risk by addressing low-level core component issues directly. This enables timely intervention and correction, preventing costly reruns further down the line.

To request a demo of Transformation Manager and to see how the software can help manage the Solvency II requirements, contact us at info@etlsolutions.com.

Case Study: Oil & Gas Data Connectors

ej@etlsolutions.com (Editor) — Thu, 12 Jan 2012 00:00:00 GMT

Our client supplies the Oil & Gas industry with first class data integration solutions. The company started development of its framework in 1997 to allow interoperability between multiple vendors' applications and data. The framework is now a highly popular integration standard, used across the industry to ensure increased productivity and improved workflow efficiency.

The Challenge

Data in the Oil & Gas industry is often held in multiple data stores, the result of a diverse range of legacy projects and inherited IT systems. Furthermore, the increasing complexity of oil and gas reserves is driving a more complicated data environment.

Our client provides a common data model which allows applications to access data held in Exploration and Production (E&P) data stores. For optimum performance, queries and updates against this common model should translate into efficient queries and updates against the underlying physical data store. This translation must be performed dynamically at runtime, a process which requires sophisticated algorithms and intimate knowledge of the metadata of the target systems.

The Solution

ETL Solutions formed a partnership with the client in 2005. The collaboration enables the client to use ETL’s Transformation Manager software for the generation of data connectors to a range of Oil & Gas data stores.

The primary objective of the partnership is to improve the development process for data connectors, enabling the client to increase the breadth and depth of its model support. A key feature of ETL Solutions’ work with the client has been the development of Transformation Manager’s Query Propagation feature.

Query Propagation allows users to very efficiently query underlying data stores in terms of the overall model. In its simplest form, the Query Propagation feature enables queries to be specified in target model terminology; for these to be automatically converted to source model terminology; and for query results to be transformed to target model terminology for use by the application.

In addition to the management of structural differences, the feature ensures that reference data, coordinate systems and units of measure are all correctly processed.

A similar cycle is implemented for writing to a data store. ETL Solutions’ professional services team have worked directly with the client to design, develop and maintain the company’s latest data connector to the Professional Petroleum Data Model (PPDM) data stores. This enables data to be seamlessly exchanged and accessed by a range of E&P systems. The PPDM data connector supports v3.8, the most recent version of the PPDM data standard, giving customers the choice of reading and writing to the popular PPDM 3.7 or the latest 3.8 data stores.

The result? Greater efficiency and a long-term partnership which continues today. Contact us at info@etlsolutions.com for more information on our products and services for the Oil & Gas industry.

Resolutions for 2012

ej@etlsolutions.com (Editor) — Wed, 4 Jan 2012 00:00:00 GMT

A fast-paced 2011 saw us expand our team, refocus our activities and beef up our marketing plans. We are grateful to our customers and partners for their support in what has been a challenging time for businesses all over the world.

We have an exciting year ahead of us, as we continue to push forward our commitment to providing our customers with data integration software, expertise and outstanding support. Building on this, we have created our own New Year resolutions for 2012.

Focus on delivering software for even the most complex data integration challenges. Our developers are working hard on a new release of our Transformation Manager software suite, due later this year. Transformation Manager is a tough integration toolkit for developers, with the flexibility to tackle the most difficult data movements. The new version of the software will include improved management of multiple projects and models, greater efficiency, a more effective debugger and testing system, plus an improved interface.

Offer the best possible support. We know that if an issue arises before, during or after a project, a quick resolution is vital. Our customer support is provided directly by our team of developers, because we believe that this is the most efficient way of resolving customer queries quickly and effectively. This year, we will undertake a complete review of our customer service system to ensure that we continue to offer industry-leading support.

Grow our expertise, delivering solutions to spec, on time and in budget . “Here I am, brain the size of a planet, and they ask me to pick up a piece of paper.” Unlike the case of poor Marvin in The Hitchhiker’s Guide to the Galaxy, we make better use of our team’s sizeable brainpower. Our data integration experts will continue to grow their experience in 2012 via a programme ranging from increased domain knowledge to collaboration with outside experts. While Marvin chases stray bits of stationery, our team’s development will translate into continuous improvement to our products and services.

And celebrate! April 2012 sees the 10^th birthday of ETL Solutions. We are lucky to enjoy the stability that comes from the fact that members of the launch team are still with us. But we will continue to expand our team as we tackle tough data in an ever-changing world.

What are your resolutions for the New Year? And how can we help you achieve them?

For more information on Transformation Manager, please contact us at info@etlsolutions.com. We’re looking for tough situations in which to stretch the software, so please get in touch!

The latest PPDM compliant tool for Oil & Gas

ej@etlsolutions.com (Editor) — Fri, 9 Dec 2011 00:00:00 GMT

The Professional Petroleum Data Management Association (PPDM) has awarded 100% Compliance Gold Certification to Transformation Manager, ETL Solutions’ data integration software.

Transformation Manager is a development toolkit used by the Oil & Gas industry for effective data migration and data integration. Now 100% PPDM compliant, the software directly supports PPDM data transformation projects via its powerful Data Loader tool.

Karl Glenn, Business Development Director of ETL Solutions said, “I am delighted that our software has achieved this rating from the PPDM. It is an important step forward for the company, particularly with our focus on data integration in the Oil & Gas industry.”

The PPDM Data Loader consists of a pre-built and fully tested PPDM model pack, user guide and PPDM-specific tutorials. The model pack includes PPDM 3.7 and PPDM 3.8 support. The company also provides packs for WITSML 1.3.1 and 1.4.1, LAS 2.0 and 3.0, and MMS Well Information.

For more information on our data integration expertise in the Oil & Gas industry, email us at info@etlsolutions.com.

How to Create a Successful Proof of Concept

ej@etlsolutions.com (Editor) — Tue, 22 Nov 2011 00:00:00 GMT

An effective proof of concept (POC) bridges the gap between expectations and reality. For developers of data migration software, it is not only the best way of illustrating features, functions and benefits, but also ensures that the end product matches the client’s expectations.

In this guide to creating a proof of concept for migration software, we’ll explain how to keep the client in focus and add some tips on successful preparation too.

Explore why the customer needs the software

Carefully scope the client company’s requirements for the final product. Ensure that the scope is accompanied by a clearly defined way of measuring success.

Also explore the business reason behind the customer’s need for the software, and clarify how the product will assist in overall business improvement. A full view of the business’s needs helps to identify any extra requirements necessary to ensure a successful migration. In addition, assess the customer’s systems to understand the impact on them of the final product.

Understand exactly what the customer is looking for

Clarify all the outstanding questions that the customer expects the POC to answer or demonstrate. Some of these will be direct, such as:

• How long will it take to implement the software?
• How will the product be structured?
• What will the interface be like?
• How much will the product cost?
• How will it be deployed within our architecture?

Also prepare for questions which are indirect and wider in scope, which may include:

• How will it fit with our current products?
• How will it fit with our current processes?
• Do our users believe in the product and are they enthused by what they see?
• How easy is the product to use?

Assess where the proof of concept will be deployed

The customer’s data stores and offices may be dispersed around the world. For each location, consider how the product will be deployed on the site’s servers.

Establish when the customer requires the proof of concept

Ensure all deadlines are clearly listed in advance to allow suitable time for follow-up actions.

Consider whether the execution of the POC should be restricted to particular times or days. Ensure that the windows for execution are fully communicated within the client company.

Clarify who the stakeholders are

It is usual for a wide range of stakeholders to have an interest in a POC. It is the first impression the customer will have of the end product, and first impressions last. Often, more people are exposed to the product in the short timeframe of the POC than will have exposure to it for a good while afterwards.

Typical stakeholders include mangers, architects, security experts and users. A successful early introduction to the project’s stakeholders will encourage decision-makers to give go-ahead quickly and cost effectively. This means that it is crucial to understand the requirements and expectations of all these groups.

Prepare for the practicalities

There are many useful ways to ensure that the POC progresses without issues. In addition to being fully prepared from your own standpoint it is important, where possible, to ensure the customer has carried out any preparation that they are required to do.

A key item is availability and access to all the components required for the POC. From the customer you will need:

• Access to:
o Buildings (if onsite);
o Networks;
o Datastores;
o Live or sample data.

• Availability of key personnel such as domain experts and systems experts.

And from your own viewpoint:

• Ensure that any customised components are complete;
• Where possible, review sample data sets and data schemas;
• Confirm that success measurement criteria are clearly defined and agreed;
• Expect users to highlight particular issues they are experiencing;
• Ensure the POC shows what happens when things go wrong and how mishaps are handled.

A POC does not have to cost a great deal, but its thorough preparation is a crucial stage in the success of a data migration project. What are your tips for a successful proof of concept?

The Middleware Solution to Oil & Gas Migration

ej@etlsolutions.com (Editor) — Thu, 27 Oct 2011 00:00:00 GMT

The complex challenges of migrating data in the oil & gas sector need to be met with similarly sophisticated solutions. There is no one-size-fits-all methodology that can be applied to all the disparate sources of data and all the proprietary systems in operation within oil & gas companies.

One of the major challenges – especially when centralising data in a Master Data store to standardise data management according to the PPDM model – is getting different hardware and software applications to communicate with each other. Development teams, whether they’re in-house or a specialist middleware provider, don’t want the data integration solution to affect the overall architecture, and want solutions that work with and around it.

Development teams often use popular enterprise frameworks including Java Enterprise Edition or CORBA to provide the infrastructure for the data integration workflow. However, hand-coding the data integration components can cause similar problems to a hand-coded data migration solution, which is labour-intensive and prone to errors.

This is where ETL’s expertise comes into its own because we have a solution that integrates easily and reduces the sometimes tedious effort required to transform data.

One major middleware company uses our Transformation Manager (TM) to develop data connectors, which are complex components with the ability to transform queries and updates against one data model into the equivalent query or update in another. TM is used to develop mappings between the two data models, in exactly the same way as if you were implementing a data migration. The generated code is then integrated into the standard connector interfaces to perform the query transformation.

As well as boosting productivity within oil & gas companies, we make it easier for third parties to develop the connectors. For example, we have designed and implemented a data connector that seamlessly integrates PPDM-based systems with other disparate oil & gas systems.

The ETL PPDM 3.7 and 3.8 Data Connectors allow data from leading oil & gas systems to be reformatted, adhering to the PPDM data standard, and written to a PPDM-based system. As well as providing write access, the PPDM Data Connector allows data to be read from a PPDM system. When reading the data, the PPDM Data Connector can be configured to reformat the data. This allows the data to be retrieved in the data format of the originating or other oil & gas systems.

A leading supplier of oil & gas systems has successfully incorporated ETL PPDM Data Connector technology. Thanks to ETL, it can now help customers migrate their data to a PPDM-based system and allow data from numerous oil & gas systems to be collated in a single, PPDM-standardised repository. The PPDM Data Connector also allows oil & gas systems to migrate data unobtrusively – creating efficiencies for the business in terms of time, productivity and resources.

To discover how the ETL PPDM Data Connector can benefit your business, email info@etlsolutions.com or call 0203 427 5726.

Meeting the Challenges of Migrating E&P Data

ej@etlsolutions.com (Editor) — Fri, 21 Oct 2011 00:00:00 GMT

Data migration challenges don’t come any bigger than in the oil, gas and petroleum industry. By its very nature, the type of data that is collected in the exploration and production (E&P) of oil and gas is geographically vast and operationally complex.

Just consider the variety of applications in use at oil fields, rigs, wells and laboratories, and you realise the breadth and depth of data that is collated within the E&P environment. How oil and gas companies manage their subsurface data is crucial to their operations, as it can give insights into how to improve productivity and increase competitiveness.

The challenge is that there are so many sources of data and so many variables to be taken into account. Oil & Gas data is typically managed by proprietary systems that could be different depending on which country they’re in, and even which department of the business they’re in.

Like many other industries, the Oil & Gas sector is looking to bring its disparate data sources into a central Master Data store, typified by the PPDM data model, to standardise, professionalise and simplify the management and analysis of its data. This is a tremendous data migration challenge.

Tools are available to help migrate applications, but they need source code access to produce the best results, which is an unlikely position for an oil company to be in. Such tools work better with business-oriented languages such as COBOL, rather than general purpose programming languages like C, Java or C#, which can be difficult for a software tool to analyse.

That’s why many companies operating in Oil & Gas turn to sophisticated middleware solutions to provide essential continuity for the business during and after the implementation of their Master Data Management solution. Regardless of the physical location and representation of data, the middleware – a way of allowing different hardware and software applications to work together – ensures that each application sees what it wants to, when it needs to.

A large chunk of the middleware solution is the data integration technology required to achieve this transparently, which is where ETL Solutions’ tools – such as Transformation Manager – and expertise have a beneficial role to play. We havea strong track record of working with Oil & Gas middleware providers, as well as in-house teams, to ensure that the transformation and migration of data works swiftly and seamlessly.

To discover how we can help you meet the challenges of Oil & Gas data migration, email info@etlsolutions.com or call 0203 427 5726.

Why You Need a Data Security Plan When Migrating

ej@etlsolutions.com (Editor) — Mon, 10 Oct 2011 00:00:00 GMT

Data security has been a much hotter issue over the last 12 months, given the number of high-profile data losses that have been reported. When carrying out data migration, you will have access to corporate data that is likely to be sensitive and business critical, which is why security cannot be left to chance.

The consequences of insufficient data security are huge. Get it wrong, and inadvertently allow data to leak or get lost, and there could be significant damage to company reputation, perhaps leading to loss of customers, and even provoking potential legal action.

It is critical in all data migrations that data should be treated with respect. All sensitive information, including common data such as customer information, should have detailed levels of security in place. Before you start any data migration, check exactly what levels are in place, and who is allowed access to the data and when.

Assess the value of the data to the business as well as the costs that could arise from the consequences of a security breach – and make sure that security requirements within the migration reflect this value. The security requirements still have to be cost-effective and not outweigh the risks highlighted in the assessment.

Check your legal obligations. Statutory measures covering data breach and data protection are now in place within many sectors, outlining the areas of security that have be in place, as well as stipulating operating procedures to keep the data secure. You have to adhere to legislative requirements during the data migration, while also taking into account any wider implications.

Don’t leave data security to someone else. Draw up data security plans early on, and embed them in the design of the solution. These security requirements can include:

Secure data transfer.
Secure server access.
Secure data access.
Increasing the number of permissions required to transfer data.
Identifying the personnel who have or need the appropriate clearance and vetting within the companies involved in the migration.
Stipulating the security courses or information sessions that personnel have to attend.
Vetting and installing the software that will be used for the migration.

For confidential data within larger organisations, this is mandated by security departments, which limit access to servers, and define which servers’ data can be transferred between. However, you should apply the same level of care, no matter what level the migration is operating at.

The most common cause of a security breach during data migration is when a number of parties are involved. They may have the ability to transfer data between parties and between themselves using email or portable storage devices. Portable devices can easily be misplaced, while emails can be intercepted.

Having a robust plan in place will prevent that from happening – and spare the company’s blushes.

For help putting together a data security plan, email js@etlsolutions.com or call 0203 427 5726.

Minimise Data Loss in Complex Data Migration

ej@etlsolutions.com (Editor) — Mon, 3 Oct 2011 00:00:00 GMT

The more complex the data migration project, the greater the margin for error. That’s why it helps to follow best practice to reduce the likelihood of data getting lost or corrupted during the migration, and to minimise the potential impact on the business.

Here are our tips for helping to reduce data loss in complex data migration projects:

1. Define the data that is required for the migration.

a) Do not migrate data that is no longer required.

i. Fields required in the source system may be superfluous to business needs and benefits.

ii. There may be fields of inaccurate data that are mandated but not required, and therefore do not need to be migrated. These fields are often filled with “.” or other irrelevant characters that were input to satisfy source system constraints when users entered the data.

b) Do not migrate data that is out of date, as this may affect marketing campaigns or statistical analysis within the new system.

c) Do migrate data to give business advantage – but make sure you define beforehand which data is business critical.

2. Use a tool to profile the data so you have a full picture of the current quality of the data, and where the gaps may be.

3. Cleanse the data where required. You can do this manually before the data migration starts, or you can do down the automated route prior or during migration.

4. Define the quality rules the data has to abide by early on in the process to give a robust framework for the migration. You can define these rules from system analysis, business analysis, and via gap analysis between the source and target systems.

5. Get sign-off on the rules. The complete set of quality rules needs to be reviewed and approved by the domain and system experts for both the source and target systems before you begin the data migration.

6. Verify the data against the rules. It is best practice in complex data migration projects to make this an ongoing process, where possible. The rules should continue to be defined, so you can maintain a constant view of the data and ensure that its quality is fit for purpose. That way, you can raise any issues early on, mitigate risks, and lower the chance of losing data during the migration.

7. Define a clear flow for data during migration, error reporting and rerun procedures.

8. Use a tool for data migration to professionalise and automate the process, thus further reducing the risk of error and data loss.

For more tips on best practice in helping to minimise data loss, email js@etlsolutions.com or call 0203 427 5726.

Tips for Data Migration After a Merger

ej@etlsolutions.com (Editor) — Thu, 22 Sep 2011 00:00:00 GMT

Migrating data after a merger or acquisition (M&A) is a major challenge for the companies concerned. It is business critical that all the data is consolidated and managed correctly and expertly to create a workable, sustainable structure for the newly merged company, and so the information it has at its fingertips is streamlined, efficient and authoritative.

Plus, the merged company has many other issues to contend with, such as managing the change among the teams, maintaining customer service levels, and updating its brand positioning. Most importantly, it will want to show that the deal has a return on investment and is in a stronger commercial position.

There are so many data factors to consider after an M&A: the two systems being merged may be completely incompatible; the quality of data may be inconsistent, problematic and out of date; data could be stored in multiple locations; and there may be data compliance issues.

Here’s some steps we recommend taking when migrating data after an M&A to help mitigate those issues:

Identify the stakeholders. This is important in any data migration, but especially so after a merger, when some key figures may still be jostling for position, and new roles may not be clearly marked out. It’s vital to ensure buy-in from the relevant parties, and to define who is responsible for signing off the various stages of the project.

Call in the experts. IT departments can easily be overwhelmed after an M&A, with so many systems to merge and manage. Define within the IT department who will carry out which task, to avoid duplication of effort – and to prevent anything slipping through the net – and call in experts if the necessary skills do not reside within the IT department. Timing is crucial after an M&A, and the company needs to be in good shape inside and out, so don’t risk overruns through lack of expertise.

Locate where the data is . The companies may have stored their data in many different ways and in many different places. Doing an inventory of what is stored where will map out the territory and make it easier to prioritise what needs to go where and when.

Profile the data. Make sure you understand the data: is it usable and valuable, or can it be retired? Identifying data quality issues early on gives an overview of what you’re dealing with, what resources will be required, and what costs will be involved. It will also save time later on if you know exactly what you’re dealing with.

Cleanse and convert the data. Standardise the data so it is workable and relevant. Remove what is out of date, what is duplicated and what is no longer needed, and cleanse any data errors from both systems – so that the only data to be migrated is useful, current and compatible within the new business.

Use appropriate and sophisticated tools. Tried and tested tools and methodologies will take the headache away, will speed up the process, and will give a more satisfactory outcome. Our integration software technology Transformation Manager is a proven solution for handling complex data migration projects such as an M&A, and help you convert and transform data so it can be delivered in the right format to the right place and in the right way.

Test throughout . Set up tests to alert any issues at all stages of the data migration, so these can be dealt with in a timely manner – and can provide reassurance to company management.

For expert help with your M&A data migration, or for more information on Transformation Manager, email js@etlsolutions.com or call 0203 427 5726

10 tips for data migration best practice

ej@etlsolutions.com (Editor) — Fri, 16 Sep 2011 00:00:00 GMT

Data migration involves so much more than just moving data from one storage place to another. A successful migration may be business critical, so it should not be treated as routine or an afterthought, and instead should be subject to best practice tools and techniques.

Here’s why:

It’s costly for data migration projects to go wrong. Bloor Research 2011 says that 38% of data migration projects fail, so there’s a strong financial reason to make sure they go right.

It’s interconnected with many other key areas of IT. You may have new software or hardware, a new data centre, or perhaps you are consolidating systems after an acquisition. Data migration can be a drain on IT departments if it is not planned properly and resources are not allocated upfront.

Cutting corners doesn’t save time or money. In many cases, it can increase the cost and effort because you have to go back and do it again using the appropriate tools and expertise.

Here are our 10 recommendations for data migration best practice:

Get buy-in from the business early. Bloor Research 2011 says that business engagement is the most critical success factor.
Root out any issues early with an initial analysis of the business requirements and the underlying data.
Use this analysis to pin down the costs, resources and timing involved to complete the project.
Use a tried-and-tested methodology to ensure the migration flows smoothly and any issues are identified so you can address them straight away. The methodology should also help ensure that system retirement plans are considered early on in the migration process.
Use tools for profiling and to identify areas of weakness within the data.
Use tools to handle complex migrations. This saves time and cost on having to debug handwritten scripts, and makes it easier to update transformations if and when changes are required.
Ensure skilled migration experts are used where and when required. Having this specific experience and expertise on board will reduce the time it takes to implement the migration; reduce the risk of major issues having an impact on the business; and increase flexibility within the project.
Run through project management thoroughly. This is especially important when the migration project involves many parties. There is a common tendency to create all the start-up documentation for projects precisely and thoroughly but as the project progresses the quality of documentation control and communication can diminish. Aim for consistency throughout by checking everything upfront.
Documents should be controlled and updated. All key documents generated within the migration should be agreed upfront, with a schedule of dates for completion and sign-off from all parties involved.
Communication of information between parties and to project managers should be regular, and must be managed and exchanged in line with a well-executed project management methodology.

For guidance on best practice for your data migration project, email js@etlsolutions.com or call 0203 427 5726.

How Companies Can Make the Most of their Data

ej@etlsolutions.com (Editor) — Thu, 15 Sep 2011 00:00:00 GMT

If your data is more of a burden than a benefit to your business, you may be wondering how you can make it work for you rather than against you.

Companies with clean, efficient data that is consolidated and structured to achieve corporate aims are more likely to be able to anticipate and respond to market needs, as they have business-critical information at their fingertips.

From our experience working with major organisations across the globe, we know that out-of-date, disparate data on incompatible systems only ever leads to operational inefficiencies, and can put the business at a competitive disadvantage.

Of course, data consolidation and migration are never carried out as an end in themselves. There are always strategic and commercial motives behind any data migration project, but we’ve found that our solutions at ETL have been used in some of the most demanding situations and with the toughest of data challenges to resolve – but our tools and techniques have assisted businesses in becoming much more effective and efficient.

Here are some examples of how we’ve helped companies make the most of their data:

We worked with an automotive company to integrate disparate data across its dealer network. All the dealers used different Dealer Management Systems with their own design and rules, which meant the systems couldn’t communicate with each other, and data couldn’t be moved without the risk of huge error. Using DataHub, we transferred data from dealers to a dedicated server at ETL, which was verified and consolidated and then analysed by the automotive company to assess stock levels and automatically order new parts to meet demand –hence improving employee communication and customer satisfaction.

ETL’s Tranformation Manager tool was key to migrating data from a range of systems for a global manufacturer. The company inherited many legacy systems after a merger – and data incompatibility was causing operational issues. We worked with the organisation to validate and standardise its data, and to migrate it from the incompatible systems into a single database – which immediately made the business much more efficient.

Data that was incomplete and in a multitude of formats was the challenge facing a heavy equipment manufacturer when it was re-engineering its supply chain management process. As often happens in these cases, the lack of clean, standardised data risked creating project overrun, with all the associated headaches and costs. ETL’s Transformation Manager consolidated the data from the many sources into a single format, allowing for consistent reliable data that eased the implementation of the new supply chain platform.

Check out our Case Studies for more detail on how we’ve helped businesses operate more efficiently, or contact us on js@etlsolutions.com or 0203 427 5726 to discuss how we can help you make the most of your data.

How to Ensure Your Data is Fit for Purpose

ej@etlsolutions.com (Editor) — Mon, 5 Sep 2011 00:00:00 GMT

Your business will only ever be as good as your data. Old, dirty, irrelevant data will not only affect the efficiency of your business, but it will be a drain on time and resources. Plus, if your data is not kept clean and up-to-date, the process of cleansing and formatting will add to the cost and effort of any data consolidation or migration project you need to carry out.

Here’s what we at ETL Solutions regard as best-practice steps you can take to keep your data fit for purpose and primed for migration:

1. Review all data in your current business landscape. It’s helpful to draft a map of the landscape so you can identify the areas that need work. This process involves profiling and analysing your data, and reviewing any overlaps or any gaps that may exist. This will help you define the quality and quantity of data that is required by the business and what needs to be achieved in the data migration project.

2. Cleanse your data. Ongoing cleansing can be part of your master database management (MDM) strategy – that is certainly best practice – but you can also carry out cleansing prior or during data migration. If you do it before migration, you may need staff to go through large volumes of data and adjust it manually, or you can automate the process, depending on preferences and how your systems are designed.

3. Standardise the data. One of the main outcomes of cleansing and reviewing your data is to provide a list of rules that define the level of quality necessary for the data to provide benefits to the business. A common initial aim in data migration is to baseline the data and have a standard set that is fit for migration which satisfies the requirements of the target system, as well as meeting business goals. In short, you need to make sure you’ve got the right data for the desired business outcome.

4. Brief the bosses about data quality. If you know in advance that there are any data quality issues, then advise the implementation partners right from the start. Scoping the migration project realistically is key to ensuring the migration is successful and resourced appropriately. Carrying out an initial profiling task upfront can help you define cost and resource, and will ensure that you put a more accurate migration plan in place early. By knowing exactly what you’re dealing with, and offering a workable and practical solution, it will be easier to get buy-in from the bosses for the data migration plan and methodology.

5. Ensure your data is correctly formatted. Data coming from various source systems may have information formatted in a number of differing ways depending on usage, the age of the system, or simply the way the original system was designed. Using a tool to carry out the data migration means these format issues can be resolved automatically. The tool will be able to load differing file formats and check the files are structurally valid, and should also be able to handle common data formats automatically, while providing mechanisms to handle bespoke format variations as required.

6. Carry out system and business analysis of the source and target systems. This process will show how the systems run at the business level (for example, you may need customer data with full address field to start a marketing campaign) and what they need to run at the system level (for example, the postcode may be a mandatory field). It will reveal any overlap between the source systems, as well as any gaps between the source and target systems. It may come to light that there are a number of potential sources for a single piece of data. There may also be areas of the target system that have no obvious source for the data. Carrying out system and business analysis will give you a full picture – thereby helping to allocate sufficient resources upfront and reducing the risk of project overrun.

For expert insight on keeping your data fit for purpose, call 0203 427 5726 or email js@etlsolutions.com

Reducing Business Downtime During Data Migration

ej@etlsolutions.com (Editor) — Thu, 25 Aug 2011 00:00:00 GMT

If a company needs to migrate its data then in most cases it will be because it wants the business to be more efficient and more effective. The business can’t grind to a halt while the data is migrated from one system to another. However, by building in several best-practice stages to the project, you can ensure that business downtime is reduced.

Here are four key ways to ensuring your data migration project will have as minimal disruption as possible on the business:

Regression Testing : Carry out thorough testing during all migration stages to ensure you stick to the original requirements and keep the project on track. It is best practice to use regression testing, which will uncover new errors (regressions) and will make sure that during the stages of the migration you haven’t introduced any new faults elsewhere in the system. Having a set of regression tests that can be run regularly as the project develops means that updates and changes made later in the project will not have a negative effect on previous migration work you have developed. Add in new regression tests to the set as additional work is completed.

Parallel Go Live: Migrating the data to the new system while continuing with the old system is a good way of reducing the risk when the new system goes live. Basically, you keep them both going until the business is satisfied that it is safe to switch off the old system. However, by choosing this option, the information will require maintenance in both systems for a period of time until the new system is signed off as fully functional. The time period for sign-off will vary depending on how exhaustively you have tested the new system. Hence the importance of rigorous testing at every stage of the project.

Staged Go Live: This is a more complex way of approaching a data migration project, but you can carry out migrations stage by stage, based around the structure of the business. There may be clear divisions within a company defined by geographic region or by business function, for example. Staged go live requires careful planning, as it can be difficult to run different parts of the company from different systems – especially as there may be many updates to information between the migration stages. If the business divisions are not clearly defined enough to maintain the separation over the staged period, then the extra work required to maintain the data during the timeframe of the project may outweigh the benefits of choosing this approach.

Minimise Target Data Errors : Put a strategy in place that identifies data issues early on, so you can resolve those data issues either within the original system or during the migration. That way, you will be able to allocate the appropriate time and resources upfront rather than backpedalling to fix the issue. Strategies that can help you reduce target errors include regression tests, as mentioned above, and dry runs. Dry runs test the system as if it were going live, by helping you verify that the migration process has met its goals, and the migrated data meets the standard required. This is an excellent ‘belt and braces’ approach that gives you peace of mind – and ensures that when the new system does go live, it will be fully functional and will deliver the anticipated business benefits.

For a tailored plan to reduce business downtime during your data migration, call 0203 427 5726 or email js@etlsolutions.com

Tips for Keeping Data Migration to Budget

ej@etlsolutions.com (Editor) — Thu, 18 Aug 2011 00:00:00 GMT

If a data migration project isn’t completed on time or on spec, it’s highly likely that it will have exceeded its budget too. Any element of a project that changes, that runs over time, needs extra resources, or has been scoped incorrectly can end up putting a strain on the costs – and risks the entire project failing.

Here are our tips for helping to keep your data migration project to budget:

Plan thoroughly and use best practice to root out project risks, identify requirements, and establish the unknowns.

Ensure the migration is set up as a project with its own parameters, timelines, and responsibilities marked out – rather than allowing it be an add-on to the responsibilities of the IT department. Crucially, the project needs to have stakeholder involvement, and be approved and supported by senior management, with agreed sign-off points along the way.

Understand the complexity of the data: The way organisations do business and store their data is complex, and often this is unclear until further down the project timeline. Poor data quality can disrupt and delay the load phase as records are rejected, for example – and functions and services will be broken when the target system comes online. Identifying any quality issues upfront means you can address them and save many budgetary headaches later.

Define the budget realistically: Shine a strong light in all areas of the project and factor in costs for all eventualities before, during and after the project. You don’t want to get tripped up by an unforeseen cost part way through, so it’s better to have those budget decisions discussed and made with project leaders before pressing go.

Put the right people in place: Data migration requires experience expertise in key tools such as data profiling and data quality. These are not commonplace, and often IT departments lack the specific skills to carry through a data migration project. If you need to bring in outside help, then factor in those costs. Make sure you have a data migration expert running the project. Account for staff training before the project starts or on the job, for consultants you need to bring in, as well as ensuring that product licences are in place for people to work on the project. Putting a strain on IT’s resources can cause internal disruption and may incur further costs you hadn’t anticipated. So, plan out all these factors in advance.

Follow a standard methodology to define the project’s stages. Allow all stages to have:

+ Definition of hardware and software

+ Definition of activities

+ Definition of artefacts to be produced

+ Definition of Quality

+ Definition of signoff required

+ Definition of resource required

+ Definition of time frame for stage

= Budget for stage

+ Budget for stage

+ All other stage budgets

= Total Budget

Following the methodology should help you to highlight risks early. It will also allow you to put resolutions in place which may not be as costly and can be done programmatically as part of the development, instead of drafting in many poorly trained personnel at the last minute to try and correct something manually.

To calculate how ETL’s methodology can keep your project to budget, call 0203 427 5726 or email js@etlsolutions.com

Is Your Data Keeping You Ahead of the Game?

ej@etlsolutions.com (Editor) — Thu, 11 Aug 2011 00:00:00 GMT

Your data is a potential goldmine: if you know how to tap into it, understand it, and use it to anticipate your customers’ needs, it’s a valuable path to profitability. However, if your data is in different places and in different formats, and it doesn’t give you an overall picture, then it’s not going to be working as hard for you as it should – and it could leave you trailing behind your competitors.

At ETL, we help companies extract their data from disparate sources across the business – even when the different systems don’t ‘talk’ to each other. We consolidate those extracts of data into a centralised Predictive Marketing database that makes it far easier to analyse what’s going on in your business – and we produce reports for you based on that data. Having up-to-date, quality data on your customer base helps you pinpoint how and when you can build revenues from your existing and potential customers.

Here are the benefits to your business of our Customer Predictive Marketing service:

Attract new customers: You can identify the customers and prospects most likely to buy your product, which can keep your sales people focused on the hottest leads – thus driving efficiency in your business by attracting new customers cost effectively. Having an up-to-date profile of your customer base means you’re not stabbing in the dark, hoping that new business will walk through your doors. You’ll know exactly who to approach, and how.

Personalise your marketing messages: You’ll have a clearer idea of the market segmentation of your audience, helping you to customise the messages you send to individual customers and prospects. By tailoring your message and the way you send it – for example, by letter, phone, email or text – you are far more likely to increase response and revenues.

Retain a higher percentage of customers: By anticipating your customers’ needs, and making suggestions to them just when they need to buy, will help you provide more attentive customer service. Your customers will feel looked after and treated as individuals – making them more likely to buy from you again and again.

Optimise your marketing programmes: You’ll be able to track accurately track and analyse the results of your marketing programmes because the data will show you just how successful the initiatives have been – making it far easier to measure return on investment, and helping you identify where to focus your efforts next time.

Be more competitive: Having an in-depth knowledge of your customers’ transactions will help you to manage supply and demand; identify which items are selling well or under-performing; and helping you to manage your sales pipeline. That way, you can competitive advantage by ensuring you have products ready to meet future demand.

For a sample Predictive Marketing report, call 0203 427 5726 or email js@etlsolutions.com

Is Your Data Migration Project Team up to Scratch?

ej@etlsolutions.com (Editor) — Wed, 3 Aug 2011 00:00:00 GMT

“You’re only as good as the people you hire” is a famous quote from the founder of an even more famous company – McDonald’s – and that principle applies to whichever sector you’re in, whether you’re consumer facing or business-to-business. Customers demand best-in-class service, and it all relies on the quality, competence and commitment of your people to meet those expectations.

In data migration terms, of course there are robust methodologies and top-notch tools that are used to carry out the practicalities of migrating the data. But before the process can begin, there’s the project scoping to be done, a plan to be put in place, and people assigned to implement and manage tasks along the way.

The issue with migrating critical business or technical data is that it’s not an everyday activity for most organisations. It can crop up as part of a larger business programme, and can often be treated as an afterthought or overlooked. Most importantly, because it’s not an issue the teams deal with every day, employees may lack the skills and knowledge of how to migrate data smoothly, efficiently and effectively – which can contribute to the 60% of data migration projects that overrun or fail.

To check that you have the appropriate people in place to carry out the project, and to identify gaps in your personnel capabilities, ETL can carry out a staff competency assessment for you. We are briefed on the project aims and scope, and outline the roles required to carry out each stage of the migration. We identify suitable staff to interview for those roles, and assess whether they match up to the required profiles. The profiles are shown on charts, mapped out against the current level of skills and competence of your team. That way, we can assess current strengths and weaknesses and recommend appropriate action, whether that’s boosting skills through training or mentoring, recruiting new project staff, or reallocating resources from different parts of the organisation.

The benefits of our Data Migration Competency Framework are that it helps you:

• Reduce your reliance on external experts.

• Ensure the project team is confident and competent to complete the project.

• Identify learning needs early on, so they can be rectified before you press go on the project.

• Clarify roles and responsibilities, so all staff involved know where their boundaries are and where they overlap with others.

• Resolve issues promptly, helping to mitigate the risk of issues occurring midway through the project that could affect the timing and costs.

• Apply best practice within a robust framework that gives you confidence and control throughout the project.

To assess whether your project team is up to scratch, call 0203 427 5726 or email js@etlsolutions.com.

The Ten Reasons Why Data Migration Projects Fail –

ej@etlsolutions.com (Editor) — Fri, 29 Jul 2011 00:00:00 GMT

The 60% failure rate for data migration projects is a stark reminder of how much time, effort and resource are needlessly wasted on such a business-critical issue.

Yet organisations are struggling to manage the explosion in the volume of business data, which is growing at an estimated rate of 60% every year. Current and historical data is extremely important, and needs special attention when new systems are introduced, for example, or when data needs to be amalgamated following a merger or acquisition.

Typically, migration projects don’t come up every day. However, getting it wrong can have a catastrophic and expensive impact on the business, because once an old system is retired there is no going back.

Here are ten factors that can cause risk, delay or failure at any stage of the project – and how you can mitigate those risks:

1. Ownership by the business

Unless the project is owned by senior management, and has significant engagement with stakeholders, then projects are often doomed to failure from the start. Get buy-in from the bosses, and ensure they sign off at various stages of the project to help keep it on track.

2. A project in its own right

The problem occurs when a data migration project is seen as tactical rather than strategic, or is considered to be solely an IT project. An unsuccessful migration has the ability to destroy the credibility of the strategic system that is being implemented, and should therefore be given the time and resources that it needs.

3. Scoping

So many projects fail because they go over budget or timescales. The lack of planning and use of best practice to root out project risks and requirements can trip up so many organisations. Correct scoping of the project helps to correctly to understand and plan resource and time requirements – and can reduce the risk of unplanned or unforeseen issues.

4. Complexity of data

It is often underestimated just how complex the client data is, particularly if it is technical data from disparate systems. It’s vital to identify those complexities from the start, and ensure that expertise – either internal or external –is brought in where and when required.

5. Quality of data

Data quality has to be addressed early on, otherwise there’s a risk that the data won’t be fit for purpose in the new system. Some businesses may not have checked their data quality for years. Yet the lack of quality data can disrupt and delay the load phase as records are rejected –and functions and services will be broken when the target system comes online. Profiling the data, showing how good or bad it is, also defines how long some of the next steps in the process will take.

6. People

This is a basic one, but often overlooked. Ensure all parties needed for the project are available, briefed and – most importantly – trained in the tools they’ll be required to use. People need to be managed so they have clear boundaries and work well together.

7. Expertise

Data migration requires expert experience and skill in key tools such as data profiling and data quality, and these are not commonplace. If an IT department finds itself having to ‘make do and mend’, this can be the undoing of a project.

8. Permissions

Do you have the right product licences? How about the passwords and access to systems across the business? Delays in getting the necessary permissions can all too easily disrupt a project.

9. Movement of data

How quickly can you move the data into the target system to see how it fits, and to assess its usability for the new system. In a new target system there will usually be many more demands placed upon the data, so it’s better to identify any issues as early as possible.

10. Best practice

Opting for best practice tools and techniques is sometimes overlooked in favour of cost savings. However, unless best practice is followed, further down the line those savings may be wiped out if business and technical issues arise because the correct approach was not applied in the first place.

Data Consolidation Can Benefit Your Business

ej@etlsolutions.com (Editor) — Thu, 14 Jul 2011 00:00:00 GMT

The Challenge

Running an efficient and profitable business relies on having a 360° view of the different parts of the organisation. Having that complete picture enables you to make decisions that are effective, timely and profitable.

That is especially true in the current economic climate, where businesses are seeking ways to become more competitive, efficient, and able to anticipate and meet client needs. Businesses with a holistic view of their departments, suppliers and customers, and how those different areas interact with each other, are in a stronger position to make more successful decisions.

All aspects of the business – including physical assets such as warehouses, offices and intellectual assets or business knowledge – produce and require information. The ability to provide the best service or product depends on the flow, control and visibility of this information. Problems tend to occur when parts of the business use different software applications or methods of storing information. This can make the process of accessing and analysing data extremely difficult and costly.

How ETL Can Help

A key part of ETL’s expertise is data consolidation. We know where the data is stored, what format it is in, and how to extract it from multiple formats and locations. We’ve been solving data integration problems for years, and we’ve developed some best-in-class tools to help us do so.

Our data migration product, Transformation Manager, extracts the data from different formats. DataHub accesses disparate sources of data via the internet, then processes it into a common format and loads it into a target data store. This is all done in a secure, encrypted way, so data is safe at all times, and can be stored either at the ETL secure data centre, or in your own. Either way, your data is centralised and visible all in one place.

We can then interrogate this data for you to use for reports or services such as customer retention or predictive marketing. Alternatively, it can be passed to your in-house team or service provider to interrogate and analyse.

The Benefits to Your Business

Management information at your fingertips.
Ability to make more informed and effective decisions.
Streamlining your data saves you time and effort.
Having data all in one place improves productivity and efficiency.
Fast and agile decision-making.
More responsive to clients’ needs.
You can communicate with customers in targeted and meaningful campaigns.
Helps improve customer retention and increase revenue.
You can improve the quality of your service, helping you to keep competitive in your market.

To discover how consolidating data will improve your business performance, call 0203 427 5726 or email ph@etlsolutions.com

Engage Senior Business Leaders in Data Migration

ej@etlsolutions.com (Editor) — Thu, 7 Jul 2011 00:00:00 GMT

There’s no doubt that having the backing of senior leaders in the business will improve the chances of a data migration (DM) project going smoothly. Not having their support can risk it heading in the direction of those six in ten DM projects that fail.

While it’s essential to have the bosses on side, the purpose behind the migration of their data is to make their lives and their businesses more efficient and more effective. The onus is therefore on the data migration partner to make the process as straightforward and timely as possible.

Here’s how we recommend engaging with senior business leaders so they will sign off and endorse the project:

Keep Your Eye on Their Prize

The project needs to be driven by areas where business leaders apply focus: these are predominately ‘improve revenue’ and/or ‘reduce cost’. They know that to provide good-quality data migration will cost more time and resources than they want to spend, so unless they can be convinced that there is real gain they will cut what they see as a superfluous cost.

Communicate in Business Terms Rather Than Data Terms

Think about the big picture and how the project fits into that. Draw their attention to the links between how their investment in improving data will also boost their business results. Don’t blind the bosses with the science of data migration. Talk in terminology they understand and translate it into how the business will benefit.

Demonstrate the Benefits

Educate the business leaders by demonstrating how current issues they are facing will benefit by the data migration. Reassure them that by following a best practice methodology, the migration will not only satisfy the data requirements for the new system but also the business requirements for the new data within the system.

Manage Expectations Throughout the Entire Project

Be honest about how long the project is going to take and what involvement will need to have along the way. Make it easier for them to delegate by ensuring regular communication and sign-off stages along the way. That way, leaders are in control of the project without having to micro-manage the detail of it.

Build in Short- and Long-Term Gains

Senior business leaders are likely to want to see short-term value added to their bottom line after making a major investment in data migration. Often, data migration projects reveal their value over time, so make sure you build in some quick wins to satisfy senior management objectives.

Promote Best Practice

Show in the scoping and strategy document/s at the outset how close to best practice they are aligning themselves, in terms of areas covered and personnel required. While many projects follow all the guidelines in the initiation, best practice can have a tendency to fade away into a more ‘get it done’ mentality as the project progresses. Of course, that’s only until things start to overrun, when best practice may well reappear – so it really is better to follow best practice all the way through.

For a best practice assessment of your next data migration project, call 01248 675070 or email ph@etlsolutions.com

Ten Ways to Keep a Data Migration Project to Time

ej@etlsolutions.com (Editor) — Thu, 30 Jun 2011 00:00:00 GMT

Time is money, as we all know in business – especially when you’re managing a data migration project that requires the input, co-operation and alignment of different departments and offices within an organisation.

You may be all too familiar with the statistic that six out of 10 data migration projects fail – and that’s often down to timing issues. Keeping projects running tightly and smoothly is why our clients hire us. Here’s our nine-point plan for keeping things on schedule:

Get buy-in early from the business so that everyone is behind the project, knows what the process and outcome will be, and what the business benefits are. This will help the migration group to work as a team.
Know the scale of the task upfront so you can allocate the appropriate times, skills and resources to it. That’s why following a tried-and-tested methodology can highlight all the areas you need to address before pressing GO on the project.
Agree boundaries and responsibilities. If company divisions are spread out, or if there are third parties involved, then define who is doing what, where, when and how – and get their agreement before you start. This will prompt line managers to chase up where necessary, and ensure everything goes smoothly.
Ensure people are available for the duration of the project. If people from different departments are helping out with the project, then get sign-off from their line managers. Build in a training schedule if you need to bring skills up to speed, and make sure that key personnel are contactable to answer any queries.
Sort out security restrictions Everyone working on the project must be checked and verified with the relevant permissions and access to data. This is an often overlooked point – but we’ve seen projects overrun by up to six months because of red tape.
Use a tool to save time. Hand-coded solutions can be time-consuming and complex, and mistakes can creep in. When you have tools involved, dealing with changes or revisions makes the process much speedier.
Train people in the tools so that they are familiar with all the functions and capabilities, so that their lack of knowledge and dexterity doesn’t hold up the project.
Plan a retirement home for historical data: there are legal and corporate governance requirements to keep data for a certain number of years, so make sure you have a retirement plan before you start the project. Again, this will show up in the methodology, but this is a time-consuming aspect often overlooked by companies doing data migration projects for the first time.
Maintain visibility: build in sign-off stages at different points during the project. That way, management can be alert to any issues that have arisen and will be in a position to resolve them promptly.
Discover the tenth way to keep your project to time by calling 01248 675070 or emailing ph@etlsolutions.com

How to Use Data to Drive Customer Retention

ej@etlsolutions.com (Editor) — Fri, 24 Jun 2011 00:00:00 GMT

If you’re running a car dealership, the relationship with your customers shouldn’t end when they drive off the forecourt in their shiny new motor; it should be the start of a long and profitable partnership.

The problem is that the longevity and profitability of that relationship depends, crucially, on the data you hold about those customers. You can’t contact them with exciting offers and updates on new models if you don’t have their contact details, service history, and a good idea of the kind of vehicle they drive.

And you need to have that business-critical data to hand if you’re going to keep them coming back to you – knowing that there is usually more profit to be made from aftermarket sales than there is from the initial sale of the car.

Typically, automotive customer data is stored in disparate Dealer Management Systems (DMS) that are often not connected, are in different locations, and require different methods to extract the data.

That’s where ETL can step in to help make the journey to retain automotive customers a smooth and rewarding one.

With 10 years’ experience of working with some of the world’s leading automotive brands, we know exactly where to look under the bonnet of your DMS. We can extract and consolidate the exact data you need – including sales, service and invoice information – and produce detailed analyses and reports.

Not only that, but ETL can improve data quality by identifying data that is missing or wrong at the point of capture – and help improve dealer performance and effectiveness in customer retention.

This consolidated, clean, up-to-date data empowers dealers with the information they need to manage the customer relationship and help them run targeted and effective customer-retention activities and proactive marketing campaigns – whether you want to approach customers with a reminder about the due date for their service or MOT, with an offer of a summer or winter check, advance notice of new models in the pipeline, or even to send valued customers a birthday card.

Data that has been consolidated in this way becomes a valuable and practical way of promoting the business and keeping the customer in your dealership and your value chain, so you can optimise the profit from the customer.

This dexterity with your data will help drive your customers back to the dealership time and again, and keep your business motoring ahead.

For a sample retention report, or to discuss your specific data requirements, call +44 (0)203 427 5726 or email ph@etlsolutions.com

How to Choose the Right Data Migration Partner

ej@etlsolutions.com (Editor) — Thu, 16 Jun 2011 00:00:00 GMT

Data migration is a specialist activity that demands specialist support – especially if the project involves tough or complex data. If you don’t possess the necessary skills or tools in-house, then bringing in external experts becomes a business-critical necessity if you want to avoid your project becoming one of the six in 10 that fails.

The challenge is: how do you choose the right data migration partner for the scale and scope of your project? Here are eight areas to consider:

Standards

If you’re looking for someone to carry out data migration for you, then you need a partner who knows which rocks to look under. They will be able to identify problem areas early on, instead of reaching the final stages with 100 different issues to sort out.

A partner that follows a set of standards will be able to identify issues at the earliest possible opportunity to mitigate the risks. AT ETL, we have Prince2 management standard, and use ISO Standards where appropriate to underline our data migration methodology.

Project scoping

If you’re not sure what you need to do and how long it will take, or if you haven’t done data migration before and have been drafted in for the project, then you would benefit from a scoping exercise to draw up a reasonable plan that sets out areas such as people, price, timings and the tools you will need to complete the project.

Methodology

This is essential if you want a staged, well-managed and robust approach to data migration. ETL has a tried-and-tested, six-step migration methodology that covers the key stages of a data migration project, from initial set-up (ensuring you have the right people and equipment in place) through to data assessment, core migration and legacy decommissioning.

Tools

Are the tools used in the data migration included in the cost, or will you have to pay extra? How flexible are the tools, and how widely used are they. Will the tools do the job, and are the people on the project skilled in using them? We have our own tool, Transformation Manager, for data migration projects, which is constantly being updated and developed to improve its effectiveness.

People

Are the people carrying out the project there for the duration – and are they skilled and knowledgeable in the toolsets they’ll be using? ETL carries out a staff competency assessment within the project according to INCOSE standards. This charts the levels of skill needed, and produces a chart showing the personnel footprint against the role they’re taking to check they have the right skills. We also provide a project manager as the point of contact, and one single technical lead, so there is consistency and direction throughout the project.

Price

When it comes to negotiating a fee for the project, be clear about your own dataset or you could end up changing requirements and having to pay more. Ensure that boundaries are clearly defined between third- party suppliers, and responsibilities are fully understood by all parties. Be honest about the scale of the project and have a clear understanding of what your needs will be before agreeing a fixed price that covers everything. Otherwise, you risk being exposed and busting your budget.

Visibility

How much control do you want to have over the data migration process. Do you want to be updated every step of the way, and be confident knowing that the plan will be executed with precision? With ETL’s standards and methodology, we are open in what we do. The plan is set and clearly defined, so you will know exactly what we are doing and when we are doing it, and can track and monitor as much or as little as it likes. We believe trust and transparency are extremely important – and they help projects run to time, spec and budget.

Experience

Depending on the project, it is hugely beneficial to work with a data migration partner that has worked in a variety of environments and sectors, and has general knowledge and skills that can be adapted to fit the current project. With ETL, we have worked with a variety of sectors. We are unfazed by working with complex or abnormal issues, and our team take in their stride.

The importance of the above areas will vary according to the detail and complexity of the project, but these eight considerations will help you decide the best partner for your project.

Why Call in the Experts When Migrating Data

ej@etlsolutions.com (Editor) — Thu, 9 Jun 2011 00:00:00 GMT

What’s unique about data migration (DM) is that it’s not something companies have to do every day. What’s puzzling is that when they do have to migrate data for strategic, commercial or practical reasons, they often attempt to do it themselves using their in-house technical people and whatever tools they find to hand – neither of which are purpose-built for the task in hand.

It’s therefore hardly surprising that 38% of data migration projects still run over time or budget, according to a preview of a new report from Bloor Research. This is an improvement on the 84% of projects that overran when the study was last carried out four years ago. However, given this relatively poor success rate, you’d think that companies would call in the experts to manage one-off, highly specialised data migration projects – especially when the data that’s being migrated is business critical (as it usually is).

This lack of an industry-leading standard for tackling tough migration projects was one of the themes we noticed at the recent Data Migration Matters 4, the most noteworthy conference in the DM arena. It wasn’t just in the informative seminars, insightful presentations and thought-provoking panel debate that this theme emerged, but in our conversations with companies currently carrying out DM projects, as well as with integrators, project managers and tools vendors.

There is a huge gap in the market for providing all the elements to help a data migration project run smoothly (on spec, on time and on budget) that ETL Solutions can fill.

Those elements are:

A formal methodology: we have teamed up with industry guru Johny Morris, whose written, robust methodology is proven and effective, to ensure DM projects run to plan. The Bloor research suggests that there may be a causal link between the use of a formal, independent methodology and the reduction in the number of overruns.
A proven toolset: our in-house DM tool, Transformation Manager, is tried, tested and trusted with a raft of major organisations, and guarantees efficiency and reliability in complex DM projects.
External expertise: in-house teams may not have the skills or experience to run DM projects, which is why ETL offers specific expertise where required – as well as training and support where, when and as often as needed.

Where data migration matters for you, we make it matter for us.

Data Migration Matters 4

ej@etlsolutions.com (Editor) — Wed, 27 Apr 2011 00:00:00 GMT

A unique opportunity to gain insight and knowledge on best practice ideas for Data Migration projects including the use of Data Quality tools and process and their place in Data Migration.

Date: 26th May 2011

Location: The BCS London WC2

Special Discount
Data Migration Matters 4 is priced at £150.00 per delegate, however we are offering a special Data Migration and Integration Group discount. Enter code DMIG50 for 50% off the prevailing price. Places are limited, register now.

International Sponsors
With sponsors like Rever from Belgium and Pervasive and DataFlux from the USA, with speakers from the USA, mainland Europe and the UK, DMM4 will be the most international yet.

There will be the opportunity to get hands on with our vendor’s software in our dedicated seminar rooms as well as listen to a range of speakers and panel events. There will also be valuable networking opportunities.

Featured Speakers Include:

Dylan Jones
Founder of Data Migration Pro and Data Quality Pro will be speaking about the false economy of not using proven Data Migration methodologies

Johny Morris
Industry expert and author of the Practical Data Migration Methodology will be speaking about the major trends and developments in Data Migration

Ken Rosenteel
Flying in from the USA to tell us about his experience off Data Migration at the Oklahoma Employment Security Commission

Visit our website for more information.

Data Migration Tool Boosts Productivity

ej@etlsolutions.com (Editor) — Tue, 21 Dec 2010 00:00:00 GMT

Transformation Manager (TM), ETL Solutions’ data migration tool-set has improved visibility for the user, reducing the time and cost spent in identifying and resolving errors associated with migration and other types of data projects.

The new runtime log reporting function enables users to configure the information that they want to record, such as issues, warnings and the traces of data movement or transformation. These logs are now created using the industry standard ‘log4j’ which is highly configurable. This enables the logs to be integrated into existing systems and processes: for example, automatically getting it added to existing applications, or receiving an email with the details.

The runtime statistics have also better enabled users to see the detail of data movement such as how many rows of data have been read, how many have been written or moved to other databases.

These features enhance the users’ experience and enable them to identify and mitigate potential issues early in the project. The flexibility in configuring the logs and integration into existing systems saves time.

Download the full version of the release notes.

Infrastructure Upgrade for Customers

ej@etlsolutions.com (Editor) — Mon, 10 May 2010 00:00:00 GMT

ETL have expanded their server capacity in order to get new projects online quicker. This increased capacity means better performance and a more secure and robust data migration environment.

The foundations have been put in place for future expansion, as part of a series of internal changes designed to facilitate an aggressive growth strategy across the business for ETL. Other improvements include a new CRM so that customers’ communication can be tracked and responded to more efficiently.

Ian France, ETL’s Chief Architect, said, “We are looking for ways to improve performance and the service that we deliver to our customers. We take security and reliability very seriously; this upgrade has improved our service offering.”

New Data Migration Tool Improves Ease of Use

ej@etlsolutions.com (Editor) — Mon, 10 May 2010 00:00:00 GMT

ETL are always looking for ways to improve their solution and service offering; they have created a tool called Transformation Packager (TP) which creates deployment packs. ETL have worked with many organisations to successfully migrate and integrate data where there are complicated types of data sets or data from disparate environments.

ETL's ability to deploy data migrations into varied systems, ranging from enterprise class networked solutions to desktop PCs in isolated locations has always been one of their strengths. TP now makes this process even easier.

With many data migration projects just simply getting the files ready to deploy the data migration or transform can be time consuming and costly. This new tool will package the files into a pack ready to execute the data migration on any system; the migration can then be executed automatically.

This new tool has several advantages, for example, the ability to easily customise the packs if the same migration is being deployed to different environments, without the need to hand code or create the pack from scratch. New connection details can be updated using a simple GUI suitable for non-technical users.

One of Your Most Valuable Assets

ej@etlsolutions.com (Editor) — Tue, 2 Mar 2010 00:00:00 GMT

Data is one of your most valuable assets; it may only become apparent when it’s no longer available. Our data migration solutions improve productivity and save many organisations time and money. Click the play button to watch our Data Migration presentation.

Save One of Your Most Valuable Assets

ej@etlsolutions.com (Editor) — Wed, 24 Feb 2010 00:00:00 GMT

Strengthened Team Improves Service

ej@etlsolutions.com (Editor) — Wed, 10 Feb 2010 00:00:00 GMT

ETL Solutions recently added two new experienced people to their team to provide enhanced services to clients and partners.

In the Automotive team, Andrew Smith brings more than 30 years experience working with Dealer Management Systems such as ADP's Kerridge Autoline. He has practical and strategic experience of working on advanced information and automation projects within the automotive industry which will add significant value to our clients and partners.

Phillippa Hallewell joins as Marketing Executive bringing more than 16 years experience of marketing and selling new technologies across multiple industries. She will be responsible for implementing the marketing strategy to help bring new solutions to market. Working closely with partners and clients to ensure ETL focuses on continually improving the data integration services they provide.

John Bigerstaff, CEO of ETL said, "I am delighted to welcome two such experienced people to the team. I am excited about the opportunities their skills give us to focus even more on delivering what our clients and partners need to be successful.”

New Automotive Services Drives ETL Forward

ej@etlsolutions.com (Editor) — Wed, 10 Feb 2010 00:00:00 GMT

ETL’s integrated data services improve customer retention and provide businesses with a new perspective. This new service portfolio enables manufacturers (OEMs) and dealers in an increasing competitive market to get feedback, improve customer retention and increase revenue.

Working with strategic partners ETL offers solutions that can measure and improve dealer performance, create loyalty and membership programmes, manage customer relationships (CRM), enable predictive marketing, create customer satisfaction surveys (CSI) and automatically replenish parts (RIM).

In addition ETL can create customised reports from Dealer Management Systems systems such as ADP's Kerridge Autoline which enable dealers to quickly produce business critical information and therefore make informed strategic decisions. Make the most of your DMS data and maximise your sales.

Transformation Manager Goes Global

ej@etlsolutions.com (Editor) — Tue, 2 Feb 2010 00:00:00 GMT

The latest version of ETL Solutions’ data migration toolset, TM, has enhanced functionality that reduces the risk of errors when migrating complex data. The new multiple language support enables multi-lingual data to be moved from one data environment to another.

Other enhancements include greater support for Oracle databases and extended Microsoft Excel compatibility including Microsoft Excel 2007. TM’s unique ModelPacks enable users to work with the best model for their data migration scenario further reducing errors and saving time.

TM is used by one of the world's largest oil companies, a major bank, and a global aerospace manufacturer where data needs to be extracted and moved from and to multiple platforms and environments.

Download the full version of the release notes.

Hidden Complexities of Data Standards

ej@etlsolutions.com (Editor) — Wed, 27 Jan 2010 00:00:00 GMT

ETL Solutions attended the PPDM Association User Group Meeting on 28th January 2010 where Dr. Tony Eastwood shared their experiences on ‘the Hidden Complexities of Data Standards Integration’; presented issues included using PPDM for the first time and the transition between different versions of the PPDM model.

The PPDM Association is a global, not for profit standards organisation that works collaboratively with industry to create and publish data management standards for the resource industry. ETL have many years experience working with complex data integration and migration problems.

Using best practice ETL’s solutions expose many of the complexities which could otherwise be a major obstacle to the ongoing adoption of PPDM standards. ETL have used their experience to write a whitepaper about their experience as data integration and management specialists working with PPDM.

Download the ‘Experiences of PPDM Integration’ whitepaper .pdf

New TM Release

ej@etlsolutions.com (Editor) — Mon, 30 Nov 2009 00:00:00 GMT

A new version of Transformation Manager – TM Version 4.14 – has been released.

Changes include:

New and Enhanced Features

Oracle Float datatype now supported as default
Unicode characters now legal in SML strings
Programmatic Build now possible

Design Tool Changes

MakeNull() use deprecated
JRE Version installed is now Java 1.6
Maximum Cursor default number changed
New SML Generation Settings introduced

API Changes

net.etltm.TMDH modified
net.etltm.TMTransformer modified
net.etltm.TMAdapter modified

TM Dependencies

New jars included

Repository Upgrade

Changes to Text Repository Character Set introduced

To download a copy of this version contact ETL support and a URL giving a direct download will be provided.

The full Release Notes are available here.

Integrating Dealer Networks

ej@etlsolutions.com (Editor) — Tue, 29 Sep 2009 00:00:00 GMT

ETL Solutions are helping to improve customer service by integrating dealer networks...

Automotive manufacturing is an intensely competitive global industry where manufacturers continually strive to balance cost reduction and efficiency improvements with increased sales and service innovations through their dealer network.

Integrating their dealer network seems simple enough – in theory at least – but the reality is different. In fact, achieving a seamless flow of data between a manufacturer and its dealers is a major problem that can have a big impact on the success of any customer service initiative.

The reason is that dealers tend to use different Dealer Management Systems (DMS) to run their business and each DMS supplier has a different approach to data integration. Some only make data available as a file extract or reports whilst others provide an interface which allows development of customised software.

Each DMS also has unique and sometimes arcane data structures and this adds to the complexity of accessing, aggregating, replicating, converting and using the data. Even within a DMS different versions are sometimes incompatible.

Some manufacturers have tried to solve this problem themselves by writing data extraction programs or by having dealers send extracts of their data as a flat file or a spreadsheet, but these are all largely ineffective, time-consuming, labour-intensive, prone to data quality and accuracy errors and just too expensive.

Honda UK had tried for many years to integrate their 200 dealer network but faced the familiar problem of different and incompatible DMS. They now use Datahub Automotive and this solves their problem and enables new DMS to be added quickly and simply.

Datahub Automotive is a new approach that extracts data directly from multiple, diverse DMS via the internet. Data is then aggregated, replicated, converted and loaded into a target data store before being transmitted to its destination. Once established, Datahub Automotive operates automatically through a central configuration management facility that manages any changes in functionality, maintenance routines or software updates.

This ability to access and interpret data enables a manufacturer to develop targeted customer service programmes based on accurate data. The results are impressive and include supply chain efficiencies and reported revenue growth through better dealer management.

DMS Data Migration Achieved

ej@etlsolutions.com (Editor) — Tue, 25 Aug 2009 00:00:00 GMT

ETL Solutions have been able to assist with the migration of an automotive dealer’s data migration to the latest Autoline DMS system.

Autoline provided an input file format defining the data required to initialise their Autoline system. ETL’s knowledge and experience, developed over a number of years of extracting data from a large number of DMS systems, allowed us to create the extracts required quickly. A test run was performed and this proved successful. This was followed by a final transfer when the dealer went live with the new system. The legacy migration performed for this dealer will be able to be used in the future for similar legacy migrations to Autoline.

MSX International selects ETL Solutions

ej@etlsolutions.com (Editor) — Wed, 22 Jul 2009 00:00:00 GMT

MSX International selects ETL Solutions as European Automotive Data Extraction Services Partner.

March 2009, Bangor, UK

European Auto Dealers to Benefit from Strategic Partnership

MSX International, Inc. (MSXI), a leading provider of outsourced business solutions and services to the global automotive marketplace, today announced that it has selected ETL Solutions Limited, a leading data integration solutions provider, as its European dealer management system (DMS) data collection and distribution service partner. Under the terms of the agreement, the two companies will integrate MSXI’s™ Automotive Business Processing Outsourced (BPO) applications solutions with ETL’s Transformation Manager (TM) Automotive Framework, which is a Web-based system that provides a direct interface to leading dealer DMS systems in Europe. MSXI will use ETL’s data extraction capabilities to seamlessly and securely access dealer client information such as service records, bookings, parts sales invoices, and new and used car sales information. This information will be used to drive MSXI’s Customer Lifecycle Management (CLM) programs, which enable dealers to create effective “touch points” with customers during the ownership of their vehicles, and create and retain long-term, loyal customers. The data will also be used in conjunction with MSXI’s Dealer – Sustain Service Profits application and training programs to develop highly effective dealer personnel and increase service profitability.

“The ability to quickly and securely access key dealer data in an efficient manner is critical to improving the customer experience and dealer profitability,” said Peter M. Leger, MSXI president and chief executive officer. “However, integrating OEM dealer networks can be a challenging issue, particularly since the information technology infrastructure used by dealers often varies widely."

Read about MSXi and ETL's Partnership which appears in the news section on Automotive-Technology.com

ETL Solutions announces DataPort

ej@etlsolutions.com (Editor) — Wed, 22 Jul 2009 00:00:00 GMT

ETL Solutions announces DataPort, a fixed price solution for Data Integration.

March 2009, Bangor, UK

DataPort for SAP, Barloworld Optomiza and Deltek OpenPlan released.

DataPort is based on Transformation Manager (TM) a Data Integration Platform specifically designed to meet the challenges of data integration and improve productivity.

TM achieves this faster, more effectively and cost efficiently than conventional hand-coding methods. It has been deployed across industry sectors and forms the base of data integration solutions used by some of the most demanding companies in the world.

DataPort is offered as a packaged service. This means that it can be implemented for a fixed fee against a fixed scope.

"The advantages to end-users and partners are huge. As well as providing peace of mind, it also delivers proven integration in the most cost effective manner available today." says Mark Williams, Industry Sales Manager at ETL Solutions. "Our product offerings portfolio based on our Integration Platform are expanding rapidly as more and more companies seek cost effective ways of managing their IT data movement & sharing issues."

DataPort offerings today include integration to Barloworld Optimiza, Deltek's OpenPlan and SAP.