Data quality assurance in BI - based on CRM data

Data analysis helps companies make informed decisions and continuously improve their performance, but is it always so? The problem is that without an established data quality process, a company may rely on bad data, and that can have a disastrous effect. Indeed, bad data costs the US $3 trillion per year, Harvard Business Review reports. The companies that understand the challenge try to resolve it by either using own resources or reaching out to BI consulting.

This article dwells on data quality assurance in the context of customer relationship management (CRM). Despite CRM is only one of many data sources, the challenges described below are quite typical.

CRM Data Quality

Key data quality challenges and why they are severe

Incomplete data

A missing lead source may be an example of incomplete data. This information is critical to understand how effective company’s communication channels are, and adjust the marketing strategy, if required.

Duplicate data

At first sight, duplicate data does not pose a challenge. However, if a customer appears more than once in the CRM, not only this causes unnecessary data storage, but also leads to a wrong customer count. Additionally, duplicate data weakens marketing analysis: it disintegrates a customer’s purchase story and, consequently, the company is able to neither understand the customers’ needs nor segment their customers properly.  

Outdated information

Imagine that a customer once completed a retailer’s questionnaire and stated that they did not have children. However, time passed – and now they have a newborn baby. The happy parents are ready to spend their budget on diapers, baby food and clothes, but is our retailer aware of that? Is this customer included in “Customers having babies” segment? Outdated data may result in wrong customer segmentation and poor knowledge of the market.

Ambiguous data interpretation

A CRM may have a field called “Loss reason” intended to help in identifying what went wrong. Usually it takes the form of a drop-down menu. However, it may include “Other” option as well. As a result, a weekly report may tell that in 80% of cases the reason for losing customers was “Other”. Thus, a company can make painful mistakes without even learning from them!

Unreliable info

Assume that a company segments their leads by headcount, and a lead profile indicates the headcount of 500. This is a signal to include a lead in the segment “Mid-sized companies”. However, in reality the headcount is 50 employees, which means that the right segment is “Small companies”. If the mistake is systematic, it leads to a wrong segmentation that, in turn, gives a misleading insight into the efficiency of lead generation channels.

Late entry/update

Late entries and updates may negatively affect data analysis and CRM reporting. For example, a sales rep should log every contact with a lead into the CRM. Imagine that the sales rep had already contacted leads, but did not log it. Later, a sales manager received a misleading report showing that there were many leads who had not been contacted for a long time.

What are the remedies?

Making it a priority

The first step is to make data quality assurance a high priority and ensure that every team member understands data quality challenges, as well as its effects on company’s decision-making.

Cleaning data

A well-known truth is that it is easier to prevent a disease than cure it. Therefore, a real-time data check for incompleteness and inaccuracy before adding data into the system will be a good starting point. Additionally, this will help to identify and eliminate the duplicates in data entries.

Automating data entry

Less manual work means fewer mistakes. Thus, the company should think how to automate the data entry in order to reduce a human factor. Whenever the system can do something automatically (for example, auto-completes, call- or e-mail logs), it is worth implementing.

Maintaining data

Data quality assurance is an ongoing process. To ensure that there is no outdated information in the system, it is necessary to revalidate the data at defined time intervals (for example, quarterly).

Make or buy? 

Every company should decide for itself whether to use internal resources for data quality assurance or to reach out for professional support of BI consulting experts. The combination of both approaches is also possible. Business intelligence services will bring best practices, professional tools and sophisticated analysis technics to set up the process, while a customer can develop internal policies and ensure data quality assurance in internal systems.

We offer BI consulting services to answer your business questions and make your analytics insightful, reliable and timely.

About the Author: Aliaksandr Bekker

Alexander Bekker is a Head of Database and BI Department. With 18 years of experience, Alexander focuses on BI solutions (data driven applications, data warehouses and ETL implementation, data analysis and data mining) in retail, healthcare, finance, and energy industries. He has been leading such large projects as private labels product analysis for 18,500+ manufacturers, global analytical system for luxury vehicle dealers and more.