Data quality assurance in BI - based on CRM data

Data analysis helps companies make informed decisions and continuously improve their performance, but is it always so? The problem is that without an established data quality process, a company may rely on bad data, and that can have a disastrous effect. Indeed, bad data costs the US $3 trillion per year, Harvard Business Review reports. The companies that understand the challenge try to resolve it by either using own resources or reaching out to BI consulting.

This article dwells on data quality assurance in the context of customer relationship management (CRM). Despite CRM is only one of many data sources, the challenges described below are quite typical.

CRM Data Quality

Key data quality challenges and why they are severe

Incomplete data

A missing lead source may be an example of incomplete data. This information is critical to understand how effective company’s communication channels are, and adjust the marketing strategy, if required.

Duplicate data

At first sight, duplicate data does not pose a challenge. However, if a customer appears more than once in the CRM, not only this causes unnecessary data storage, but also leads to a wrong customer count. Additionally, duplicate data weakens marketing analysis: it disintegrates a customer’s purchase story and, consequently, the company is able to neither understand the customers’ needs nor segment their customers properly.  

Outdated information

Imagine that a customer once completed a retailer’s questionnaire and stated that they did not have children. However, time passed – and now they have a newborn baby. The happy parents are ready to spend their budget on diapers, baby food and clothes, but is our retailer aware of that? Is this customer included in “Customers having babies” segment? Outdated data may result in wrong customer segmentation and poor knowledge of the market.

Ambiguous data interpretation

A CRM may have a field called “Loss reason” intended to help in identifying what went wrong. Usually it takes the form of a drop-down menu. However, it may include “Other” option as well. As a result, a weekly report may tell that in 80% of cases the reason for losing customers was “Other”. Thus, a company can make painful mistakes without even learning from them!

Unreliable info

Assume that a company segments their leads by headcount, and a lead profile indicates the headcount of 500. This is a signal to include a lead in the segment “Mid-sized companies”. However, in reality the headcount is 50 employees, which means that the right segment is “Small companies”. If the mistake is systematic, it leads to a wrong segmentation that, in turn, gives a misleading insight into the efficiency of lead generation channels.

Late entry/update

Late entries and updates may negatively affect data analysis and CRM reporting. For example, a sales rep should log every contact with a lead into the CRM. Imagine that the sales rep had already contacted leads, but did not log it. Later, a sales manager received a misleading report showing that there were many leads who had not been contacted for a long time.

What are the remedies?

Making it a priority

The first step is to make data quality assurance a high priority and ensure that every team member understands data quality challenges, as well as its effects on company’s decision-making.

Cleaning data

A well-known truth is that it is easier to prevent a disease than cure it. Therefore, a real-time data check for incompleteness and inaccuracy before adding data into the system will be a good starting point. Additionally, this will help to identify and eliminate the duplicates in data entries.

Automating data entry

Less manual work means fewer mistakes. Thus, the company should think how to automate the data entry in order to reduce a human factor. Whenever the system can do something automatically (for example, auto-completes, call- or e-mail logs), it is worth implementing.

Maintaining data

Data quality assurance is an ongoing process. To ensure that there is no outdated information in the system, it is necessary to revalidate the data at defined time intervals (for example, quarterly).

Make or buy? 

Every company should decide for itself whether to use internal resources for data quality assurance or to reach out for professional support of BI consulting experts. The combination of both approaches is also possible. Business intelligence services will bring best practices, professional tools and sophisticated analysis technics to set up the process, while a customer can develop internal policies and ensure data quality assurance in internal systems.

We offer BI consulting services to answer your business questions and make your analytics insightful, reliable and timely.

About the Author: Alex Bekker

Alex Bekker is the Head of Data Analytics Department at ScienceSoft, an IT consulting and software development company headquartered in McKinney, Texas. Combining 20+ years of expertise in delivering data analytics solutions with 10+ years in project management, Alex has been leading both business intelligence and big data projects, as well as helping companies embrace the advantages that data science and machine learning can bring. Among his largest projects are: big data analytics revealing media consumption patterns in 10+ countries, private labels product analysis for 18,500+ manufacturers, BI for 200 healthcare centers.

Ask the author

Sending the message ...


Sharing Information

In compliance with GDPR, your personal information will be collected and stored for five years on servers located in the Untied States. After this term is expired, your information will be erased. We will share your information with our development center, located at 2 Leanida Biady str., Minsk, Belarus, where it will be processed. At our headquarters and our development center we apply the same level of care in respect of your information as prescribed with GDPR rules. For more information, please refer to our Privacy PolicyYou may request erasing or updating your personal information here.