Friday, January 18, 2013

Data Quality Myths – Understand and Save money

“The data stored in my systems has tremendous potential. But somehow, I am not able to unlock its true value”
 
Are you also facing the above problem?
 
Data Quality Issues are the major roadblocks which prevent enterprises to realize the true potential of their data.
 
The 1-10-100 Rule of total quality management is very much applicable for Data Quality.
 
It takes $ 1 per record to verify quality of data upfront (prevention), $ 10 to cleanse and de-dupe (correction) and $ 100 per record if nothing is done- ramification of mistakes felt over and over again (failure).
 
Now that you understand the 1-10-100 rule, let’s look at some of the myths related to data quality which have been followed by many enterprises.
 
Myth 1 - My data is accurate as I have been using this for years without any problems.
Most people believe that if there are no reported problems or issues, their data is accurate. But have they realized lately that they may have missed many business opportunities which did not substantiated because of bad data? The worst part is that they have no clue about those missed chances.

A recent report from Artemis Ventures indicated that poor data quality costs the United States economy roughly $3.1 trillion per year. To provide some perspective on this unimaginably large figure, that’s twice the size of the US Federal deficit. An estimate from the US Insurance Data Management Association puts the cost of poor quality data at 15% to 20% of corporations’ operating revenue.
 
Myth 2 - I am getting my data enriched regularly and paying per record for enrichment.
There are various vendors that provide data enrichment services and charge on a per record basis. Let’s assume you are sending 100,000 records and the vendor is charging 30 cents per record, then the total cost of data enrichment is $ 30,000. At a later stage, you realized that 40,000 records were duplicates and ideally, these records should not have been provided for enrichment and that may have saved $12,000.
Generally, the clients send the data to vendor at periodic intervals and they may be losing huge amount of money every time.
 
Myth 3 - I have been using my data for regulatory compliance and there have been no issues lately.
Pharmaceutical and financial institutions need to provide data to regulators for regulatory compliance. It is a critical task and slightest of the non-compliance can result in serious financial and legal implications. In such situations where stakes are high, one should not wait for issues to arise; they should proactively look for various measures to prevent Data Quality issues.
 
A report recently issued by Aberdeen Research indicates that almost half of finance employees are “challenged by the fact that their organizations are leveraging risk and compliance data in different formats, making it difficult to compare data.”  According to the report, complying with regulations is a key concern for CFOs. And a distressing number of respondents indicated that the existing IT infrastructure is lacking in the advanced capabilities needed to support governance, risk and compliance (GRC) initiatives.
 
Data has always been the king. The sooner you realize this, the greater you are expected to save.
The best there is, the best there was and the best there ever will be - and that is Data Quality.
It takes just a tiny bit of invalid or bad information to create monumental issues. Bad data multiplies at an exponential rate, corrupting not only the system in which it originates, but also the many other data sources it interacts as it moves across the business. Thus, the longer a company waits to detect and correct a bad record, the more and severe damage it can do.
 
Thus, there is a need to establish a Data Governance framework - a combination of disciplines, enhanced processes and the right mix of tools and technology addressing the critical data issues that will drive the biggest returns, resulting in clean data that deliver results and information that is accurate.
 
References:
http://disastermapping.wordpress.com/2012/02/16/the-costs-of-data-quality-failure-2/
http://blog.match2lists.com/general-information/the-costs-of-data-quality-failure/
http://www.accountancysa.org.za/resources/ShowItemArticle.asp?Article=Data+and+Regulation%3A+Compliance&ArticleId=2398&Issue=1113