May 16, 2006

How Clean is Your Data?

An article entitled Addressing Data Quality appeared in the Feb. 8 issue of This Week (download it at and made clear the need to banish dirty data problems.

"To realize the full benefits of investments in enterprise computing systems," the article stated, "your organization must have a complete understanding of the quality of its data, how to clean it up, and how to keep it error-free. By making data quality a top strategic priority, your company can better position itself to streamline operations, increase revenue, keep costs in check, and achieve a long term competitive advantage."

In other words, companies with a computer system should feed it stuff that will enable it to work every moment of every day. Dirty data will not do that. In fact, business processes blow up when forced to process dirty data. Consider the issues that plague the electrical industry- completeness, accuracy, and timeliness of manufacturer data, to name a few-and some actual problems encountered during the past few years:

  • Poor or invalid product descriptions (e.g., a catalog number or part number doesn’t suffice as a product description).
  • Invalid or duplicate UPCs. In today’s marketplace it is essential that all manufacturers have a manufacturer’s prefix (UCC number), which forms the basis for UPC and barcodes.
  • Incorrect pricing, or when pricing data that resides in a manufacturer’s business system is not what is in the marketplace or in a distributor’s business system.
  • Missing or mislisted product codes. Product codes are key numerical identifiers that facilitate POS/POT reporting as well as discounts.
  • A manufacturer making changes to its data format and telling no one. As a result, fill rates at a large regional distributorship could go from the 90% range to the 30% range overnight.
  • Vantage Group (a national account marketing organization) promoted its "data scrubbing" operations along with its ability to save potential customers money on big electrical material purchases.* It’s a service not normally promoted by folks in electrical distribution- and it is, according to President Sean Leahy, a service eagerly taken up by national account customers (some of whom pay for it).

A dirty data solution

The Industry Data Exchange Association (IDEA) initiated its Data Audit & Certification (DAC) program in November 2003; today, the DAC program is embedded in the Industry Data Warehouse (IDW2).

In today’s business environment, information has replaced inventory as the most important strategic asset. The DAC program is all about pinpointing business information errors and processes to help manufacturers deliver complete and clean data. A company that has earned its DAC certification has worked hard to ensure that it won’t cause problems. Assuming IDEA ever gets to a place where all of the industry’s manufacturers are participating in IDW2 and are DAC-certified, the industry will have a shot at thriving.

There has been a lot written about dirty data. It seems that within the past year or so, e-business gurus came to realize that they had put too much emphasis on the idea that businesses had to clean their data.

One guess is that the gurus must have received a tidal wave of push-back from CEOs-to the effect of, "If I gotta do some kind of unholy data-cleaning exercise, the heck with the whole darn thing."

In the past six months or so, articles and presentations are conveying a new message: Data should be clean, but don’t use dirty data as an excuse to stay away from e-business. From this evidence, it can be inferred that there was too much emphasis put on dirty data in the 2003 to 2004 period.

Focusing on M2M

E-business provides pretty profits for distributors and manufacturers via something that, in other industries, is called machineto- machine (M2M) communication. Essentially, a distributor’s purchase order hits the manufacturer’s system-which seamlessly spits out circuit breakers (or whatever) in response. All of the communication, back and forth, is between machines-including advanced shipping notices, credits, debits, POS transaction info, SPAs, and much more.

If such a system works in real life as it’s designed on paper, a human being gets involved only to fix the transaction if something goes wrong-and yes, a heck of a lot can go wrong. But if the catalog numbers all match and errors were avoided in the original inputting on the distributor’s end, the transaction crosses with no glitches-and everything works.

So while a distributor might need a lot of clerical people in "old style" ordering, this won’t be the case here. As a distributorship gets better at e-business, it will find itself handling more and more business volume with the same amount of people, which allows for growth without added back-office costs.

Finally, it’s important to note that this article is focusing on a segment of e-business; there are other payoffs, and other potentials, not mentioned here.