Intelligent Enterprise

Enough Truth?

I just finished reading Joshua Greenbaum's column, "The Truth About the Truth" (Sept. 18, 2004). Although I agree with the points made in the column, I feel something was left out. In many cases, whether in business processes, military action, or intelligence gathering, we need to act with a limited amount of truth. Were we to wait for the whole truth, we would likely be stricken with paralysis by analysis. Whether or not it's wise to proceed without the complete truth, situations usually require it.

I'd be interested in seeing Greenbaum write about how to know when you have enough of the truth to make an acceptable decision. Having some ideas to use when evaluating the level of truth would be very helpful to me.

Dave Pettengill
Bellevue, Nebraska

Greenbaum responds: You raise the essential question, and one that deserves more of a response than space here will allow. Questioning the initial assumptions behind information and retaining a healthy skepticism about received wisdom are perhaps the best safeguards I can offer.

If, after careful scrutiny, the data is relatively clean and consistent, and it comes from a reliable source, then there's some safety in taking action even if the information is incomplete. But taking action based on data at hand, merely because it's at hand, simply isn't a rational way to proceed.

If that's the only option, look for intuition and experience to guide you. Sometimes gut feelings contain more wisdom than terabytes of quantitative information.

Data Quality

Joe Celko's "A Question of Quality" (Sept. 4, 2004) was very timely and thought provoking. I've been working in a dynamic environment with a lot of distributed data, owned by numerous different organizational units. We've been grappling with how to tackle our data quality problems.

We routinely hear "We can't trust that data" or "The data in their system is crap." Everyone passes judgment on quality, but nobody's able to accurately quantify his or her opinion. In short, we have no basis for evaluating data quality. Being able to measure data quality is one of our near-term objectives. We're planning to define data quality criteria, formulate sampling plans, periodically measure, and trend quality performance over time.

Of the dimensions Celko mentions, trust and spoilage, trust is the most critical in my work environment. I'm involved with a large Homeland Security program that's built on a highly integrated configuration of distributed databases. Our ability to maintain defect-free data has a direct bearing on a core program mission of ensuring uniqueness in identity.

We're spending a lot of time thinking about how we'll fit our data-quality programs into the larger development methodology. Ours is a new program that uses a variety of methodologies ranging from older SDLC waterfall approaches to elements of SEI CMM. Historically, in this environment, testing and defect identification centers on the application, not the data.

I'm confident we'll climb up the maturity ladder. I just want to get there as soon as is practical, without making any unnecessary pauses along the way.

Chris Chamberlin

Correction

In the interview with the Hyperion CEO, Godfrey Sullivan ("In Good Position," Sept. 18, 2004), he mentioned a product Hyperion acquired. We printed it as "Strategic Science," but the product name is actually "Strategic Finance." The editors regret the error.