Now that we have more than a decade of experience with modern business intelligence (BI) technology, an increasing number of "new" BI applications aren't really new at all. They're second-generation projects. We use this term to describe replacements for existing applications that have become obsolete for several reasons, including poor scalability and flexibility and limited acceptance by the user community.

Even highly successful implementations must some day be replaced. The technology platforms upon which they were built may have fallen out of favor; or, the original developers are gone and didn't leave a well-documented legacy. Other possible reasons for replacement might include new decision processes, new business challenges, changes in user profiles, merger and acquisition activity, and so on.

Some projects can certainly move forward with existing toolsets. But many organizations will choose to put the application "in play" and go through a vendor selection process. Their intent will be to break with the past and go with the best currently available tools — as opposed to what was available when the original toolset was chosen.

BI professionals have proven methods for driving a software selection process. However, best practices must change if you're dealing with a second-generation project. BI users and developers bring biases to the process that can hinder objectivity and lead to something other than true, fact-based decision-making. In this article, we'll explore some of the unique challenges of second-generation BI projects and identify where you must keep bias in check so that your organization doesn't get trapped by history on the way to reaching strategic business objectives.

Experience: The Biggest Bias

The most obvious obstacle to effective vendor selection is decision makers' biases acquired during their individual histories with BI. Some will say that it's important to keep the competition "fair" and level the playing field. However, the true direction should be to bring objectivity and efficiency to the process. Then, your organization can find the best possible fit of tools to applications and pursue strong return on investment (ROI). The products that offer the best fit will rightly have an advantage going into a competitive sales situation.

Recent experience is perhaps the strongest source of bias. Both users and IT professionals tend to associate their opinions about tools and vendors with their views on recent experiences with the application of these tools. Some even go so far as to name the application after the vendor: for example, "the organization gets its daily detail reports from Cognos" — meaning, of course, the application built using the vendor's BI tools, not from Cognos itself. If the application is successful, such close brand-name identification works in the vendor's favor.

Users and developers will, however, associate the weaknesses of the existing application with the vendor's tools, whether merited or not. As a result, incumbent vendors often go into the application replacement process with one strike against them. You'd think that the opposite would be true: The incumbent vendor already has license arrangements with the client, developers and users are already trained, and a code base has been established. Such advantages are negated if the vendor isn't perceived as supportive when the system hasn't proved to be satisfactory.

Bias based on recent experience can lead users and IT professionals to overestimate their knowledge of the tools' capabilities. In recent years, we've seen rapid enhancement and innovation in mainstream BI tools, much of it the result of the current wave of industry consolidation. Thus, previous vendor evaluations could be obsolete if they haven't been revised. It's rare to see applications rebuilt to leverage new product release enhancements, assuming they're installed.

Bias: The IT View

The IT perspective generally offers strong biases both for and against the status quo. For example, if the original technology buyers are still around, they'll tend to be defensive about their decisions unless convinced that requirements have changed or that another vendor has come up with a demonstrably superior product. Also, over time, vendors will often cultivate "preferred customer" relationships with IT personnel. Trusted relationships form between developers and vendor support personnel. IT managers become active in vendor user organizations. They occasionally benefit directly or indirectly from making their organization a valued vendor reference site.