ERP Vendors: Bring Down Those Walls

Ralph Kimball                

Now is a stressful time for data warehouse designers. in the last year or so, data warehousing has moved onto center stage. Everyone agrees that we need data warehouses; everybody wants to manage their businesses “by the numbers.” But just when the world asks us to deliver, it has decided to undertake a major paradigm shift.

Setting the hype aside, it really is becoming a Web world. Not only must we browser-enable our query tools and user interfaces, but change the way we do business.

What Are the New Rules?

First, as Patricia Seybold states so clearly in her book Customers.Com (Times Books, 1998), we must redesign our customer-facing processes from the bottom up. In other words, our customers and business partners must be able to navigate smoothly through all of our business processes, from original order to final cash settlement. They should be able to access each step along the way with just one more click. Customers and business partners need to see the detailed status of an individual order or an individual delivery (perhaps placed five minutes ago) and also be able to switch to a historical summary immediately.

Second, as a company, we must embrace a very dynamic, Web-oriented world where our supply chain is far more dynamic than it used to be. We will enter into short-lived partnerships with suppliers, based on Web auctions. These partnerships will be project-oriented and may have lifetimes of only weeks or months. Since the economics of inventory management are so compelling when we can source parts, products, and services at a lower cost and move them more quickly, we will feel huge pressure to move to the project-oriented view of the world.

As companies and vendors scramble to accommodate these new rules, the data warehouse designer must, paradoxically, take a long-term view. By definition, the data warehouse defines a perspective that is stable over a multi-year period. The data warehouse designer must resist the intense pressures to lock into a particular vendor’s technology if that means their company cannot follow the Web evolution.

Let’s try to summarize the data warehouse design criteria that have changed the most in the last 24 months. Today’s data Webhouse must be:

•Fully Web-deployed, with every end-user and administrative capability usable through a standard Web browser from any location

•Historically accurate up-to-the-moment, so that the longer view of history extends seamlessly to include the order a customer placed five minutes ago

•Profoundly distributed, so that internal business processes, external data sources, and the databases of far-flung suppliers and customers are all part of the “enterprise data warehouse”

•Dynamically changing, so that as we venture into new lines of business, and as we initiate contracts with new suppliers, the Webhouse smoothly accommodates the new data types and interfaces.

Any one of these changes would be sufficient cause for rethinking our data warehouse strategies, but all four together are definitely daunting. I think we are in a period of significant change in data warehousing and it will take another year or two before vendors and IT consumers agree upon clear product categories. As evidence of this creative turmoil, try to sort out the differences among business intelligence (BI), customer relationship management (CRM), business-to-customer (B2C), business-to-business (B2B), enterprise application integration (EAI), enterprise information portals (EIP), interenterprise application cooperation (IAC), virtual enterprises (VE), plug-and-play bondware (P&PB), wide-area workflow management (WAWM), and restructurable modeling of organization players (RMOP). Intelligent Enterprise has featured each one of these topics recently, and in many ways the magazine’s long-term mission is to sort out these differences.

The Role of ERP in the New Webhouse

I learned long ago never to do too good a job of describing a problem without also providing at least one solution. You don’t get many points for leaving the boss (or an Intelligent Enterprise reader) with unresolved frustration.

In thinking about these new data warehouse challenges over the past few months, I have become increasingly convinced that among all the players, the enterprise resource planning (ERP) system providers are best positioned to lead us to the solution, if these providers are willing to remake themselves in response to the demands of the Web business revolution. The biggest ERP vendors include Baan, J.D. Edwards, Oracle, PeopleSoft, and SAP.

ERP systems’ fundamental goals have always been reengineering and replacing an organization’s primary operational systems with a comprehensive and smoothly integrated suite of applications that implements a whole set of business processes. The most successful installations of ERP systems have been those where the customer has built their business processes around the ERP software, rather than the other way around.

ERP installations have a reputation for being lengthy and difficult. Most ERP applications take years to install and require changes in data capture screens, detailed operational procedures, descriptive terminology, accounting rules, and final management reports. But this fact should not surprise you. Usually, those organizations most in need of adopting an ERP solution are trying for the first time in their histories to design a completely coherent set of business processes. In data warehouse terms, they are making serious efforts to conform their dimensions and conform their facts across their organizations. In many ways, ERP projects have been the most ambitious projects that large organizations have ever undertaken to create conformed data environments.

In spite of ERP systems’ basic constructive approach, until now they have had a minor, disappointing effect on the data warehouse world, in my opinion, for several reasons:

•ERP systems have been far more concerned with transaction processing than decision support. Data warehousing has been a late arrival and an afterthought for most ERP vendors. Many ERP vendors have a back-room culture that doesn’t validate the needs of front-room end users.

•ERP systems’ primary database schemas are absurdly complex, including thousands of database tables. Although it is possible to extract data from an ERP system into a data warehouse, it requires specialized knowledge and powerful extraction software.

•ERP vendors have introduced data warehouse environments within their product offerings, but have not until now convincingly demonstrated their willingness to openly import and export data when that is what the customer desires.

Within the past year, ERP vendors have begun to respond more effectively to these issues. After the run-up to the Y2K transition, ERP vendors have seen a pronounced flattening of their revenues. In order to respond to the Web revolution, the IT marketplace is already wondering if ERP implementations are “legacy applications” that we need to augment with something more modern.

Although ERP vendors have always argued effectively that single-vendor solutions eliminate interfacing nightmares among vendors, the IT marketplace for applications has been growing so fast that the ERP vendors realize that they cannot keep up. ERP vendors more or less completely missed the new customer-oriented developments, including call center management and CRM applications. And the marketplace is not going to wait for them to develop these applications in a proprietary way. We are now on Web time, remember?

In the past year, all the ERP vendors have seriously embraced the Web revolution and have taken major steps to add customer-oriented functionality and better reporting facilities.

But I believe that the ERP vendors are only halfway there in terms of joining the new Web business paradigm. An organization must be able to use its own ERP installation’s solidity to define internal processes and data, and, at the same time, flexibly accommodate a constantly changing array of external partners and interfaces. The ERP vendors must be willing to participate much more openly and dynamically than they have in the past.

Given this fact, can we build a true enterprise data warehouse within a particular ERP vendor’s walls?

Well, maybe.

I think any enterprise data warehouse we implement within an ERP environment can only be successful if the ERP vendor makes a full commitment to open data warehousing with:

•A competitive level of query performance, equal to the best performance available from an extracted, independent, and aggregate-aware set of data warehouse tables

•Competitive level of user interface excellence, and ease-of-use if the ERP vendor expects the application to use its own tools

•Full data import interfaces that encourage the easy integration of foreign data sources, including syndicated data suppliers, legacy applications, packaged applications, and competitor ERP systems

•Full data export interfaces that let the customer extract the ERP data to an independent data warehouse if that is what the customer wants to do

•The flexible attachment of third-party query and reporting tools directly to the ERP vendor’s data warehouse tables

•The flexible attachment of external transaction processing applications and other ERP systems to the vendor’s operational and/or data warehouse data through extensible markup language (XML) interfaces (thereby sending the clear message that it is OK for us to view the ERP vendor’s system as part of a larger system)

•A seamless connection of current status reporting to historical record reporting

•A seamless transition between analysis (data warehousing) and update (transaction processing)

•A seamless integration of standardized facts and dimensions with local, user-defined facts and dimensions.

ERP vendors have a profound natural advantage because of their success in reengineering the internal processes of many businesses. But they are poised on a cusp. Significant new forces are sweeping the marketplace that can either strengthen the ERP vendors’ position or make them less important. In my opinion, the ERP vendors can grab the advantage in this complex new marketplace by aggressively pursuing the end customer, the end user, and above all, open systems.

Ralph Kimball co-invented the Star Workstation at Xerox and founded Red Brick Systems. He has written three best-selling data warehousing books, including the newly released The Data Webhouse Toolkit (Wiley, 2000). Ralph teaches dimensional data warehouse design through Kimball University and critically reviews large data warehouse projects. You can reach Ralph through his Web site at www.ralphkimball.com.