David Stodder

The world may be flat, as author and New York Times columnist Thomas Friedman has observed: but these days, it feels more like a deflated soufflé. Great sums of real and imaginary wealth blew out of the economy like overheated gas, leaving behind a sticky, gooey mess. Financial crises have been swirling about the globe, sucking the life out of commerce, launching a global recession and exposing weakness in nearly every business model. Sophisticated use of information and the deployment of advanced algorithms and software processes clearly helped organizations uncover opportunities and develop information-based products when things were going up. But apparently too few employed their wealth of knowledge to avoiding the downsides of risk. Now, organizations will need many of the same sorts of tools to scrutinize processes and decisions that exposed them to their current troubles and discover how to survive and prosper in a remade economic world.

During such difficult and rapidly changing times, it would be hard to argue against the importance of business intelligence for providing executives, managers and front-line knowledge workers with the tools they need to use information effectively. While it has always been tough to measure the exact impact of BI and data warehousing on overall business performance, it is certain that in many organizations, implementation of BI and data warehouse tools and technologies has forever changed how users access, analyze, report and share data. Excellence in these activities is for many organizations as important as transaction or process efficiency, and essential to the cycle of improvement. In other words, no matter how bad things get, it is unlikely that organizations will turn away from BI and go back to more primitive tools and methods.

Yet, there is also little doubt that BI and data warehousing providers and professionals will have to adjust to a deflated world and refashioned expectations and objectives. There will be less patience with delays in the development and deployment of systems and promised returns on investment. The margin for error will be tighter, with fewer second chances. CIOs, CTOs and business analysts will be under pressure to manage costs associated with information analysis and reporting.

With the omnipresent economic challenges serving as backdrop, what are this year's BI megatrends? Which BI technology developments are most important to your organization? This article describes nine important trends that are shaping the future of business intelligence and information management, from open source software and embedded BI to analysis of data relationships and business models to data warehousing alternatives including MapReduce and column-store databases. But first, let's consider what happened with last year's top megatrend.

Consolidation Conundrums

Twelve months ago, the uber-megatrend was industry consolidation. IBM, Oracle and SAP had just gobbled up Cognos, Hyperion and Business Objects respectively; together with Microsoft, these vendors seemed poised to bring down the curtain on the old BI industry and begin a new phase dominated by enterprise application and information management technology heavyweights.

As it turned out, in 2008 these four giants made significant technology moves but spent a good deal of their time under the hood, retooling their newly integrated and sometimes overlapping product, marketing and sales strategies. As expected, they have positioned BI as part of something they offer that's bigger; in fact, the term "business intelligence" doesn't get much respect anymore. IBM, for example, positions BI as part of a larger thrust toward "business optimization," which it sees as different from mere application automation. Optimization is about using information as a strategic asset to improve business processes, performance management, models, forecasts and policies; it can involve a range of technologies, not just traditional BI tools for data access, reporting and analysis. Oracle and SAP take a less visionary approach, offering the tools as items in their middleware or solutions portfolios (although Business Objects, as an SAP company, still has high visibility). You have to do a bit of searching on Microsoft's site to find information about BI, but that could be because it takes a village; Microsoft's Business Division, Data and Storage Platform Division and the Office Business Platform Group all contribute technologies that BI developers, IT managers and business users are seeking.

The internal machinations of the big four have left room for independents, including Actuate, Information Builders, Jaspersoft, MicroStrategy, Pentaho, SAS, TIBCO Spotfire and others in analytics, performance management, search and vertical industry areas. On the data warehouse systems and middleware side, IBM, Informatica, Oracle and Teradata come into 2009 as heavyweights. Not much was heard in 2008 from Hewlett-Packard's NeoView, but HP made its presence felt through partnerships, particularly with Oracle to produce a data warehouse appliance. Innovation in information management is being driven by increasing customer demand for more timely and diverse data, faster query performance, better data quality, master data management (MDM) and more. Appliances, column-oriented databases and other new approaches are providing organizations with more options and are sending a message to the heavyweights that complacency equals vulnerability. Even Teradata has loosened up its enterprise data warehouse dogma to bring to market an appliance that fits the price and performance requirements for departmental data marts.