/ Are Data Warehouses Just a Stop-Gap?

Some technologies are heralded as having arrived, only to falter for years, sometimes decades. In movies, 3D took about half a century to gain traction. “Smart” appliances, such as refrigerators that track your grocery inventory, were the next big thing—a decade ago. Smart appliances are still around, and we’re all still waiting for them to become popular.

When reality doesn’t meet expectations.

In the category of late bloomers, business intelligence (BI) and data warehousing can be added to the list. In use for more than 20 years, BI and data warehousing’s ability to provide substantive benefits remains elusive for many companies. While hard-and-fast statistics are difficult to get, widely reported failure rates for BI projects range from 50 percent to as high as 80 percent.

The tech world has long talked about relational databases, but in a data warehouse, relating data is tough because the data comes from so many sources. Not only are the sources plentiful, but so are their protocols. As the Database Trends article noted, each source often has its own access mechanisms, syntax and security.

Turning problems into a solution.

With both sources and their data changing constantly, it is no wonder that BI projects so frequently fail. Of course, users and vendors have sought solutions, though with limited success. Traditional BI vendors have even gone towards the leading edge by resorting to such solutions as iPad apps or cloud-based tools. But while these are the latest technologies, they are often no more effective because they, like previous technologies, require the use of a data warehouse whose shortcomings they cannot overcome.

Needed are BI tools that easily (ease of use is the key) expand beyond the core data warehouse and allow end-users to integrate a myriad of data sources into the BI conversation without requiring timely ETL conversions and data mapping.

A half-mile per gallon increase, thanks to data.

The data warehouse is very valuable—it is an effective hub for major software systems—but it is important to recognize its limitations. Rather than trying to eliminate exceptions, you would do better by accommodating them. If you do not, you will always have incomplete data and therefore decisions based on incomplete information.

This shift happened with a national trucking company that, using leading-edge visualization technology, was able to take data from SAP, legacy and Microsoft systems and, according to an executive at the firm, “combine it and show it in one picture.”

The payoff was significant. Rather than having the typical BI solution, which he described as a “next-day problem solver,” the firm got “real-time problem solving.” He reported that productivity on his shipping docks improved 20 percent and on-time deliveries went from 97 percent to well over 99 percent. In addition, drivers were able to see how to improve their driving techniques, which boosted gas mileage by more than half a mile per gallon, resulting in projected fuel savings of between $12 million and $15 million a year.

Such results demonstrate the power of BI when it is implemented effectively. You want to put real intelligence into the hands of the average business user. To do so, you need to allow the user to rope in data from spreadsheets, a custom database, a cloud-based application—wherever it sits—so that decisions are made from all the data you have.

For more details about the pros and cons of data warehousing and BI, get the full whitepaper, here.

Tags: big data, data