Six Vendors with Middleware Products for Data Warehousing
Middleware Services: Data Quality Assurance
Business Intelligence Architecture and Data Warehousing

Integrate Big Data with the Traditional Data Warehouse

While the worlds of big data and the traditional data warehouse will intersect, they are unlikely to merge anytime soon. Think of a data warehouse as a system of record for business intelligence, much like a customer relationship management (CRM) or accounting system. These systems are highly structured and optimized for specific purposes. In addition, these systems of record tend to be highly centralized.

The diagram shows a typical approach to data flows with warehouses and marts:


Organizations will inevitably continue to use data warehouses to manage the type of structured and operational data that characterizes systems of record. These data warehouses will still provide business analysts with the ability to analyze key data, trends, and so on. However, the advent of big data is both challenging the role of the data warehouse and providing a complementary approach.

Think of the relationship between the data warehouse and big data as merging to become a hybrid structure. In this hybrid model, the highly structured optimized operational data remains in the tightly controlled data warehouse, while the data that is highly distributed and subject to change in real time is controlled by a Hadoop-based (or similar NoSQL) infrastructure.

It's inevitable that operational and structured data will have to interact in the world of big data, where the information sources have not (necessarily) been cleansed or profiled. Increasingly, organizations are understanding that they have a business requirement to be able to combine traditional data warehouses with their historical business data sources with less structured and vetted big data sources. A hybrid approach supporting traditional and big data sources can help to accomplish these business goals.

  • Add a Comment
  • Print
  • Share
blog comments powered by Disqus
Determine What External Data You Really Need
The Vision of Collaborative Business Intelligence
The Data Warehouse Deluxe
Data Mining
Five Vendors with Querying and Reporting Products for Data Warehousing