Data Extraction, Movement, and Loading for Data Warehouse Supreme - dummies

Data Extraction, Movement, and Loading for Data Warehouse Supreme

By Thomas C. Hammergren

A data warehouse has large-scale data distribution and advanced technologies that can integrate various “run the business” systems, improving the overall quality of the data assets across business information analytical needs and transactional needs. A data warehouse supreme most likely consists of a database environment that meets these requirements:

  • It’s distributed across many different platforms.

  • It operates in a location-transparent manner: Users make queries that access data from the appropriate platform without the users having to know the physical location (in much the same way that you access Internet web sites by name, rather than by network address).

  • It has object-oriented capabilities to store images, videos, and text in addition to the traditional data, such as numeric and date information.

  • Because of dramatically faster performance than current data warehouses, it increasingly permits you to access data directly from transactional databases without having to copy the information to a separate data warehouse database.

Here’s how the extraction, movement, and loading of data occurs in a data warehouse supreme:

  • Data that’s moved (copied) from a source application’s database or file system into a separate database in the data warehouse is handled almost identically to how you perform those tasks in a data warehouse deluxe.

  • The increasing use of operational data stores, or ODSs (real-time availability of analytical data so that you don’t have to deal with delayed access) means that more messaging occurs between your data sources and your warehouse database.

    The data source determines when data should be moved into the warehouse environment, so the warehouse doesn’t have the responsibility to request updates and additions. When new data is inserted into the source database (or existing data is modified or deleted), the appropriate instructions and accompanying data are sent to the warehouse.

This figure shows an example of what the architecture of a data warehouse supreme might look like. But with all the upcoming technology trends and improvements discussed here, your data warehouse supreme can look like (almost) anything you want.