Decisions about Your Company's Existing Data - dummies

Decisions about Your Company’s Existing Data

By Thomas C. Hammergren

When you begin to plan for a data warehouse solution for your company, you will want to ferret out a complete inventory of extract files that fill the role of prehistoric data warehouse, each one most likely serving a single organization’s needs.

Next, you and the business users have reached consensus about what’s good about each file, what needs to be improved, and other aspects of the data use throughout the organization.

Now, it’s decision time.

Although it might come as a surprise to you, you have no reason not to build a data warehousing environment to contain one or more existing extract files. Although a single, monolithic data warehouse probably would have difficulty interacting with these extract files, an environment constructed in a mixed-mode, component-oriented manner can encompass a long-in-the-tooth component or two.

You must test each extract file, and then you will be left with one of three answers regarding what to do with that extract file:

  • Discard it.

  • Replace it.

  • Retain it, possibly with some upgrades or enhancements.

Choice 1: Get rid of the extract file

If, and only if, you have universal agreement from every corner of the organization that an extract file has absolutely no use (for example, no one looks anymore at the reports it generates, and no one has updated the data-extraction and data-input processes, so the system is generally doing little, other than wasting disk space and processing time), you can probably just dump the file without any type of replacement or upgrade.

Although some individual data elements might eventually find their way into the data warehouse, they have no business value in how the data is currently organized (data organized in a specific way).

Be brave: Throw it out.

Choice 2: Replace the extract file

Suppose that an organization does actually use an extract file, but, honestly, that file is somewhat cumbersome and difficult to use. It doesn’t include all the necessary information, users struggle to change the extraction processes to extend the list of attributes, and it meets only about 50 percent (or less) of the users’ needs.

In this situation, replace the file by following these steps:

  1. Retain the extract file’s functionality that users want as part of your data warehousing environment.

  2. Create designs and plans for the functionality that users need but don’t have in the extract files.

  3. Replace the extract file by folding the existing functionality into the data warehouse, along with the newly designed features.

    Just get rid of the old, antiquated environment.

Choice 3: Retain the extract file

If these conditions exist with the current extract-file setup, strongly consider retaining this type of environment:

  • The extract file is a relatively recent addition to the organization’s capabilities (within the past one to three years, for example).

  • The data is stored not in a flat file, but rather in a relational database.

  • The data quality is excellent.

  • Users have access to basic reporting tools and are doing some degree of ad hoc querying on their own.

  • The environment generally doesn’t look too bad. (A grandmotherly saying is appropriate here: “Your data warehouse should look and feel so good when it’s that age.”)

Although you might want to consider upgrading the extract file a little, definitely don’t throw it out and try to replace it.

These kinds of upgrades might be appropriate:

  • Add data elements closely related to ones that already exist (their point of origin is the same application file or database, for example), which people in other organizations might use.

  • Do a little performance tuning to increase response time if a larger group of users will access the database.

  • Increase the frequency of updates, and improve the freshness of the data if business needs dictate doing so.

  • Equip users with new tools, in addition to their existing ones, to expand their horizons in business analysis and use.

  • Add — only if you’re daring — a new subject area or two to this environment to provide an even richer set of data for business analysis purposes. Be careful to avoid disrupting existing functionality.

If you’re considering adding new subject areas to your extract, look at an extract file that’s worth salvaging as a ’67 Corvette that needs a little work and has been garaged for a while. Although you can always sell the car and buy a new one, when you step back and consider all your options, it’s probably less expensive to invest in the required maintenance.

Besides, you’re already familiar with the car. It might not be new, and it doesn’t have all kinds of advanced computer controls, GPS, and antilock braking; it might not have a back seat or a rear window; but it still gets you where you want to go.