Facing the EII Infrastructure Challenge - dummies

Facing the EII Infrastructure Challenge

By Thomas C. Hammergren

Wow! Virtual data warehousing sounds like a fairly neat, state-of-the-art idea. Why isn’t it more widespread? The answer, in a word, is infrastructure. Although you can talk all you want about emerging networking and communications technologies, and the tremendous throughput we’ll all have someday, most corporations are still several generations behind the state of the art in their networking infrastructure.

They’re struggling to deal with the major investment required for what’s essentially the Internet and Distributed Computing Age.

Delivering virtual data warehousing that includes EII services requires significant throughput of data in what’s often an unpredictable manner. The irony is that a virtual data warehousing environment almost assuredly requires, over time, significantly less data movement across the enterprise than traditional data warehousing, which has a philosophy of copying millions (perhaps billions) of bytes, just in case someone wants to ask a question possibly requiring that data.

Traditional data warehousing has the advantage, however, of working the clock (as they say in football) by scheduling large-scale batch loads for either system downtimes or relatively light operational loads, using offline data-transfer means (tape, for example), rather than network file transfers and other tricks of the trade.

Until an organization’s communications infrastructure can pump the data through, EII services and concepts (such as virtual data warehousing) probably will remain on the fringes, just out of reach.

If you’re interested in virtual data warehousing as an architecture that uses EII services, here’s a list of what you have to do:

  • Supercharge your networking infrastructure. Yes, it’s expensive, but you gotta do what you gotta do.

  • Stop thinking about data warehousing in centralized terms. Think distributed but interconnected subject areas of data. Free your mind!

  • Insist that any new application developed for your organization, no matter who does it, be warehouse-enabled.

  • Supercharge your EII servers with memory and processors. Reading data from memory is far faster than across the network or off a disk. Most EII servers enable you to cache the results of a query in memory and share them across users. By providing the proper memory and processor configuration, the EII server gets faster with each user request through reuse.

  • Think enterprise! Just because you’re dealing with bottom-up component development doesn’t mean that you’re developing a stand-alone piece of data. The last thing you want to wind up with is an islands-of-data-mart problem to replace the islands-of-data situation you’ve been dealing with your entire career.

    In a traditional data warehousing environment, you’re forced to deal with these enterprise-scale data architecture issues. Don’t let your foray into virtual data warehousing undermine all the good that you’ve done in your organization when it comes to dealing with enterprise data architecture.