The Keys to Successfully Adopting Hadoop
In any serious Hadoop project, you should start by teaming IT with business leaders from VPs on down to help solve your business’s pain points — those problems (real or perceived) that loom large in everyone’s mind.
Businesses want to see value from their IT investments, and with Hadoop it may come in a variety of ways. For example, you may pursue a project whose goal is to create lower licensing and storage costs for warehouse data or to find insight from large-scale data analysis. The best way to request resources to fund interesting Hadoop projects is by working with your business’s leaders.
Also examine the perspectives of people and processes that are adopting Hadoop in your organization. Hadoop deployments tend to be most successful when adopters make the effort to create a culture that’s supportive of data science by fostering experimentation and data exploration. Quite simply, after you’ve created a Hadoop cluster, you still have work to do — you still need to enable people to experiment in a hands-on manner.
Practically speaking, you should keep an eye on these three important goals:
Ensure that your business users and analysts have access to as much data as possible. Of course, you still have to respect regulatory requirements for criteria such as data privacy.
Mandate that your Hadoop developers expose their logic so that results are accessible through standard tools in your organization. The logic and any results must remain easily consumed and reusable.
Recognize the governance requirements for the data you plan to store in Hadoop. Any data under governance control in a relational database management system (RDBMS) also needs to be under the same controls in Hadoop. After all, personally identifiable information has the same privacy requirements no matter where it’s stored. Quite simply, you should ensure that you can pass a data audit for both RDBMS and Hadoop!
When you combine Hadoop with the broader business and its repositories like databases and document stores, you can build a more complete picture of what's happening in your business. For example, social sentiment analysis performed in Hadoop might alert you to what people are saying, but do you know why they’re saying it?
This concept requires thinking beyond Hadoop and linking your company’s systems of record (sales, for example) with its systems of engagement (like call center records — the data where you may draw the sentiment from).