Big data is only in the first stages, but it is never too early to get started with best practices. As with every important upcoming technology, it is important to have a strategy in place and know where you’re headed.
Establish a big data road map
At this stage, you have experimented with big data and determined your company’s goals and objectives. You have a good understanding of what upper management and business units need to accomplish. It is time to establish a road map.
You clearly can’t do all the projects and meet all the demands from your company simultaneously. Your road map needs to begin with the set of foundational services that can help your company get started. Part of your road map should include the existing data services. Make sure that your road map has benchmarks that are reasonable and achievable.
If you take on too much, you will not be able to demonstrate to management that you are executing well. Therefore, you don’t need a ten-year road map. Begin with a one- to two-year road map thar includes both business and technical goals as part of the road map.
Discover your big data
No company ever complains of too little data. In reality, companies are swimming in data. The problem is that companies often don’t know how to use that data pragmatically to be able to predict the future, execute on important business processes, or simply gain new insights. The goal of your big data strategy and plan should be to find a way to leverage data for more predictable business outcomes.
Start by embarking on a discovery process. You need to get a handle on what data you already have, where it is, who owns and controls it, and how it is currently used. What are the third-party data sources that your company relies on? This process will give you a lot of insights.
For example, it will let you know how many data sources you have and how much overlap exists. This process will also help you to understand the gaps in knowledge about those sources. You might discover that lots of duplicate data exists in one area of the business and almost no data exists in another area.
This discovery process will be the foundation for your planning and execution of your big data strategy.
Figure out what big data you don’t have
Now that you have discovered what data you have, it is time to think about what is missing. Take advantage of the task force you have set up. Business leaders are your best source of information. These leaders will understand better than anyone else what is keeping them from making even better decisions.
When you start this process of determining what you need and what is missing, it is good to encourage people to think out of the box. For example, you might want to ask something like this: “If you could have any information at any speed to support the business and cost were no issue, what would you want?”
Understand the big data technology options
At this point, you understand your company’s goals, you have an understanding of what data you have, and you know what data is missing. But how do you take actions to execute your strategy? You have to know what technologies are available and how they might be able to assist your company to produce better outcomes.
Begin to understand the value of technologies such as Hadoop, streaming data offerings, and complex event-processing products. You should look at different types of databases such as in-memory databases, spatial databases, and so on. Get familiar with the tools and techniques that are emerging as part of the big data ecosystem.
Continually test your big data assumptions
You will begin to find that making use of new data sources and massive amounts of data that could never be processed in the past can help make your company much better at anticipating the future. You will be able to determine the best actions to take in near real time based on what your data tells you about a customer or a decision you need to make.
Even if you have all the processes in place to ensure that you have the right controls and the right metadata defined, it is still important to test continuously. If you are getting results that seem hard to believe, it is important to evaluate outcomes.
After you have more accurate data, you will be able to achieve better and more accurate outcomes. However, in some cases, you may see a problem that wasn’t apparent. Therefore, don’t just assume that the data is always right. Test your assumptions and what you know about your business.