Marcia Kaufman

To stay competitive today, companies must find practical ways to deal with big data — that is, to learn new ways to capture and analyze growing amounts of information about customers, products, and services.Data is becoming increasingly complex in structured and unstructured ways. New sources of data come from machines, such as sensors; social business sites; and website interaction, such as click-stream data.

What does your business now do with all the data in all its forms? Big data requires many different approaches to analysis, traditional or advanced, depending on the problem being solved. Some analyses will use a traditional data warehouse, while other analyses will take advantage of advanced predictive analytics.

Security and privacy requirements, layer 1 of the big data stack, are similar to the requirements for conventional data environments. The security requirements have to be closely aligned to specific business needs. Some unique challenges arise when big data becomes part of the strategy: Data access: User access to raw or computed big data has about the same level of technical requirements as non-big data implementations.

Cloud computing has evolved in recent years. The new world of the hybrid cloud is an environment that employs both private and public cloud services. Companies are realizing that they need many different types of cloud services in order to meet a variety of customer needs. The growing importance of hybrid cloud environments is transforming the entire computing industry as well as the way businesses are able to leverage technology to innovate.

Business Process as a Service (BPaaS) is any type of horizontal or vertical business process that’s delivered based on the cloud services model. These cloud services — which include Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS) — are therefore dependent on related services.

While the worlds of big data and the traditional data warehouse will intersect, they are unlikely to merge anytime soon. Think of a data warehouse as a system of record for business intelligence, much like a customer relationship management (CRM) or accounting system. These systems are highly structured and optimized for specific purposes.

Many companies are exploring big data problems and coming up with some innovative solutions. Now is the time to pay attention to some best practices, or basic principles, that will serve you well as you begin your big data journey. In reality, big data integration fits into the overall process of integration of data across your company.

Big data is most useful if you can do something with it, but how do you analyze it? Companies like Amazon and Google are masters at analyzing big data. And they use the resulting knowledge to gain a competitive advantage. Just think about Amazon's recommendation engine. The company takes all your buying history together with what it knows about you, your buying patterns, and the buying patterns of people like you to come up with some pretty good suggestions.

Four stages are part of the planning process that applies to big data. As more businesses begin to use the cloud as a way to deploy new and innovative services to customers, the role of data analysis will explode. Therefore, consider another part of your planning process and add three more stages to your data cycle.