Just having access to big data sources is not enough. You will need to integrate these sources. Soon there will be petabytes of data and hundreds of access mechanisms for you to choose from. But which streams and what kinds of data do you need?
Understand the problem you are trying to solve
Identify the processes involved
Identify the information required to solve the problem
Gather the data, process it, and analyze the results
This process may sound familiar because businesses have been doing a variation of this algorithm for decades. So is big data different? Yes, even though businesses have been dealing with large amounts of operational data for years, big data introduces new types of data into people's professional and personal lives.
Twitter streams, Facebook posts, sensor data, RFID data, security logs, video data, and many other new sources of information are emerging almost daily. As these sources of big data emerge and expand, people are trying to find ways to use this data to better serve customers, partners, and suppliers. Organizations are looking for ways to use this data to predict the future and to take better actions.
Healthcare is one of the most important and complex areas of investment today. It is also an area that increasingly produces more data in more forms than most industries. Therefore, healthcare is likely to greatly benefit by new forms of big data. The healthcare providers, insurers, researchers, and healthcare practitioners often make decisions about treatment options with data that is incomplete or not relevant to specific illnesses.
Part of the reason for this disparity is that it is very difficult to effectively gather and process data for individual patients. Data elements are often stored and managed in different places by different organizations. In addition, clinical research that is being conducted all over the world can be helpful in determining the context for how a specific disease or illness might be approached and managed.
Apply the algorithm to a standard data healthcare scenario:
Understand the problem we are trying to solve:
Need to treat a patient with a specific type of cancer
Identify the processes involved:
Diagnosis and testing
Results analysis including researching treatment options
Definition of treatment protocol
Monitor patient and adjust treatment as needed
Identify the information required to solve the problem:
Patient history
Blood, tissue, test results, and so on
Statistical results of treatment options
Gather the data, process it, and analyze the results:
Commence treatment
Monitor patient and adjust treatment as needed
This is how medical practitioners work with patients today. Most of the data is local to a healthcare network, and physicians have little time to go outside the network to find the latest information or practice.