With the governance challenges presented by big data, it is wise and absolutely necessary to have practices in place to ensure that you are protecting your information. While the degree to which you do these will vary depending on your business, make sure you are taking necessary precautions.
Audit your big data process
At the end of the day, you have to be able to demonstrate to internal and external auditors that you are meeting the rules necessary to support the operations of the business. You will need a way to show logs or other evidence that the data you are using is secure and clean. You will need to explain the sources of that data.
Will you be able to validate the results so that you minimize the risk to the company? You may have to prove that you have archived the data that you are using to make decisions and run the business. This may be well-managed for your traditional databases and your data warehouse, but your unstructured big data sources have not been added to this process.
Although external auditors may not analyze the accuracy of the data warehouse–based data with external big data sources, your internal process will dictate that these sources be well synchronized. For example, the data warehouse will have a clear set of master data definitions, but the big data sources may not have documented metadata.
Therefore, it is important that external data sources be managed in a way that metadata definitions are codified so that you can have a set of consistent metadata across these sources. Thinking through this process can make the difference between business success and failure.
Identify the key big data stakeholders
One of the characteristics of big data is that it is typically tied to specific business initiatives. For example, the Marketing organization wants to be able to use the huge volumes of data generated by social media sites such as Facebook, Twitter, and so on.
Operations teams will want to manage their supply chain leveraging RFID data. The Human Resources department will be eager to keep track of what employees are publishing on social media sites to make sure that they are not violating internal and external regulations. A medical claims department will want to keep track of the regulations determining how patient claim information is managed so that privacy rules are not violated.
All of these constituents may reside within the same company, so it is critical that everyone has a common understanding of what the rules are and that the infrastructure is in place to keep the company consistently safe.