Bernard Marr

Cheat Sheet / Updated 04-12-2022

Big data makes big headlines, but it’s much more than just a buzz phrase or the latest business fad. The phenomenon is very real and it’s producing concrete benefits in so many different areas – particularly in business. Here you will get to the heart of big data as a business owner or manager: You will take a look at the key terminology you need to understand the crucial big data skills for businesses, ten steps to using big data to make better decisions, and tips for communicating insights from data to your colleagues.

Article / Updated 03-26-2016

Big data is a fast-moving field and sometimes innovations come pretty much out of nowhere and take everyone by surprise. But sometimes it’s useful to take a peek into the crystal ball and see where things might be heading. Here are some predictions for the big moves and developments in big data in the near future: The big data market will grow by more than 25 per cent a year. The figure comes from market research specialists IDC (International Data Corporation) that predicts that the big data technology and services market will grow at a compound annual growth rate of 26.4 per cent to $41.5 billion by 2018. That’s about six times the growth rate of the overall information technology market. The Internet of Things will go mainstream. Wearable technologies and data-enabled devices are breaking out of the gadget-geek and early-adopter markets as more and more people want to be connected at all times. Expect to see (or perhaps be crashed into by!) your first person wearing smart glasses in the street pretty soon. Machines will get better at making decisions. More often than not, big data acts as guidance for decisions mostly still made by people. Expect this to change in the near future as advances in machine learning bring us closer to the point where machines are capable of making more accurate and reliable decisions than people. Textual analysis will become more widely used. Textual analysis has become increasingly sophisticated in the last few years and that trend will continue. Computers will become more proficient at reading a piece of text and spotting themes and sentiments – meaning this type of data can be classified and analysed in the same way as structured data. Data visualisation tools will dominate the market. Specialised software designed to create visualisations from data, making it easier for us to spot patterns and links between cause and effect, will become increasingly sophisticated and widely used. This market is expected to grow two-and-a-half times more quickly than that for other business intelligence software products. Personal data and privacy will become more important. It seems that more people than ever believe that giving corporations personal information is a small price to pay for the convenience and utility offered by new technology. But that may change soon – hackers have shown they’re able to compromise even the most secure systems. A devastating hack might be enough to start changing people’s attitudes and restoring a bit of common sense over how personal data is shared. Retail will go very Big Brother on you. Plenty of retailers are already tracking how customers physically move around stores and what they stop to look at. Some even use personal information (for instance, if you’ve downloaded a store app) to target promotions and recommend products while you’re in-store. Increasingly, Wi-Fi data from your mobile phone will be combined with video data and even facial recognition software to identify your age and gender, even your exact identity – all so that products and promotions can be targeted with scary precision. . . . And so will employers. Connected wearable devices are making their way into the workplace, allowing employers to track their employees’ activity and health. In fact, fitness tracking device manufacturer Fitbit recently said that employers make up their fastest growing market, so you can expect to see this trend continue. The struggle to find big data talent will continue. This year, there are expected to be 4.4 million people employed worldwide in positions directly involved with big data analysis. But this isn’t enough, especially when you consider that almost 70 per cent of US businesses already have a big data strategy or are planning one for the near future. The number of colleges offering big data courses is growing rapidly, but the shortage of skilled workers is likely to continue for the foreseeable future. Small businesses will become just as obsessed with data as the big guys. In a data-driven world, those companies that can turn big data into valuable insights are the ones that will thrive. Companies that merely dip their toes in the big data pond on an ad-hoc basis will be left behind, and those that ignore big data altogether will wither away. This is true whether you’re a multinational corporation or a small business.

Article / Updated 03-26-2016

When companies start dipping into big data, a common mistake is to head straight for the data itself. The sheer array of data available, not to mention the possibilities for collecting new data in the future, is incredibly exciting. But when your first thought is, ‘Hmmm, what data can I get my hands on?’ (accompanied by evil-genius-style hand rubbing), you’re already missing the enormous potential of big data. Data is a powerful tool that helps you answer strategic business questions, make smarter decisions, improve the way you do business and, ultimately, achieve your long-term business goals. But, for it to do that, you need to start with strategy – not the data itself. Instead of focusing on what data you already have or what data is available out in the big wide world, start by working out exactly what it is you want to achieve in your business. Are you looking to decrease staff turnover? Improve product development? Understand more about your customers? Each of these objectives requires a different approach and, you guessed it, different types of data. When you know exactly what it is you’re trying to achieve, only then can you turn your attention to the data (or combination of data) that can help you do that. Treat big data like any other big business investment, such as expanding into a new retail location or upgrading your manufacturing equipment. You wouldn’t jump into either of those things without a lot of careful thought about what you want to do and why. For instance, you’d probably set out the pros and cons, weigh up the costs and benefits and make a clear business case for the investment. It’s exactly the same with data: You should be able to make a strong business argument for using big data in your business. To do that, you need to know where your business is heading and how data can help you get there.

Article / Updated 03-26-2016

Big data skills are in short supply. As the amount of digital information generated by businesses has grown exponentially, a challenge (some people even call it a crisis) has arisen: there just aren’t enough people with the necessary skills to analyse and interpret all this big data. In one recent survey, more than half of business leaders questioned felt their ability to carry out big data analytics was limited by the challenge of finding the right talent. More and more courses are springing up to meet this skills shortage and big data is undoubtedly becoming a desirable career route for college leavers. But it’ll take time for the number of qualified people to catch up to the sheer demand for big data skills. So, at least for the next few years, the big data skills shortage is a problem that all companies interested in big data (which should be all companies) will have to face. With stiff competition to attract the best talent, companies are turning to creative ways of tapping into big data skills. Walmart, for example, decided to apply the power of the crowd, turning to crowdsourced analytics competition platform Kaggle. At Kaggle, armchair data scientists apply their skills to analytical problems submitted by companies, with the designer of the best solution being rewarded – sometimes financially or, in the case of Walmart, with a job. In Walmart’s first competition, which took place in 2014, candidates were given a set of historical sales data from a sample of stores, along with associated sales events, such as clearance sales and price rollbacks. They were asked to come up with models showing how these events would affect sales across a number of departments. As a result of the competition, several people were hired into the analytics team. Best of all, this crowdsourced approach led to some interesting appointments – people who may not have been considered for an interview based on their CVs alone. One appointee, for example, had a very strong background in physics but no formal analytics background. What does this mean for smaller businesses? Even if you can afford to hire an in-house data scientist, you may find yourself up against fierce competition from bigger companies. The Walmart example shows us that, in order to tap into big data skills, you may need to get a little creative. Maybe you, too, could crowdsource data projects (even if the end result is a simple financial reward, as opposed to a full-time job). Or perhaps you could partner with a local university or college, wherein students crunch your data in return for some business mentoring. Or maybe you already have strong analytical thinkers and communicators in your business who, with a little extra help and training, could set up and run big data projects in the future.

Article / Updated 03-26-2016

How does the Internet of Things relate to big data? The Internet of Things describes the fact that many everyday objects, from diapers to self-driving cars, have (or soon will have) the ability to send and receive data via the Internet. You can find out more about the Internet of Things and the explosion of data in Big Data for Small Business For Dummies. Much of this big data is driven by sensor technology, as well as the increasing connectivity of, well, just about everything in our world. This huge increase in data gives you the ability to create a smarter world where buildings sense and predict temperatures outside and adjust heating or air conditioning systems inside, where cars drive themselves and where your baby’s diaper tweets you when it needs changing. A futuristic vision? No, all of these things are already possible today. Here are a few more very real examples of what the Internet of Things is already doing: Wearable devices such as the Fitbit and UP fitness trackers collect data on how many steps you take, how well you sleep, how many calories you consume and much more. The next big push in wearable technology is in smart watches – Apple’s version was launched in 2015. These watches collect data on almost anything – your geographical location, your speed or your body functions – and this data can be synced to phones or tablets and analysed in apps. Sensors are everywhere. The oceans are full of sensors that track sea temperature and currents. A number of companies put sensors into farmland to track soil health and predict the right level of fertilizers required to obtain an optimal crop. Your toothbrush can even sense how well you’re brushing and send the analysis to your smartphone so you can improve your brushing technique! There are security alarms that connect to the Internet and alert you about any intruder. Bathroom scales are connected to the Internet and not only monitor weight, body mass index and heart rate, but also the air quality in the house. There are light bulbs that link to wireless networks, which allows you to control them with an iPhone. There are also sensors in gardens and indoor plants that send a message to a phone when they need watering. There are endless opportunities for businesses, science and governments to exploit this new data tidal wave. Just imagine what will happen when you connect all these devices in even smarter ways – when your refrigerator knows what items are past their use-by date and re-orders them for the next shop; when your smart watch makes an appointment with your doctor because it detected some abnormalities; when buses wait for a delayed train to arrive; or, when your alarm automatically adjusts its wake-up time because your early morning appointment cancelled overnight.

Article / Updated 03-26-2016

What are the key skills required to use big data successfully? The list here includes six key skills that all businesses should develop, either through recruiting data scientists who match these attributes, or by developing these skills in existing employees: Analytics: This involves determining which data is relevant to the question you’re hoping to answer and interpreting the data in order to derive those answers. Key skills include a knack for spotting patterns and establishing links, the ability to make sense of a range of data (both structured and unstructured) and a sound knowledge of industry-standard analytics packages like SAS Analytics and Oracle Data Mining. Creativity: Anyone can be formulaic – you need to aim for innovation that will set your business apart from the pack. Creativity is especially important for any business hoping to make sense of unstructured data – data that doesn’t fit comfortably into tables and charts. Valuable creative skills include a knack for problem solving (perhaps even spotting problems others aren’t yet aware of) and the ability to come up with new ways of gathering and interpreting data. Maths and statistics: People with a strong background in maths or statistics have a good grounding for big data-related work. You’re looking for at least a basic grasp of statistics and the ability to wrangle messy data into figures that can be quantified so that you can draw conclusions from them. Computer science: This very broad category covers a whole range of subfields, such as machine learning, databases and cloud computing. It may cover everything from plugging together the cables to creating sophisticated machine learning and natural language processing algorithms. Key skills include a solid understanding of database technology and a firm grasp of technologies such as Hadoop, Java and Python. Business acumen: People who work with big data need a firm grasp of the company’s goals and objectives, as well as an understanding of whether the business is heading in the right direction. This includes understanding what makes the company tick, what makes it thrive and why it stands out from its competitors (and if it’s not thriving, why it’s not). Communication: You can have the best analytical skills in the world, but unless you’re able to present findings in a clear way and demonstrate how they can help to improve performance and drive success, all that analysis will go to waste. Great interpersonal and written communication skills are vital, as is the ability to add value to data through insights and analysis. A knack for storytelling and being able to bring data to life through visualisation techniques will also help immensely.

Article / Updated 03-26-2016

Data should be at the heart of strategic decision making in business, whether you run a huge multinational or a small family-run business. Big data can provide insights that help you answer your key business questions, such as ‘How can I improve customer satisfaction?’. Data leads to insights; business owners and managers can turn those insights into decisions and actions that improve the business. Use this ten-step process for making data-based decisions: Start with strategy. Instead of starting with what data you could or should access, start by working out what your business is looking to achieve. In a nutshell, you need to work out what your strategic goals are, for example, increasing your customer base. Hone in on the business area; identify your strategic objectives. Identify the areas most important to achieving your overall strategy. For most businesses, the customer, finance and operations areas are key. Identify unanswered questions. Work out which questions you need to answer in order to achieve those goals. By working out exactly what you need to know, you can focus on the data that you really need. Find the data that will help answer those questions. Focus on identifying the ideal data for you – the data that could help you answer your most pressing questions and deliver on your strategic objectives. Identify what data you already have or have access to. After you identify the data you need, it makes sense to see if you’re already sitting on some of that information, even if it isn’t immediately obvious. Work out if the costs and effort are justified. Only after you know the costs can you work out if the tangible benefits outweigh those costs. In this respect, you should treat data like any other key business investment. You need to make a clear case for the investment that outlines the long-term value of data to the business strategy. Collect the data. Much of this step comes down to setting up the processes and people to gather and manage your data. You may be buying access to an analysis-ready data set, in which case there’s no need to collect data as such. But, in reality, many data projects require some amount of data collection. Analyse the data. You need to analyse the data in order to extract meaningful and useful business insights. After all, there’s no point coming this far if you don’t then discover something new from the data. Present and distribute the insights. Unless the results are presented to the right people at the right time in a meaningful way, then the size of the data sets or the sophistication of the analytics tools don’t really matter. You need to make sure the insights gained from your data are used to inform decision making and, ultimately, improve performance. Incorporate the learning into the business. Finally, you need to apply the insights from the data to your decision making, making the decisions that will transform your business for the better – and then acting on those decisions. For me, this is the most rewarding part of the data journey: turning data into action.

Article / Updated 03-26-2016

The technical jargon surrounding big data can seem a little daunting at first. The key phrases and terms you’re likely to come across, with easy-to-understand definitions for each, follow: Big data: Increasingly, everything you do leaves a digital trace (or data), which you (and others) can use and analyse. The phrase big data refers to that data being collected and the ability to make use of it. Big data analytics: This is the process of collecting, processing and analysing data to generate insights that inform fact-based decision making. In many cases it involves software-based analysis using algorithms. Algorithm: A mathematical formula or statistical process run by software to analyse data. It usually involves multiple calculation steps and can be used to automatically process data or solve problems. Cloud computing: Software or data running on remote servers, rather than locally. So instead of storing or computing things on your own machine, you can use other computers that are connected to your computer via a network (such as the Internet). Structured data: Any data or information located in a fixed field within a defined record or file, such as a database or spreadsheet. Its inherent structure makes it quick, easy and cheap to analyse. Unstructured data: All the data not easily stored and indexed in traditional formats or databases. It includes email conversations, social media posts, video content, photos, voice recordings, sounds and so on. Its lack of structure makes it more difficult to analyse using traditional computer programs. Semi-structured data: You guessed it, this is a cross between unstructured and structured data. It’s data that may have some structure that can be used for analysis but lacks the strict structure found in databases or spreadsheets. For example, a Facebook post can be categorised by author, date, length and even sentiment, but the content is generally unstructured. Internal data: This accounts for all the data your business currently has or could potentially access or generate in future. It could be structured in format (for example, a customer database) or it could be unstructured (conversational data from customer service calls). External data: Put simply, this is the infinite array of information that exists outside your business. It can be publically available or privately held and it can also be structured or unstructured in format. The Internet of Things: A network that connects devices (the things referred to in the name) so that they can communicate with each other. This encompasses technology like smart televisions, smart phones, and sensors, and it’s all possible thanks to the massive increase in connectivity between devices, systems and services.

Article / Updated 03-26-2016

Big data can help you gain insight. Businesses gain competitive advantage when the right information is delivered to the right people at the right time. This means extracting insights and information from data and communicating them to decision makers in a way they’ll easily understand. After all, people are less likely to act if they have to work hard to understand the information in front of them. Make sure your insights shine through with these top tips: Identify your target audience. Who your audience is depends on your strategic questions. The audience may be you if you’re the business owner, or it could be your human resources team, your marketing team or a combination. Ask yourself who’s going to see these results. What do they already know about the issues being discussed? What do they need and want to know? And, what will they do with the information? Customise the information for your audience. Be prepared to customise your information to meet the specific requirements of each decision maker. Remember what you’re trying to achieve. Try not to get distracted by interesting insights that have nothing to do with answering your strategic questions and achieving your business goals. There may be scope to revisit those other insights in future but, for now, focus on what you set out to achieve. Avoid creating a wall of text. Remember that data can be presented as a number, a short written narrative, a table, a graph or a chart. In fact, the best approach is likely to involve a combination of these formats. Use data visualisation techniques. Visuals are great for conveying information because they’re quick and direct, they’re (usually) easy to understand, they’re memorable and they add interest, being much more likely to hold the reader’s attention than a full page of text. But don’t neglect the text. Numbers, charts and visuals may only give a snapshot; narrative allows you to embellish on key points. Use short narratives to introduce what you’re showing and highlight the key insights. Use clear headings to make the important points stand out. This way, even at a quick glance, the key points will be obvious. Link the information to your strategy. If you’re presenting information that directly answers a strategic business question, such as ‘How do we reduce staff turnover by ten per cent?’, include that question in the opening narrative and maybe even the headline.