Big data skills are in short supply. As the amount of digital information generated by businesses has grown exponentially, a challenge (some people even call it a crisis) has arisen: there just aren’t enough people with the necessary skills to analyse and interpret all this big data. In one recent survey, more than half of business leaders questioned felt their ability to carry out big data analytics was limited by the challenge of finding the right talent.
More and more courses are springing up to meet this skills shortage and big data is undoubtedly becoming a desirable career route for college leavers. But it’ll take time for the number of qualified people to catch up to the sheer demand for big data skills. So, at least for the next few years, the big data skills shortage is a problem that all companies interested in big data (which should be all companies) will have to face.
With stiff competition to attract the best talent, companies are turning to creative ways of tapping into big data skills. Walmart, for example, decided to apply the power of the crowd, turning to crowdsourced analytics competition platform Kaggle. At Kaggle, armchair data scientists apply their skills to analytical problems submitted by companies, with the designer of the best solution being rewarded – sometimes financially or, in the case of Walmart, with a job.
In Walmart’s first competition, which took place in 2014, candidates were given a set of historical sales data from a sample of stores, along with associated sales events, such as clearance sales and price rollbacks. They were asked to come up with models showing how these events would affect sales across a number of departments. As a result of the competition, several people were hired into the analytics team.
Best of all, this crowdsourced approach led to some interesting appointments – people who may not have been considered for an interview based on their CVs alone. One appointee, for example, had a very strong background in physics but no formal analytics background.
What does this mean for smaller businesses? Even if you can afford to hire an in-house data scientist, you may find yourself up against fierce competition from bigger companies. The Walmart example shows us that, in order to tap into big data skills, you may need to get a little creative. Maybe you, too, could crowdsource data projects (even if the end result is a simple financial reward, as opposed to a full-time job).
Or perhaps you could partner with a local university or college, wherein students crunch your data in return for some business mentoring. Or maybe you already have strong analytical thinkers and communicators in your business who, with a little extra help and training, could set up and run big data projects in the future.