Do You Have the Necessary Skills for a Career in Big Data?

By Jason Williamson

People who choose to seek a career in big data come from many different backgrounds. In such a growing field, a diverse knowledge base is desirable. But, do you have the skills necessary for a big data career? Read on to find out.

Evaluating your aptitude for big data

You start by figuring out if you’re a good fit for big data. The real question to answer is: What type of role do you want to pursue and how you are going to get there? Here is a short overview of big data roles and some questions to ask yourself as you consider them:

  • Software developer: This role is for programmers who largely support implementing big data solutions through software development. The jobs that fit these roles vary within an organization based on specific job functions. In traditional programming roles, developers are expected to focus on one, two, or maybe three languages in their day-to-day job.

    More often than not, it’s just one job, like a C++ programmer or a Java programmer. Big data programmers are usually expected to be skilled in many languages and work at both the logic and data layers.

    Big data software developers have to be fluid and able to adapt to the fast pace of change. Can you shift gears between writing SQL, software logic, and back in three different languages in the same day, or do you like to be head down and focus on only a few tasks?

  • Business analyst: Business analysts usually craft and answer questions to drive new insights, revenue, or costs savings for their organization. These people can be hired directly by the technology teams to help bridge the gap between the business owners and the technical developers.

    Big data business analysts are expected to not only do deep traditional business analysis, but also have technical understanding, as compared to their non–big data business analyst colleagues.

    Can you see problems from several angles? Are you able to get to the root cause of consumer behavior? Do you see the tie-in with information and business value? Can you build and execute a business model? Can you navigate between the technical and the business easily? Are you good in presentations to executives? Can you write well?

  • Data scientist: This role tends to be highly mathematically and statistically oriented. Data scientists usually have advanced degrees and are involved with designing complicated modeling, advanced algorithms, and applied math. If advanced statistics, mathematics, and number theory are your thing, the role of data scientist is for you!

    You should have a very high math aptitude if you want to be a data scientist. Do you already have, or are you working toward, a math or statistics degree? Have you dreamed of winning the Fields Medal since before you could talk? If you don’t know what the Fields Medal is, you probably aren’t meant for this job.

Here are the four areas you need to consider in the following order: aptitude evaluation, skills self-assessment, plan to fill gaps, and execution plan. For example, when you have an understanding of your aptitude, you can begin to do a self-assessment so that you know what your education plan should be.

Doing a self-assessment plan

This part of the process may require you to conduct some research and additional reading if you aren’t fully informed. It may take some time to write down the skills you have and the ones you need based on your review of job descriptions and discussions with people in your field.

Rate yourself on a scale of 1 to 10, with 1 being “I’ve heard of it” to 10 being “I could write a book on the subject.” A score of 5 means you’re capable of executing that skill at a reasonably acceptable level in a professional setting — you won’t be a rock star, but you’ll get the job done.


The skills assessment worksheet you see below is not comprehensive. However, a fair evaluation will give you a good idea about what you’ll need to know and how you may go about building a plan for getting that job.

You need to understand your baseline numbers in relation to the amount of experience you have today. If you’re a professional programmer with ten years of Java, you may enter 8 for Java and 3 to 5 on other programming skills. The key is to pick a value that you know represents your level of understanding. Then you can compare the other values in relation to that.

It’s always a challenge to take subjective value and assign an objective number to it, but simply going through the exercise gives you a point of reference. It will be different for everyone. If you’re a new grad or in the middle of college, your answers may only peak at 5 based on what you’ve been exposed to in school.


Finding your gaps

At this point in the process, you need to identify what’s required of you to gain a job. This depends on two critical factors:

  • Where you are in your professional career: If you’re in your early 20s and you’re expecting a relatively junior role, you should work toward getting your numbers up to a 5+. If you’re farther along and you want a leadership role or a senior developer role, the numbers have to be higher.

  • Your shortcomings: Highlight those areas that are a 5 or below. Remember to think about this with respect to your relative potential strength given where you are today in your professional career. Those highlighted skills are what you need to work on to get that job.