How to Gain the Skills You Need for a Career in Big Data - dummies

How to Gain the Skills You Need for a Career in Big Data

By Jason Williamson

Are you hoping to switch fields and go into big data? You need to figure out how you’re going to learn what you need to know to get your abilities up to the level required to land that great position. Some of the skills you need can be accomplished through formal training, do-it-yourself learning, or a combination of both.

When to fill the gaps with education

Gaining necessary big data skills through formal education depends on several key factors:

  • Your best fit: If you’re focused on becoming a data scientist, you’ll likely have to continue (or go back) to school for a master’s degree or PhD.

  • Time: Formal education takes time. The benefit is that you can easily track and predict when you’ll be able to fill your gaps with knowledge.

  • Cost: Training isn’t free, and not all training is equal. Carefully consider the opportunity costs of not going to school and calculate the expected return on investment (ROI) if you do. Time is an additional cost you must consider. What would you have accomplished for profit or pleasure if you chose not to go?

  • Job requirements: Many job postings require formal education and specific degrees as a prerequisite for applying. In highly sought-after jobs like top-tier consulting firms, the school you go to is also a contributing factor. For example, if you’re going to work in the field of data science, you need a degree in statistics or math.

  • Desire: Going to school or taking formal training takes commitment. It requires motivation over an extended period of time. If you’re currently employed, this will be increasingly difficult — the demands of life, family, and work come into play. You have to be a self-starter and need a support system in place to enable you to finish strong.

The decision to change majors, go back to school, or pay for specific training is not an easy one. Because of the fast-paced changes in the field, rather than get a new degree you can probably get the training you need to land a job by taking vendor training. Take time to think about each of these areas. The more data you have to reflect on, the better.

Filling gaps with experience

Many of the skills in the programming category can be accomplished through on-the-job training or self-directed education. This path is more easily accomplished if you’re already employed with a few years in technology under your belt.

Emerging technologies that are pervasive are still relatively new and don’t have a lot of formal training opportunities. The best way to learn them is often through hands-on projects or self-directed study. Conferences and trade shows and even the vendors often provide free hands-on boot camps with the objective to get people trained with enough skill to continue on their own.

The best place to get more experience is from the job you have. Also leverage your skills. For example

  • If you have ETL experience, this is directly related to the data component of big data.

  • If you have user experience (UX) skills, you can easily translate them for big data projects.

  • If you have database experience that can be translated into Hadoop or NoSQL. Oracle Exadata and AWS Redshift are relational databases to begin with.

Planning your milestones and timeline

The next step is to build a task list and write down your objectives as well as estimate the time it will take to get to the next level. This list will be based on your relative self-assessment. If you’re already at a 5 for Python, for example, you may feel that to get to level 6 you need to execute a real-world project at work, or something at home or school that could be used on a résumé.


For this planning worksheet, notice that the tasks are building upon one another to construct a set of skills that can be translated into new projects at work or something that you can talk about on a job interview.

As you look at this worksheet, you’re asked to estimate the number of days it will take to complete the training. Figure out how to allot the time based on an eight-hour day. Is this a few hours a day at work, a couple of hours in the evening, or full time focus all day for 90 days? It doesn’t really matter as long as you have a plan you are following and you understand how you will achieve it.

Fill out your worksheet so you can finish your project in a prescribed amount of days. Also notice that the last task is a task to evaluate what you’ve done.

Measuring your results

When you hit your final milestone, don’t forget to do the evaluation step. Many projects skip this step. Big mistake. Spend some time thinking about not only what you learned but how you learned. Was it effective? Did you accomplish your goals? What would you do differently? Project managers and team members have a tendency to skip this important step for several reasons:

  • You don’t think you can spare the time to look back. The reality is that you can’t afford not to take the time. Otherwise you run the risk of making the same mistakes over and over. Reflections provide a framework to identify those errors and improve.

  • You think it’s too expensive — time is money. Lessons learned make you more efficient and profitable in the future.

  • You’re afraid of being held accountable. If you don’t meet your goals, it isn’t fun to sit around and talk about it. Rather than take a posture of blame, use the time to figure out how to avoid the same trouble again in the future.

The takeaway is to spend the time it takes to measure your results and reflect on the overall learning process.