2016 open source and big data market report_small

40
MARKET REPORT 2016 OPEN SOURCE & BIG DATA

Upload: tamara-duran

Post on 22-Jan-2018

97 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 2016 Open Source and Big Data Market Report_small

MARKET REPORT 2016

OPEN SOURCE & BIG DATA

Page 2: 2016 Open Source and Big Data Market Report_small
Page 3: 2016 Open Source and Big Data Market Report_small

CONTENTS

INTRODUCTION ...................................................4

DEMOGRAPHICS .................................5Gender .........................................................................6Age ...............................................................................6Location ........................................................................6Profession .....................................................................7Working Week ...............................................................8

MARKET PERSPECTIVE ......................9Staff .............................................................................10

FULL TIME EMPLOYMENT ................11Compensation .............................................................12Length of Tenure ..........................................................13Benefits .......................................................................14

CONTRACT EMPLOYMENT ...............17Compensation .............................................................18Benefits .......................................................................19

CAREER MOVES .................................20Motivation ....................................................................21Current ........................................................................23Future ..........................................................................25

SALARY SUMMARY ...........................27

NEWS & INSIGHT ................................32Why & How to Get Started in Open Source .................. 33You Might Not Need a Data Scientist ............................ 34The Future of Technology .............................................. 354 Data Trends to Watch in 2016 ................................... 36

Page 4: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

4

INTRODUCTION At the end of 2015, we surveyed IT professionals about their careers and the current job market conditions. These findings allow us to present you with clear insight into key trends and drivers in the IT market.

This report provides some of the most comprehensive information available. We hope this data allows job seekers to uncover their secret earning powers, and that it protects employers against a powerful business kryptonite: lack of knowledge about compensation trends.

Thank you to everyone who took part in the survey.

Ben Weber, Managing Director [email protected]+1 425 460 4285

In Q4 2015, Greythorn conducted its fifth survey with its candidates focused on the demographics, length of tenure, working week, job security, career motivations, department changes, reward and bonus trends of IT professionals. 611 technology professionals responded to the survey, helping us determine key trends within the market for you.

This market report is based on the data received through the online survey, as well as the expert opin-ions of our experienced recruiters.

Please contact us if you would like to receive a report tailored to your business needs.

[email protected]+1 425 460 4285

METHODOLOGY

Page 5: 2016 Open Source and Big Data Market Report_small

DEMOGRAPHICSGender .........................................................6Age ...............................................................6Location ........................................................6Profession .....................................................7Working Week ...............................................8

Page 6: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

6

DEMOGRAPHICS

73% 21%Male Female

Where is your role located?

59%

10%

2%

11%

16% 1%

West Coast

Southwest

West

Southeast

MidwestNortheast

2% 19%26% 5%30% 18%18 - 24 years old

43 - 50 years old

35 - 42 years old

60+ years old

25 - 34 years old

51 - 60 years old

NOTE: 6% of respondents preferred not to say.

Page 7: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

7

Which of the following technologies are you currently working with?

34%Linux

30% 13%VMWareJavascript

13% 17%HadoopOracle

21% 13%MySQL Ajax

17%15%XMLNode.js

13% 31%C++ HTML5

31%Java

12%Android

33%LinuxSQL

12%.NET

11%MongoDB

Python

12%C#

32%

15%jQuery CSS3

14%

DEMOGRAPHICSPROFESSION

Page 8: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

8

DEMOGRAPHICS

WORKING WEEK How many hours per week do you work on average?

Which of the following best describes your role?

15%Software Development

11%Architect

9%Engineer

8%Big Data/Analytics

7%Project Management/ Business Analysis

6%Executive

6%Strategy/Consulting

5%DevOps

4% 4%Data ScientistProduct

Management

2%Less than 20 hours

1%20 - 30 hours

23%31 - 40 hours

38%41 - 45 hours

22%46 - 50 hours

6%51 - 55 hours

4%56- 60 hours

4%More than 60 hours

Page 9: 2016 Open Source and Big Data Market Report_small

MARKET PERSPECTIVE

Page 10: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

10

STAFFHow are you currently employed?

How are you feeling about the future of the job market over the next 12 months?

1%Extremely pessimistic

7%Pessimistic

10%Indifferent

57%Optimistic

24%Extremely optimistic

81%Full time

19%

Contract

MARKET PERSPECTIVE

Page 11: 2016 Open Source and Big Data Market Report_small

FULL TIME EMPLOYMENTCompensation ......................................12Length of Tenure ...................................13Benefits ................................................14

Page 12: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

12

COMPENSATION

What level is your current base salary?

6% 7%

14% 14% 16%

6%9%

28%

Less than $50,000

$50,000 - $69,999

$70,000 - $89,999

$90,000 - $109,999

$110,000 - $129,999

$130,000 - $169,999

$170,000 - $200,000

More than $200,000

5%Yes, more than 20%

16%No

24%Yes, between 6% and 10%

6%

49%

Yes, between 11% and 20%

Yes, up to 5%

Would you expect a salary increase over the next 12 months in your current role?

FULL TIME EMPLOYMENT

Page 13: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

13

LENGTH OF TENURE

How long have you been in your current role?

7%

9%

17%

16%

8%

0 - 6 months

More than 10 years

1 - 3 years

3 - 5 years

65- 10 years

6 - 12 months

33%

Although there are exceptions, it is usually best to stay in a role at least 2-3 years to avoid the “job hopper” label.

FULL TIME EMPLOYMENT

Page 14: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

14

FULL TIME EMPLOYMENT

BENEFITS

Share incentive/stock options

Which benefits are included in your current package?

Life insurance

25 days PTO/holiday or more

Vision insurance

Flex benefits

Car/transportation allowance

Educational reimbursement

Dental insurance

Other

401k match

None

85%

90%

61%

35%

74%

78%

61%

65%

50%

24%

41%

2%

• 62% of respondents have the option of working remotely.

• 32% of respondents get a gym membership/fitness and wellness program as part of their benefits package. • 20% of respondents have their equipment purchases reimbursed.• 12% of respondents utilize a subsidized meal plan.

Medical insurance

Bonus Do you receive any additional perks?

5%

Page 15: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

15

Did you receive a bonus in 2015?

FULL TIME EMPLOYMENT

57%Yes

43%No

What is your bonus calculated on?

11% Company performance

26% Personal performance

63% Both company and personal performance

Page 16: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

16

FULL TIME EMPLOYMENT

As a percentage of your base salary, what level was your last annual bonus?

28%

27%

13%

14%

9%

4%

5%

0 - 5%

6 - 10%

11 - 15%

16 - 20%

21 - 30%

31- 40%

More than 40%

• A greater number of people reported in 2015 that they earned a bonus that equaled 10% or more of their base salary. Last year, that number was 44%; this year it was 48%.

18% 64% 12%Highly satisfied Satisfied Dissatisfied

Were you satisfied with your bonus?

7%Highly dissatisfied

Page 17: 2016 Open Source and Big Data Market Report_small

CONTRACT EMPLOYMENTCompensation ..................................18Benefits ............................................19

Page 18: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

18

LENGTH OF TENURE

How long have you been in your current role?

14%

12%

15%

17%

8%

0 - 6 months

More than 10 years

1 - 3 years

3 - 5 years

5 - 10 years

6 - 12 months

34%

CONTRACT EMPLOYMENT

Less than $40 $40 - $50 $51 - $60 $61 - $70 $71 - $80 $81 - $90 $101 - $125$91 - $100 More than $125

11%

COMPENSATION

What level is your current hourly rate?

15%10% 12%

17%

12% 10%6%7%

Page 19: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

19

CONTRACT EMPLOYMENT

BENEFITS

Bonus/retention bonus

Do you receive any additional perks?

Do you receive benefits as part of your package?

Dental insurance

Vision insurance

None of the above

Car/transportation allowance

Life insurance

Education reimbursement

Medical insurance

Flexible benefits

401k match

25 days PTO/holiday or more20%

76%

20%

64%

24%

16%

60%

24%

12%

48%

• The number of contract workers who saw benefits packages included in their compensation jumped by 31% since last year’s survey, from 22% up to 32%.

32%

68%

• 64% of respondents can work remotely.• 52% of respondents take advantage of flex time.• 16% of respondents get their equipment purchases reimbursed.• 20% of respondents eat company-provided meals and snacks.• 16% of respondents receive gym memberships or access to fitness and wellness programs.

8%

Page 20: 2016 Open Source and Big Data Market Report_small

CAREER MOVESMotivation .............................................21Current .................................................23Future ...................................................25

Page 21: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

21

CAREER MOVES

How vital are these employee perks when considering a new role?

Bonus

Vision insurance

401k match

25 days PTO/holiday or more

Share incentive/stock options

Education reimbursement

Gym membership/fitness and wellness program

Dental insurance

Life insurance

49%

80%

29%

19%

37%

46%

30%

34%

27%

12%

23%

Remote work

Medical insurance

Page 22: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

22

CAREER MOVES

More than 50% increase3%

Why did you leave your previous role?

Job security

3%

Travel

1%

Career development /advancement

23%

End of contract/assignment4%

Higher salary6%

Not applicable/first job

1%

Internal promotion

1%

Questionable ethics

1%

Relocation5%

Work/life balance4%

Company sold

Laid off

9%

New challenge / more interesting work

13%

Management/leadership

11%Commute/proximity2%

Company mission & values3%

Improved relationship with colleagues

1%

41-50% increase1%

31-40% increase

3%

21-30% increase

12%

16-20% increase

17%

11-15% increase

27%

6-10% increase

27%

1-5% increase

7%

0% increase

4%

What would be an acceptable increase in your base compensation for your next role?

3%

Page 23: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

23

CAREER MOVES

CURRENT

How would you assess your level of satisfaction with your current role?

Management/leadership Job description/responsibilities Benefits

Culture Other Salary

37% 24% 2%

14% 13% 9%

Reasons for dissatisfaction

24%Highly satisfied

20%Dissatisfied

52%Satisfied

4%Highly dissatisfied

Page 24: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

24

CAREER MOVES

Awaiting bonus payment

Main reason

Second reason

Tertiary reason

Who or what inspires you in your current role?

Flexibility

Challenge

Career progression

20 40 600

• Bonuses are motivating to IT professionals, but they’re more inspired by interesting workplace challenges, opportunities for career progression, and working with a likable and capable team. Colleagues/

boss

Enjoyment

Company mission & values

80 100

Page 25: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

25

FUTURE

How likely are you to change organizations in the next 12 months?

18% Very likely

22% Likely

37% Unlikely

24% Very unlikely

CAREER MOVES

Page 26: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

26

CAREER MOVES

What are your key priorities when considering a new job?

Career development/advancement

Company mission & values

More responsibility/accountability

New challenge/more interesting work

Job security

Improved benefits package

Bonus potentialImproved relationship with manager/colleagues

Commute/proximity

Higher salary

Relocation Internal promotion

End of contract

Training

Work/life balance

Main priority Second priority Tertiary priority

100

100

100

80

80

80

60

60

60

40

40

40

20

20

20

%0

0%

0%

Page 27: 2016 Open Source and Big Data Market Report_small

SALARY SUMMARY

Page 28: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

28

SALARY SUMMARY

OPEN SOURCE & BIG DATA

BUSINESS INTELLIGENCE & DATABASE

Job title Full time Contract hourly

Min Max Min Max

Junior Software Engineer $80,000 $110,000 $75 $90

Mid-level Software Engineer $115,000 $140,000 $90 $110

Senior/Lead Software Engineer $140,000 $185,000 $120 $145

Principal Engineer $160,000 $200,000 $125 $170

Junior Data Scientist $120,000 $140,000 $95 $120

Senior Data Scientist $145,000 $180,000 $105 $140

Architect $135,000 $150,000 $110 $130

Chief Architect $155,000 $200,000 $130 $180

DevOps Engineer $110,000 $150,000 $70 $110

Software Engineer, Test $90,000 $150,000 $70 $125

Job title Full time Contract hourly

Min Max Min Max

Business Intelligence Analyst $90,000 $120,000 $75 $90

Business Intelligence Developer $100,000 $125,000 $95 $115

ETL Developer $95,000 $120,000 $90 $110

Database Developer $100,000 $130,000 $95 $120

Database Administrator $90,000 $120,000 $85 $110

Data Warehouse Architect $125,000 $150,000 $120 $140

NOTE: Add an additional $20,000 to full time salary if candidate is an open source contributor.

NOTE: Add an additional $20,000 to full time salary if candidate is an open source contributor.

Page 29: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

29

SALARY SUMMARY

INFORMATION SECURITY

MICROSOFT STACK

Job title Full time Contract hourly

Min Max Min Max

Sr. Management and Executive $140,000 $200,000 N / A N / A

Sr. to Principal Security Engineer $100,000 $150,000 $90 $160

Security Architect $90,000 $130,000 $85 $110

Job title Full time Contract hourly

Min Max Min Max

System Administrator / Network Administrator $70,000 $100,000 $50 $80

Systems Engineer / Operations Engineer $70,000 $120,000 $60 $100

Network Engineer $80,000 $130,000 $70 $115

Software Developer $85,000 $160,000 $75 $120

QA Engineer $80,000 $130,000 $65 $105

Project / Program / Product Manager $85,000 $130,000 $70 $120

Data Warehouse Architect $100,000 $135,000 $120 $160

SALARY SUMMARY

NOTE: Add an additional $20,000 to full time salary if candidate is an open source contributor.

NOTE: Add an additional $10,000 to full time salary if certifications such as CISSP are required.

Page 30: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

30

SALARY SUMMARYSALARY SUMMARY

Job title Full time Contract hourly

Min Max Min Max

Front End Developer $80,000 $140,000 $70 $130

PHP Developer $100,000 $125,000 $90 $105

UX Designer $80,000 $135,000 $90 $130

UI / UX Developer $90,000 $140,000 $85 $130

Web Development Manager $100,000 $140,000 N/A N/A

WEB DEVELOPMENT

Page 31: 2016 Open Source and Big Data Market Report_small

Why & How to Get Started in Open Source ................................ 33You Might Not Need a Data Scientist ....................................... 34The Future of Technology...............354 Data Trends to Watch in 2016..........................................36

NEWS & INSIGHTS

Page 32: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

32

NEWS & INSIGHT

I fell in love with the open source community when I went to my first OSCON (Open Source Conference). It was 2012 and after years of working in staffing and HR, I took a job in big data recruiting. I’d only been learning the ropes for a few months before OSCON and was impressed by how many brilliant people were all in one place. Even more impressive was that everyone was willing to share what they knew.

It wasn’t because they were trying to sell anything, but because they were so passionate about what they were working on. Four years later, and I can say with confidence that my experience was no fluke—this passion and collaboration are typical for the open source community.

WHY YOU SHOULD GET INVOLVEDFrom my perspective, it’s easy to see the advantages of giving back to the community you work in. My clients tell me that they want someone who has an exceptional technical

mind, but they also want a candidate who really likes this stuff. When you are really passionate about something, you do it even when you aren’t getting paid. Clients often ask: Do they code in their spare time? Can I find their work anywhere? What do they really enjoy? Being involved in open source helps demonstrate the passion employers are looking for.

HOW TO GET STARTEDSo how do you get started in open source? There are many ways and quite a few organizations, but I suggest checking out The Apache Software Foundation (ASF). Visit their website—it gives a ton of instructions on how to get started. You will also want to pick a project that you are passionate about and consider doing research about the project members. After all, they are the ones who will be helping you. Then, join the mailing list, and start paying attention to how everyone collaborates and what goals they’re trying to accomplish; jump in when you’re ready with something to contribute. If you want the speed dial version, attend ApacheCon—you’ll find yourself in hallway conversations about all kinds of cool projects, with many chances to get involved!

The ASF also has a mentorship program that you can apply to if you don’t like the ambiguity of figuring things out on your own. If you are currently in college, you can also apply to Google Summer of Code. They

work with multiple open source communities and partner you with a mentor. This is a great opportunity to get a leg up on your technical skills, as well as grow your professional network.

BOTTOM LINEGet involved in open source, whether you’re a newbie or an old hand. I promise that your job searches will become much easier in the long run!

Lindsey Thorne Manager Open Source & Big Data Practice

WHY AND HOW TO GET STARTED IN OPEN SOURCE

Page 33: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

33

NEWS & INSIGHT

Data science has gained tremendous momentum over the last few decades, but it has also led to some ambiguity around the differences between data engineers and data scientists. We’ve all seen statistics showing how few data science professionals there are compared to open positions, but it’s hard to know how much credence to give this data when every hiring manager has a different definition of what it means to be a data scientist. So before you worry about how you’ll ever snag one, let’s determine if you need one in the first place.

KNOW WHAT YOU NEEDKnowing exactly what you need (not necessarily want) is key to hiring right the first time and accessing the appropriate talent pool. It seems obvious, but unfortunately it’s not. Hiring managers often call us proclaiming they want a data scientist, when

it’s clear they need a data engineer. Sometimes, they want a data scientist on staff, but haven’t considered what they need that person to do and are instead leaving it to their future employee “to define the data initiatives.” What they fail to realize is that if we can figure out that they don’t know what they want, so can their pool of candidates. Once candidates have talked to a few friends and colleagues, the organization has cemented a reputation it can’t undo, one that harms their future hiring prospects.

It’s never a good idea to use the interview process to determine what you’re looking for. It’s a time for you to find the right person based on what you need. Don’t sacrifice your reputation and limit your talent pool due to a lack of preparedness.

AVOID HYPEHow do you clearly define what you want when even industry experts don’t agree? First and foremost, avoid hype at all costs—hire for your needs and your needs alone. Once you accept that mantra, the easiest way to answer the question is to break it down as simply as possible: Do you need someone who’s going to code 80% of the time? If so, that’s not a data scientist. If you call it a data scientist, and give data scien-tists engineering-focused coding challenges, you are not going to find the type of talent you need and will quickly damage your reputation in the community.

You may be thinking, “But now I think I need someone with both skill sets.” A word of caution: much like we rarely find full-stack engineers who are equally skilled (and passionate!) in the backend and the frontend, it is just as difficult to find a data scientist who is equally skilled in engineering and analytics. If you decide this is absolutely necessary, it’s not impossible, but I highly recommend giving yourself a few extra months for your search.

GET WHAT YOU WANTOnce you know exactly what you want, the final step is to work with trusted recruiters (whether internal or external) to actively search for the right fit. After all, your perfect candidate probably isn’t looking at job postings every month—so you need to go find them!

In short, if you clearly know what you want, take care of your reputation, and work with trusted advisors, you’ll have better access to the type of talent you want to attract and who want to work with you.

Mary KypreosRecruiting ManagerOpen Source & Big Data Practice

YOU MIGHT NOT NEED A DATA SCIENTIST

Page 34: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

34

NEWS & INSIGHTNEWS & INSIGHTTHE FUTURE OF TECHNOLOGY

I’m excited about developing and consuming emerging technologies, which continue to drive down the barrier to entry for undertaking data analysis over increasingly growing datasets. Within the domain of Scientific Instrumentation Software and Science Data Systems at NASA JPL, we are working hard to further state-of-the-art solutions to solve big scientific, social, environmental, ecological and exploratory questions.

Lewis John McGibbneyData ScientistNASA Jet Propulsion Laboratory & Apache Software Foundation Member

As we continue to abstract away the concept of resources from the machines via containers and clustering technologies, such as Docker and Apache Mesos, we are starting to see a first: having powerful mechanisms for cost optimization working in concert with, and even facilitating, more rapid innovation and project delivery.

Nate McCallCo-FounderThe Last Pickle

Databases and data technologies are going to be turned upside down in the near future as disruptive memory technologies make CPU the new bottleneck and not I/O. Newer storage technologies, such as FiloDB, will take advantage of this trend.

Evan ChanDistinguished EngineerTuplejump

Over the last five years I have seen a change in the mindset from fighting with CISOs, CFOs and CEOs on why you would ever consider Open Source Software to now witnessing the firing of a CIO for refusing to embrace it. It has been an amazing change and with the technology innovation driven by OSS, and it is only speeding up. It is a great time to be in the space and an amazing time for technical innovation.

Eddie SatterlyCo-FounderDataNexus Inc.

We asked a group of leaders and innovators about the changes they’ve observed in the technology industry, and where they saw it going in 2016.

To anyone who has been working in tech for many years, it’s no surprise that Open Source Software continues to power technical innovation. What I think people forget is that it’s also a great way to fuel your career and invest in portable skills.

Sarah NovotnyKubernetes CommunityGoogle

This survey is amazing validation that not only is OSS here to stay, but it’s also a great career choice. As more companies rely on OSS to make their products work, this is only going to continue and grow.

Patrick McFadinChief EvangelistApache Cassandra

Page 35: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

35

NEWS & INSIGHT

Paco Nathan is the Director of the Learning Group at O’Reilly Media, and below he shares in his own words the four big data trends worth watching in 2016.

TREND #1Spark and other contemporary architectures are moving outside of traditional JVM con-straints, leveraging off-heap memory, multi-core CPUs, integrating GPUs, etc. Hardware and OS now advance faster than production software, so the popular frameworks must struggle to leverage what they can at a low level. Kudos to Typesafe, Intel, and NVidia for leading this.

TREND #2So much of the real-world big data can be represented as complex graphs, time series, geospatial, etc. We spend a lot of time cod-ing to make those problems “fit” into matrix libraries, when in fact the general case for many data problems are tensors. Moreo-ver, our predictive analytics, ML models, etc., inevitably end up within the context of control systems: Uber is a ginormous control system, and so are Airbnb, AdSense, and Twitter. Tensor factorization libraries built atop robust distributed systems would wipe out massive numbers of libraries: the data crunching and the adjustments for control

systems parameters can be performed at the same time, also with our current sense of linear constraints being relaxed. This is what UC Irvine (anima Anandkumar) is doing in collaboration with AMPLab.

TREND #3To paraphrase Doge circa 2015, much IDE, such API. On the other hand, workflows are systems that bring together people and automation. Not everything needs to fit into an API; in fact, I believe we’ve swung way too far in that kind of expectation. Project Jupyter is about architecture and social systems, how people collaborate—outside of IDEs and scrum meetings—by extending the semantics of REPLs for many program-ming environments. It takes a cue from the semantics of Git, and provides a rev beyond the mechanisms that HTTP gave the world. Watch carefully as Jupyter expands, and learn to think (and work with people) outside of IDEs and APIs.

TREND #4Watch the space studied by Chris Ré, where cognitive computing (and likely with that, probabilistic programming) makes a leap ahead of how people leverage data and au-tomation. Expect that to accelerate. If you’re stuck programming in Java on Hadoop clusters with Intellij, brace yourself. At least go learn Java 9, so you won’t be completely passed up.

4 DATA TRENDS TO WATCH IN 2016 Increased education into the applications of data science and machine learning will break us from industry teams focused on social science problems, and into the vastly exciting real of data science products for sensors, signals, and systems. We will start to really use the aca-demic and professional experience of all those engineers, physicists, and mathematicians converting to our field. Cross-functional teams of developers, engineers, designers, and data scientists will infiltrate traditional science laboratories, engineering firms, and non-profits to work with experts on how we can help advance their work.

A.M. CasariSenior Data ScientistConcur Technologies

In the data space, we are seeing ecosystems develop of partners sharing data and insight to their mu-tual benefit. For example, Starwood and Uber have teamed up to share data to better understand how to meet the needs of travelers. This has implications in terms of the skills and technologies that data engineers will need (APIs, micro services, etc.), the technologies that will experi-ence rapid growth as a result (Kafka, Cassandra, etc.), and the investment that enterprises need to make to compete.

John AkredChief Technology OfficerSilicon Valley Data Science

Page 36: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

36

ABOUT US

Greythorn is a specialty technology recruitment agency, and our Open Source & Big Data Practice primarily focuses on working with clients to find the specialized IC and leadership talent they require. Our reputation is founded on the trusted relationships we’ve built with the industry’s thought leaders, ranging from Apache Foundation Committers and PMS Members to technical text authors and leading edge code creators. You can find us at ApacheCon, OSCON, Strata, the Cassandra Summit, regional Meetups, and more.

Whether our clients need an experienced data engineer, data scientist, or an open source committer, we’ve built relationships with quite a few of those unicorns.

ABOUT US

@GreythornNA

Sub Groups: • Open Source & Big Data• Healthcare IT• LIVESite

SEATTLE

40 Lake Bellevue DriveSuite 200Bellevue, WA 98005

+1 425 635 [email protected]

Greythorn North America

Page 37: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

37

ACKNOWLEDGEMENTS

This research was carried out by means of an electronic questionnaire and supplemented with data and market information from Greythorn and other industry experts. The results are provided as generic market information only.

Greythorn does not make any warranties regarding the use, validity, accuracy or reliability of the results and information obtained. Greythorn will not be liable for any damages of any kind arising out of or relating to the use of this information.

We would like to thank all those who completed our survey—without your contribution, this market report would not be possible. If you did not complete our survey this time, please do so next year to ensure we can continue to develop the depth and quality of this report, and continue to provide you with a fair and balanced picture.

This market report is intended to give you a very general overview on the changing nature and complexity of the IT employment market; for a more tailored and confidential discussion on how this will affect your business or your own career, please get in touch.

ACKNOWLEDGEMENTS

DISCLAIMER

The Inside Technology Market Report was made possible and brought to you in part by our valued partners.

OUR PARTNERS

Page 38: 2016 Open Source and Big Data Market Report_small

INS

IDE

TE

CH

NO

LOG

Y M

AR

KE

T R

EP

OR

T 2016

38

ACKNOWLEDGEMENTS

For every response to our survey, we donated $1 to the Apache Software Foundation. This organization offers support for the Apache community of open source software projects, which are defined by collaborative consensus-based processes; an open, pragmatic software license; and a desire to create high-quality software that leads the way in its field.

The Apache Software Foundation is made up of more than 150 top-level projects, which cover a wide range of technologies. We are proud to support an organization providing open source software products for the public good.

THE APACHE SOFTWARE FOUNDATION

Page 39: 2016 Open Source and Big Data Market Report_small
Page 40: 2016 Open Source and Big Data Market Report_small