Data productsOne feature at a time

May 4th 2016 by Nataliya Nadtoka

About Centro

http://www.centro.net/ image above belongs to Centro

Change the World one ETL at a time

made with

What I think I do

What I really do

DATA BUGDATA BUGFEATUREBUG

Challenges in Building Data products

• Business - related• Technical• Non - technical

Disclaimer: opinions expressed in this presentation are my own. They do not reflect the situation at any of my current/former/future employers.

Business - related challenges

• Lack of predictability of the outcome of Data Science/ML projects

• “Just throw some data at it” approach• Scale and fault tolerance may be a solved

problem, but it comes with a hefty price-tag• Moving fast versus focusing on data quality

and then paying big for large unusable data

http://www.mammothdb.com/the-big-data-landscape-by-mammothdb/

It’s only a partial picture, there is more to it

• Too much choice

• Not everything is enterprise ready

• Tech is a moving target

Technical challenges

Technical challenges

Lack of standardized data formats (IOT)

Technical challengesBuilding first, thinking of security and/or scalability later

I wish I thought

of it!

Non - technical challenges

• Requires a strong advocacy and investment from the leadership

• Lack of focus “Too many great ideas”

• The quest for Data Scientist aka the Unicorn

I’m not saying that unicorns don’t exist…

Non technical challenges

• The dream of Agile Data Science• Regulations of data ownership differ by

country• Data ownership is still being defined

Some solutions

• Build multidisciplinary Data team instead of searching for unicorns

• Ruthlessly prioritize ideas• Focus on actionable data, not on data size• Communicate across the organization

Some solutions

• Keep an eye on the budget• Build with necessary security/scalability from

day one• Stay informed with trends, but take decisions

that work for your organization

Conclusions

• Keep your focus• Design with necessary security/scalability in

mind• Communicate

Do we have time for questions?..