big data public private forumbig-project.eu/sites/default/files/big_big data ppp_nuria...big big...

BIG

Big Data Public Private Forum

THE BIG DATA VALUE PPP

BIG Final EventSeptember 30, HeidelbergNuria de Lama, Representative of Atos Research & Innovation to the ECBig Data Public Private Forum responsible

BIGBig Data Public Private Forum

2BIG Final Event, Heidelberg, 30th Sept 2014 BIG 318062

CONTENTS

• The Big Data Value PPP in a nutshell

• Process and timing• SRIA: Innovation concepts• Implementation aspects, Structure & Governance model



CONTENTS





THE BIG DATA PPP

• The goal of the Big Data Public Private partnership is to increase the amount of productive European economic activities and the number of European jobs that depend on the availability of high quality data assets and the technologies needed to derive value from them.

European cross-organizational and cross-sector environments

Meeting point for different stakeholders (small, big companies, academia…supply & demand) to discover economic opportunities based on data integration and analysis Resources to develop working prototypes to test the viability of actual business development

Availablity of data assets (secure environments to enhance data sharing; i.e. not only open data)Technologies to derive value from them (this could entail bringing analytics close to data)



WHY TO JOIN? VALUE PROPOSITION

• Technology and data assets development• Means for benchmarking and testing

– performance of some core technologies (querying, indexing, feature extraction, predictive analytics, visualization…)

– business applications evaluated according to different criteria (ex. usability)

• Development of business models– Optimizing existing industries– New business models along new value chains

• Improvement of the skills of data scientists and domain practitioners (enrich educational offering)

• Dissemination of best practices showcases to stimulate big data adoption and transfer of solutions across sectors

• Analysis of societal impact transfer of data management practices to domains of societal interest (health, environment…)



CONTENTS





DRIVERS: NESSI & BIG

NESSI PartnersNESSI Membership

> 450 members



PREPARATION TIMING



HIGH-LEVEL TIME PLAN

Launch of the BDV PPP



WORKSHOPS & CONSULTATION ON BDV SRIA JAN-MAY 2014

• Workshops organised on Energy, Manufacturing, Retail, Mobility, Health, Media, Content, Geospatial/Environment, Public Sector/SMEs to feed into SRIA–More than 80 organisations

• Open consultation on the SRIA draft ran between 9 April and 5 May 2014–More than 2700 downloads of the SRIA–More than 100 responses to the Public consultation questionnaire

–Various separate feedback via email etc.



RESPONSES TO QUESTIONNAIRE / SRIA

Consultation/ questionnaire•University of Zurich, Binarypark, Hitachi, Hitachi Data Systems, Locali Data, Information technologies Institute (ITI), TNO Innovation for Life, Sintef Petroleum Research, OpenData.sk / Utopia + EEA, OKFN Belgium, My Digital Footprint, GC Venture Consulting (formerly IceStat), SAP Deutschland AG & Co. KG, Technical University of Cluj-Napoca, Statistics Netherlands, Oxford Sustainable Development Enterprise (EEIG), Paradigma Tecnológico, DialogCONNECTION Ltd, Wolters Kluwer Deutschland GmbH, Telefonica, Paluno, Fuhrwerk, Euroalert, ICM, University of Warsaw, Numeral Advance, Karlsruhe Institute of Technology, ELIXIR, Fundacion centro tecnoloxico de telecomunicacio ns de Galicia, TU Delft, Universidad Lyola Andalucia, CGI Nederland B.V., TECNALIA RESEARCH & INNOVATION, Engineering Ingegneria Informatica, Comunidad de Madrid, TU Dortmund, RENAM, Telefonica 2, GISIG, ESTeam AB, CERTH-ITI, Universitat Politècnica de València, inmark, Orange, data2knowledge gmbh, WU Wien, Fraunhofer, Zurko Research, Surveying and mapping authority of the Republic of Slovenia, ELRA, Santer Reply SpA, Athens Information TechnologyDirect inputs•Beuth Hochschule für Technik Berlin, Compass (roadmap), Department of Planning and Design/ Municipality of Neo Iraklio Attikis, ESTeam AB/ LT-Innovate, INSEE, LT-Innovate / Philippe Wacker (input and report), National Technical University of Athens, Reed Elsevier•Rethink Big, Rovertshaw Steve, Synergetics, Telefonica, TransLectures project, UC, Vienna University of Technology /Institute for Software Technology and Interactive Systems, Wolters Kluwer Deutschland GmbH, Zabala Innovation Consulting



Strength Weakness• European Aspects

o Strong medium-sized sectoro EU is a stable environment still (life standards,

currency, …)• Market and Business

o European capacity to start in Niches and then grow Business Potential

o Lots of SMEs who are dynamic and flexible, can react quickly to market changes

o Cooperation between content providers in several domains

o Existing and strong content/data market in Europe

• Technicalo Computer clusters and Clouds resources

availableo Growing interest in archiving, sensing, in situ

data, model data and citizens Big data• Data and Content

o Wealth of content datao Existing ecosystems and portals (INSPIRE,

Copernicus and GEOSS, National....)o Foundations from geospatial and environmental

data sets and supporting infrastructures• Education and Skills

o Broad & detailed domain know-how, process know-how

o Some domains with lots of good Technology and People

o Capacity and Universities to develop skillso Good education regarding engineering /domain

specific• Policy, Legal and Security

o European background is free/open process• Usage

o No items

• European Aspectso Decentralized Europeo Conservatism and Long Innovation Cycles of some domainso Lack of start-up culture, risk aversion, no failure tolerance,

move to USo Almost no European players in the data analytics

solution/suit arenao Few larger companies, many with small sizeo Focus on national market, and then small extension

international• Market and Business

o Access to affordable big data facilities and make data more easily accessible

o Visibility of ecosystem service offeringso A lack of business models for how long to preserve the data,

what data, etco EU cloud providers not as reliable and sophisticated as US-

players eg Amazon• Technical

o Processable linked data, aggregate/combine datao Interoperability, Harmonisation/standardisation of content

assets formatso Access, interconnectivity -Information in silos , data sharing,

long-term datao Technical framework for data sharing across Heterogeneous

environmentso Migration of datao Amount of data is often so huge that you cannot give the

data to the usero From 2D to the handling of 3D and 4D data

• Data and Contento Missing public data in EUo Multilingualism / Language barriero Low quality of data in open data portalso Structural Data identifies, ontologies and semanticso English bias to data and content assetso Metadata managemento Access to sub-sets of data

• Education and Skillso Integrated education of data analysts– and amount of people

• Policy, Legal and Securityo Legislative restrictions on data sharingo Restricted/Few European initiativeso Non-pragmatic regulation (like USA regulation)o Fragmented rules & regulation across Europe: e.g. Open Gov

Datao Security demands and need for more cost/benefit modelso IPR, Privacy, Legal issues

• Usageo Changing consumer behaviouro Citizen science - data from citizens, how to qualify, useo Big Data (Value) for SME use



Opportunity Threat• European Aspects

o Various cultures, various strengths can result in creative thinking if mixed

o Co-opetition enablers such as Big Data Value PPP, Industry 4.0o Strengthen the European Market, e.g. by fusing the emerging

start-up nucleio Lots of SMEs for the low hanging fruits of big data for which

agility is required o Investment into the entirety of the innovation chain – beyond

basic researcho Big Data Start-ups have a lot of creativity potentialo Useful investment support mechanisms for SMEs (like e.g.

European loans).• Market and Business

o Opening up of private content assetso Increasing use of analyticso Extending the INSPIRE and COPERNICUS at global scale (not

only Europe). o Improve creativity, cost effective solutions. An opportunity to

change. o Completely new and different business areas and services

possible o New applications for the whole Big data ecosystems from

acquisition, data extraction, visualization and usage of obtained patterns.

• Technicalo Syndication of data and contento Micropaymentso Wearables and Sensorso Device type explosiono Suitable API for access.o Interoperabilityo Visibility and greater use of Directory services for data sources

• Data and Contento Greater use of European cultural and data assetso Interconnected content, semantics and datao Navigable data and Data curationo Context and Personalisationo Increased volume of electronic data

• Education and Skillso New research areas for universities / Training with innovationo Educate the younger generation of students to use big datasets

and explore them • Policy, Legal and Security

o Storing data information on safe grounds (data security) under European legislation

o Need for uniform policies for data access • Usage

o User Generated Content and Crowd sourcing/recommendationo Data(driven) as a service - can enable entry into new markets

for Europeanso Opportunity in shift from technology push to end user workflow

• European Aspectso Europe vs the rest of the world; US / China are

running with an open mindo US Companies understand Data and

Information has a value and can be traded o Dominance of US players and their bottom-up

Ideas o No Big Data and Data-sharing culture in Europe)

• Market and Businesso Dominant large corporations that own

important datao Consolidation of different stakeholders and

marketplace in sectoro Barriers in market for SMEs – eg in owning datao Outflow from EU, in order to be nearer to the

customers or due to production costs• Technical

o No items• Data and Content

o No items• Education and Skills

o Professional Brain Draino Lack of skilled professional resources and

graduates in field• Policy, Legal and Security

o Legislation missing / falling behind; responsible too connected to ‘old data’ world

o Full analysis of ethical and privacy issues needo Over regulation, protectionism in Europe;

privacy regulations in 3rd countries are less restrictive

o Political and legal aspects: eg imagery / high definition is security risk (terrorism).

o Expectations/laws: if you provide some data, then you cannot stop providing the data or asked to provide the same data for others

o Data availability policies; eg Companies not willing to put data available eg if a branch in US because data linked to data might have to be provided

• Usageo No one wants to share data but everyone wants

to share others datao Cross border data flowso Data-driven services are not tied to a particular

location; e.g. maintenance is location-based but the service can be orchestrated from anywhere



CONSULTATION RESPONSES ON DATA



QUESTIONNAIRE RESPONSES ON DATA



OUTCOMES FROM THE PROCESS

• Formal documents of the process: BDV Frame, BDV SRIA, Template for BDV PPP

Public Consultation



RESULTS PUBLICLY AVAILABLE



CONTENTS





MUTIDISCIPLINARITY



Resources to invest, but not unlimited…Initial steps (planning): Data assets and technologies will be prioritized based on their economic potentialReporting stage: quantitative evidence on increases in performance for core technologies or reduction in costs for business processesCapitalize on existing environments and initiatives (avoid duplication of resources: incubators, PPPs, EU infrastructures..:); Welcome federation approachesPPP KPIs

APPROACH: EUROPEAN INNOVATIONS SPACES / ENVIRONMENTS (EIS/E)

• The EIS/E– Should support the legitimate ownership, privacy and security claims of corporate data

owners (and their customers) pre-condition

– Ethical considerations and legal/regulatory requirements– Motivation for data owners: get access to advanced technologies; discover

business opportunities thanks to participants interactions– Motivation for researchers, entrepreneurs, small and large companies: ease of

experimentation and business opportunities



BUILDING UPON EXISTING INFRASTRUCTURES AND TECHNOLOGIES: DATA INCUBATORS

• TeraLab (FR): digital services platform that provides both the research community and businesses, with an environment conducive to research and experimentation focused on innovative applications and industrial prototypes in the field of Big Data– Physical resources (including a substantial processing capacity with several teraoctets of

RAM), huge databases and various cutting-edge applications and tools (through SAAS/PAAS model)

– Facilitating batch or real-time processing and storage of huge amounts of data– Data assets: anonymous, publicly-available information (e.g. OpenStreetMap, Common

Crawl), and open data, but also data which has been processed to render it anonymous, provided by professional sources

– Access via secure and ultra-secure systems using technology provided by the CASD (Centre for Secure Remote Access).

• SDIL (DE): Similar approach in the domains of

• Industry 4.0• Energy• Smart Cities• Health



EX. SDIL: SOME INSIGTHS

• A Smart Data Centre and a Smart Data Community • Prompt Co-Working on valuable Problems with valuable „Big

Data“: Requires Structure (SDIL Orga) and Trust (SDIL Contract) • Knowledge Transfer from Research to Industry and vice versa • Exploration of

– Smart Data Potential / Innovation Potential – Best Practice and Standards

• Research in – Analytics, Systems, Application Domain specific – Smart Data Tools, Support, Management, Privacy, Legal and Curation – HCI and Usability

• First focus: short term high impact projects



R&I CYCLE & GOVERNANCE



BUILDING UPON EXISTING INFRASTRUCTURES AND TECHNOLOGIESFI PPP: FI-ARE/FI-LAB

• The Big Data Analysis Support GE offers a solution for both Big Data Batch processing (BD Crunching) and Big Data Streaming; unified set of tools and APIs allowing developers to program the analysis on large amount of data and extract relevant insights in both scenarios using Map&Reduce (ex. Social Networks analysis, real-time recommendations, etc).

TechnologyA true open innovation ecosystem



MAIN COMPONENTS & RESEARCH PRIORITIES

Privacy and anonymisation mechanisms

Privacy and anonymisation mechanisms

Improving understanding of data by deep analytics (e.g. predictive modelling, graph mining, ...)

Improving understanding of data by deep analytics (e.g. predictive modelling, graph mining, ...)

Optimized Architectures for analysing data including real-time data(e.g.recommendation engines, ...)

Optimized Architectures for analysing data including real-time data(e.g.recommendation engines, ...)

Visualization and user experience (e.g. User adaptive systems, search capabilities, ...)

Visualization and user experience (e.g. User adaptive systems, search capabilities, ...)

Data management engineering (e.g. Data integration, data integrity, ...)

Data management engineering (e.g. Data integration, data integrity, ...)

Innovation Spacesserve as hubs for bringing the technology and application developments together and cater for the development of skills, competence, and best practices.

Innovation Spacesserve as hubs for bringing the technology and application developments together and cater for the development of skills, competence, and best practices.

Lighthouse Projects

Large scale demonstrations focusing on certain sectors and domains

Lighthouse Projects

Large scale demonstrations focusing on certain sectors and domains



TIMEFRAME OF MAJOR EXPECTED OUTCOMES

Year 1 Year 2 Year 3 Year 4 Year 5

Priority : privacy and anonymisation

Generalisation of Secure Remote Data Access Centre techniques.

Method for deletion of data and data minimization. Robust anonymisation algorithms

Priority : deep analysis

Improved statistical models by enabling fast non-linear approximations in very large datasets

Predictive modeling Graph mining techniques applied on extremely large graphs

Real-time analyticsSemantic analysis in near-real-timeAlgorithms for multimedia data mining

Deep learning techniquesDescriptive language for deep analytics.

Contextualisation.

Priority : architectures for analytics of data at rest and in motion

Optimized tools for the integration of existing components to new types of platforms with both data at rest and in motion.

Synergies between massively parallel architectures (MPP) and batch processing/stream processing architectures

Priority : advance visualisation

New data search solutions / paradigms

Semantic driven data visualisation – stronger links between visualization and analytics

User adaptation

Collaborative real-time, dynamic 3D solutions

Priority : engineering data management

APIs for improving the process of data transformation

Collaborative Tools and techniques for Data Quality (including integrity and veracity check)Harmonized description format for meta-data

Methodology, models and tools for data lifecycle management

Data management as a service



SKILLS (DATA SCIENTISTS VS DATA ENGINEERS)

27

• Actions for skills– Work with higher education institutes and education providers to support the establishment of:

New educational programmes with a core focus on data science including interdisciplinary curricula with a focus on specific application domains

Novel courses to educate and train the current workforce with the specialized skillsets needed to be Data Engineers.

Foundational modules in data science, statistical techniques and data management within related disciplines such as law and humanities.

– Leverage Innovation Spaces to develop a network between scientists (academia) and industry that foster the exchange of ideas and challenges

– Encourage industry to enhance the industrial relevance of courses by making datasets and infrastructure resources available

• Actions for Best Practices– Establish Innovation spaces to foster learning, develop competence, and identify best practices

for Data Engineers. Leverage the innovation space to: Package best practices in the form of frameworks, toolboxes and integrated models to bring

individual solution “components” into a system perspective Formally define a Big Data Value Body of Knowledge (BDVBoK) Deliver online seminars to highlight the latest in best practice developed in the innovation

space and encourage and promote the adoption of best practices across sectors, especially targeting SMEs.



BUSINESS MODELS

• Business model methodology: cross-enterprise, cross-sector level approach comprising all stakeholders of the Big Data Ecosystem (within Innovation Spaces)– Traditional consulting, consultation workshops, employee idea

competitions, utilization of social networks…to foster information exchange and best practice sharing

• Secure environments for testing Business models• Data service clearing house

– European and national level support to companies, including SMEs, in terms of legal practice and advice



POLICY & REGULATION

• Contribution by the PPP: Bring together participants in an integrated and collaborative way, with the aim of creating a forum for a structured dialogue around data-driven innovation issues– Involvement of the citizen as active stakeholder, to address concerns and raise awareness– Input to legislative discussions to embrace data-driven innovation. – Information to data owners on the benefits of data sharing– Develop a cross domain understanding of key issues of the big data value economy– Identify crucial elements of future eco-systems that represent a challenge or roadblock to faster

take up

• Topics– Identification of issues from data acquisition, ownership of raw data, processed data, augmented

data, aggregated data; – Liability of data analysis, to deal with potential damage caused by incorrect analysis – Measurement the value of data, the succession of data rights and the legacy of stored data for

new, merged or bankrupt companies; – Delete mechanisms, policies for data processing and data usage. – Identifying the barriers with regard to ownership of data and intellectual property rights incl.

copyright and access rights



CONTENTS





STAKEHOLDERS PARTICIPATION



RELEVANCE OF USER INDUSTRIES

Big Data Strategies of User Industry, Source: Morgan Stanley



IMPLEMENTATION TIMELINE



ESTIMATED BUDGET



BDV PPP STRUCTURE

BIG

Big Data Public Private Forum

Q&A

THANK YOU

Nuria de LamaRepresentative to the European CommissionResearch & Innovation [email protected]

mailto:[email protected]

big data public private forumbig-project.eu/sites/default/files/big_big data ppp_nuria...big big...

Documents