open source and open data in the age of the cloud

Open Source and Open Datain the Age of the Cloud

Tim O’Reilly

April 13, 2010

Monday, April 19, 2010

“Major Strasser has been shot...Round up the usual suspects”


Database

“You keep using that word. I do not think it means what you think it means.”

Web 2.0 Cloud Computing


“I’m an inventor. I became interested in long term trends because an invention has to make sense in the world in which it is finished, not the world in which it is started.”

-Ray Kurzweil


You may think of me as a book publisher


What We Really Do At O'Reilly

Change the world by spreading the knowledge of innovators


O’Reilly Radar Methodology

“The future is here. It’s just not evenly distributed yet.” - William Gibson

We “watch the alpha geeks” and think about the futures they are living in

We then look for trend data that tells us that a particular future is becoming mainstream

I’m going to tell you some seemingly unconnected technology stories from the front lines of innovation. Then we’re going to connect the dots.


<1>


The cloud future includes...

Devices acting as sensors for intelligent data collection

Devices whose UI is on the web rather than the device

Feeding data into multiple online services that will turn into a full-on sensor web

Setting the stage for robotics, augmented reality, and the next generation of personal electronics


<2>


What we see here

Peer-to-peer credit card payments Social networks used for risk evaluation

The PC is out of the loop The phone is a sensor platform

– Hardware add-on innovation– Location based sensing– Touch screen UI

Processing is done in real time in the cloud– Allowing processing that can’t be done on the device– Big data analysis– Building new networks on the back of existing ones

Reinventing a major industry


<3>


The Yelp Monocle

Find cafes nearby.


<4>


Crowdsourcing includes the use of humans as sensors


AMEE - the world’s energy meter


We’re moving to a world in which every device generates useful data, in which every action creates “information shadows” on the net.


<5>


The smart phone plus local search. Today pizza,

Round Table Pizza, 4300 Great America ParkwayGiovanni’s NY Pizzeria 1127 Lawrence ExpwyLittle Caesar’s Pizza, 4767 Lafayette Street


An application running on a mobile device whose user interface is driven by sensors:

- Touch screen- Motion and proximity sensors- Microphone- GPS or cell tower triangulation


An application that depends on cooperating cloud data services: - Speech recognition- Search- Location data


An application that applies context-sensitive filters to give users just the information they need.


In real time


<6>


•Search in plain English•Search by voice•Traffic view•Search along route•Satellite view•Street view


An application that depends on cooperating cloud data services: - Location- Search- Speech recognition- Live Traffic- Imagery


Database

“You keep using that word. I do not think it means what you think it means.”


The Internet Operating System is A Data Operating System


The Internet Operating System is a Data Operating System

It helps applications find out about– People– Places– Things– Prices– Documents– Images– Sounds– Relationships– ...

and helps people interact with them through services– Search– Payment– Matching and Recognition– ...


In Real Time


This is the 21st century data challenge

Not transactions Not data warehouses and business intelligence Not database-backed web sites Not even NoSQL-backed web services

Real time cloud-based intelligence delivered to mobile applications

Algorithmic intelligence


This is the world your database engines must be designed to support


Who will own the Internet Operating System?


Do we want anyone to own it?


If not, we’d better get busy!


Other

Infrastructure As a Service

Storage — —

Computation — —

Hosted SaaS Apps —

Media access

Music —

Video —

Books — — — — —

Web Content —

Photos — — —

Monetization

Advertising — —

Payment

Location

Maps & Directions — — —

Real Time Location Detection — —

Calendaring/Scheduling —

Social Graph — —

Communications

Email — —

Voice — —

Chat — —

Video — —

Sensor Management

Speech Recognition — —

Image Recognition — — —

Mobile Device OS —

Mobile Device Hardware —

Web Browser — —

Chart Title?

Strong o!eringMedium o!eringGetting started

— Not on the board yet

Who Owns What


Interoperable web services, open data, and standard protocols are at least as important as open source


The underdog is the ally of open source


Potential open source supporters

Search: Microsoft Maps: Microsoft, Nokia,Yelp, Foursquare Speech: Nuance, Microsoft Social Graph: Google Payment: Paypal Cloud infrastructure: VMware Smartphones: Google Device Operating Systems: Google


Corporate data sharing

There’s a lot of data in the world


Switching tracks (a bit)...


For more information

The Open Source Paradigm Shift (2003) http://bit.ly/cKLSUP

What is Web 2.0? (2005) http://oreil.ly/a0zT65

Web Squared: Web 2.0 Five Years On (2009) http://bit.ly/kEKgs

Government as a Platform (2010) http://opengovernment.labs.oreilly.com/

Ongoing commentaryhttp://radar.oreilly.comhttp://twitter.com/timoreillyhttp://buzz.google.com/timoreilly


http://bit.ly/cKLSUP

http://bit.ly/cKLSUP

http://oreil.ly/a0zT65

http://oreil.ly/a0zT65

http://bit.ly/kEKgs

http://bit.ly/kEKgs

http://opengovernment.labs.oreilly.com

http://opengovernment.labs.oreilly.com

http://radar.oreilly.com

http://radar.oreilly.com

http://twitter.com/timoreilly

http://twitter.com/timoreilly

open source and open data in the age of the cloud

Technology