big data and html5 (devcon tlv 2012)
DESCRIPTION
In this talk from DevCon TLV we covered: ● The power of HTML5 APIs and how you can use them in your next modern Web Apps. ● On the server side how you can use: Google Cloud Endpoints to scale your API and gain more productivity. ● We did some live Demos and talked about Big Query interfaces.TRANSCRIPT
Big Data And HTML5@DevCon TLV
Ido GreenDeveloper Advocatehttp://plus.ly/greenido@greenido DevCon TLV
September 5th 2012#devcon
●The power of HTML5 APIs●Modern Web Apps
●Google Cloud Endpoints●Live Code & Demo
●Big Query●Q&A
Agenda
#devcon
Modern Web Applications
● Self Contained & Functional
● "Offline First"
● Client Side Architecture & MVC Frameworks
● Device Aware / 60fps
#devcon
Client Side Frameworks
● Ember.js - Don't waste time making trivial choices
● Backbone.js - Gives structure to web applications by providing models with binding, collections and views
● Angular.js - AngularJS lets you extend HTML vocabulary for your application
More: addyosmani.github.com/todomvc/
#devcon
● Airplane, road trip, deserted island
● Flaky connections (e.g. cafes, car)
● Better performance
● Consolidates the concept of permanent app you will have always available
* We will use: Lawnchair for our demo.
Offline - Why?
#devcon
● Storing assets: AppCache
● Storing data: localStorage, IndexedDB, File API.
● Offline first:○ Pretend that there's no internet connection○ Implement a sync layer that works only when
online.
Offline - How?
navigator.onLine & window.(ononline|onoffline)
#devcon
Do More For Your Users
Web Intents is a framework for client-side service discovery and inter-application communication
#devcon
Google Chrome Frame
Chrome Frame is an open source plug-in that seamlessly brings Google Chrome's open web technologies and speedy JavaScript engine to IE
<meta http-equiv="X-UA-Compatible" content="chrome=1">
X-UA-Compatible: chrome=1
#devcon
Modern Web Apps and The Server Conundrum
All modern web apps have to deal with a "home" server○ Offload Computation○ Sharing and Collaboration
But who wants to run a server○ Spikey traffic ○ Client Server communication○ Serialization○ OAuth Dance
#devcon
● Make it easy to build
● Easy to run and manage
● Easy to scale
● Free to get started, pay for what you use
● Not just for web serving, but also data persistence
Google App Engine To The Rescue
#devcon
Google Cloud Endpoints:
Business Logic
APIs for Mobile and Web Backends Made Easy!
Storage(DataStore, SQL, Drive, etc)
Web APIs
End
poin
ts
Trusted Tester
#devcon
Demo
https://github.com/greenido/backbone-bira
#devcon
Google Cloud Endpoints for Beer!
Business Logic
Beer Rating and Review Application
StorageNoSql Datastore
End
poin
ts
Access Control, Sort, Filter
Client UI for managing Beers
#devcon
Demo!Beers On!https://birra-io2012.appspot.com/
Load Test Results - From laptop wifi
% ab -n9000 -c100 http://birra-io2012.appspot.com/
Percentage of the requests served within a certain time (ms) 50% 263 66% 321 75% 370 80% 390 90% 449 95% 536 98% 649 99% 715 100% 3858 (longest request)
> 90% of r
equests b
elow
1/2 a se
c
#devcon
Load Test Results - From Compute Engine
% ab -n1000000 -c10 http://birra-io2012.appspot.com/
Percentage of the requests served within a certain time (ms) 50% 6 66% 6 75% 6 80% 6 90% 7 95% 7 98% 11 99% 17 100% 3019 (longest request)
> 95% less
than 7m
s
#devcon
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
10 million rows?
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
100 million rows?
1 million
1 million1 million 1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million
1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million
1 million
1 million1 million1 million
1 million1 million1 million1 million
1 million
1 million
1 million1 million1 million
1 million1 million1 million1 million
1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
1 million1 million1 million1 million1 million1 million1 million1 million1 million1 million
720 million rows!
1 million
BigQuery gives you this power
Store data with reliability, redundancy and consistency
Go from data to meaning
Quickly!
At scale & secure way...
#devcon
How are developers using it?
Game and social media analytics
Advertising campaign optimization
Sensor data analysis
Infrastructure monitoring
#devcon
Big Data at Google - Finding slow servers
SELECT count(*) AS count, source_machine AS machineFROM product.product_log.liveWHERE elapsed_time > 4000GROUP BY source_machineORDER BY count DESC
Result in ~20 seconds!#devcon
Load your data into BigQuery"jobReference":{ "projectId":"605902584318"}, "configuration":{ "load":{ "destinationTable":{ "projectId":"605902584318", "datasetId":"my_dataset", "tableId":"widget_sales"}, "sourceUris":[ "gs://widget-sales-data/2012080100.csv"], "schema":{ "fields":[{ "name":"widget", "type":"string"}, ...
POST https://www.googleapis.com/bigquery/v2/projects/605902584318/jobs
#devcon
Query Away!
"jobReference":{ "projectId":"605902584318"}, "query":"SELECT TOP(widget, 50), COUNT(*) AS sale_count FROM widget_sales", "maxResults":100, "apiVersion":"v2"}
POST https://www.googleapis.com/bigquery/v2/projects/605902584318/jobs
#devcon
Libraries - Example JavaScript query
var request = gapi.client.bigquery.jobs.query({ 'projectId': project_id, 'timeoutMs': '30000', 'query': 'SELECT state, AVG(mother_age) AS theav FROM [publicdata:samples.natality] WHERE year=2000 AND ever_born=1 GROUP BY state ORDER BY theav DESC;'});
request.execute(function(response) { console.log(response); $.each(response.result.rows, function(i, item) { ...
#devcon
Prepare your data / Batch Jobs
●App Engine MapReduce●Commercial ETL tools
● Pervasive● Informatica● Talend
●UNIX command-line
Don't need interactive queries for some jobs?● priority: "BATCH"
#devcon
Key Take Aways
Building modern applications with HTML5 and App Engine
AppEngine makes for easy deployment at scale
Use Big Query to 'feel' your data
Build interactivity by leveraging Cloud Endpoints
Exploit Modern Browser Features:○ Offline○ Geo○ Web Intents
#devcon
Toda & Questions?
Ido GreenDeveloper RelationsGoogle Chrome Platformplus.ly/greenido
App:: http://birra-io2012.appspot.com/
Code: https://github.com/greenido/Slides: ido-green.appspot.com
Access: http://endpoints-trusted-tester.appspot.com
#devcon