nosql on the move
TRANSCRIPT
![Page 1: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/1.jpg)
NoSQL on the move Glynn Bird
ROME 18-19 MARCH 2016
![Page 2: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/2.jpg)
NoSQL…
![Page 3: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/3.jpg)
Relational Databases • Defined Schema • Tables/Columns/Rows
• Constraints/Stored Procedures/Triggers
• Indexes/Views
• Structured Query Language (SQL)
![Page 4: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/4.jpg)
NoSQL Databases • Any database that isn't a RDBMS! • Types include: • Document stores • Key/Value stores • Graph databases • Often schemaless • Often without SQL
![Page 5: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/5.jpg)
…On the Move
![Page 6: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/6.jpg)
6
![Page 7: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/7.jpg)
7
![Page 8: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/8.jpg)
8
![Page 9: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/9.jpg)
Image Credit: Joan Touzet (@wohali), ASF Member, CouchDB PMC Member
9
![Page 10: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/10.jpg)
Frameworks
10
![Page 11: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/11.jpg)
Image Credit: Device landscape by Jeremy Keith, on Flickr
11
Not just mobile first…
![Page 12: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/12.jpg)
Image Credit: NASA New Horizons
12
Offline First
Offline-First
![Page 13: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/13.jpg)
Offline, online and somewhere in-between
13
![Page 14: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/14.jpg)
The Eight Fallacies of Distributed Computing 1. The network is reliable 2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous
14
Text Credit: The Eight Fallacies of Distributed Computing by Peter Deutsch | Image Credit: Pneumatic Central by Sleestak, on Flickr
![Page 15: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/15.jpg)
15
Offline-first is the only way to achieve a true, 100%
always-on user experience.*
*assuming the device is reliable
![Page 16: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/16.jpg)
Benefits of Offline First
16
• Better, faster user experience, both offline and online • Allow your users to work offline or with limited connectivity • Potentially saves battery life and bandwidth usage
![Page 17: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/17.jpg)
Offline Patterns & Anti-Patterns • Don't return an error for no reason • Do let users view cached/saved data
• Do synchronize data when connected
• Consider letting your users decide when to sync
• Think about the UX of users seeing stale data
17
![Page 18: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/18.jpg)
Difficulties of Offline First
18
Image credit http://www.sneakerheadvc.com/wp-content/uploads/2012/02/Apple_iSync1.png
![Page 19: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/19.jpg)
Introducing CouchDB & IBM Cloudant
![Page 20: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/20.jpg)
Apache CouchDB • JSON document store • HTTP API
• Replication
• Free, open-source
20
![Page 21: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/21.jpg)
Apache CouchDB – built to replicate • MVCC for document versioning • Replication
• One-way or Two-way
• One-shot or continuous
21
![Page 22: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/22.jpg)
Apache CouchDB – built to replicate
22
2.0
multi-node clustering
Cloudant Geo
Cloudant Query (Mango)
Cloudant Search (Lucene)
Dashboard
![Page 23: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/23.jpg)
IBM Cloudant – built for scale
23
![Page 24: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/24.jpg)
IBM Cloudant • Globally distributed data layer for
web and mobile applications • Run as-a-service • MongoDB-style queries
• Advanced geospatial capabilities
• Full text search indexing
24
![Page 25: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/25.jpg)
Mobile Apps - PouchDB
![Page 26: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/26.jpg)
PouchDB • A database in your web browser • Can synchronize with any
database that implements the CouchDB Replication Protocol
• Makes create, read, update and delete operations extremely quickly
• Free, open-source
26
![Page 27: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/27.jpg)
Getting started with PouchDB
27
<script src="https://cdn.jsdelivr.net/pouchdb/5.1.0/pouchdb.min.js"></script> <script type="javascript">
var db = new PouchDB("smart-meter"); </script>
![Page 28: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/28.jpg)
Adding documents - callbacks
28
db.post({ date: "2014-11-12T23:27:03.794Z",
kilowatt_hours: 14
}, function(err, data) {
console.log(err,data); });
![Page 29: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/29.jpg)
Adding documents - bring your own id
29
var db = new PouchDB("smart-meter"); var obj = {
_id: "abc123",
timestamp: "2014-11-12T23:27:03.794Z",
kilowatt_hours: 14 };
db.put(obj, callback);
https://github.com/bradley-holt/offline-first/blob/master/pouchdb/04-create-document-put.js
![Page 30: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/30.jpg)
Getting a document
30
db.get("abc123", callback);
// calls back with // {
// _id: "abc123", // _rev: "1-27695d5f483ac267d03ad0e3cb54bd2c",
// timestamp: "2014-11-12T23:27:03.794Z",
// kilowatt_hours: 14
// }
![Page 31: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/31.jpg)
Getting multiple documents
31
db.allDocs({include_docs:true}, callback);
// calls back with // {
// "offset": 0, // "total_rows": 24,
// "rows": [{...},{...}]
// }
![Page 32: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/32.jpg)
Querying
32
• Primary Index • MapReduce
• PouchDB-find plugin
• PouchDB-quick search plugin
![Page 33: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/33.jpg)
Querying a Database with PouchDB Find • Based on Cloudant Query (Mango) • MongoDB-style query language
33
Image Credit: Mango with section on a white background by bangdoll, on Flickr
db.find({
selector:{
name:'Mario'
debut:{'$gt':1990}
},
fields:['_id','lastname'],
sort:['lastname']
})...
![Page 34: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/34.jpg)
Replicating a PouchDB Database
34
var db = new PouchDB("smart-meter");
var remoteDb = new PouchDB("https://bradley-holt.cloudant.com/smart-meter");
db.replicateTo(remoteDb);
https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
![Page 35: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/35.jpg)
Bidirectionally Replicating a PouchDB Database
35
db.sync(remoteDb).on("change", function(info) {
// Replication has written a new document
console.log(info);
}).on("complete", function(info) {
// Replication has completed or been cancelled
console.log(info);
});
https://github.com/bradley-holt/offline-first/blob/master/pouchdb/09-replicate-database-bidirectional.js
![Page 36: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/36.jpg)
Listening for Database Changes
36
var changes = remoteDb.changes({
since: "now"
}).on("change", function(change) {
// A document has changed
console.log(change);
}).on("complete", function(info) {
// changes() was canceled
console.log(info);
});
https://github.com/bradley-holt/offline-first/blob/master/pouchdb/11-database-changes.js
![Page 37: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/37.jpg)
PouchDB Framework Adapters
37
![Page 38: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/38.jpg)
Web Apps Going Offline
![Page 39: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/39.jpg)
HTML5 Offline Application Cache • Enables fully-functional offline
web apps
• Stores files and assets for offline browsing
• Makes page loads very fast, even when online
39
![Page 40: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/40.jpg)
Cache Manifest File
40
<html manifest="example.appcache"> … </html>
CACHE MANIFEST # v1 - 2015-01-08 index.html logo.png app.css app.js
![Page 41: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/41.jpg)
Service Workers (Beta)
41
Client-side scripting framework for • programmable cache
• sync
• push messaging
• geo-fencing
• background tasks
https://jakearchibald.github.io/isserviceworkerready/
![Page 42: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/42.jpg)
Hybrid or Responsive Mobile Web Apps
![Page 43: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/43.jpg)
Hybrid Mobile Web Apps • Native mobile web apps built with
HTML5, CSS and JavaScript
• Good for: • Fully-featured, cross-platform native apps
• High-fidelity prototypes
43
![Page 44: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/44.jpg)
Responsive Mobile Web Apps • HTML5, CSS and JavaScript mobile
web apps
• Responsive design
• Enhanced to enable offline usage
44
![Page 45: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/45.jpg)
Native iOS & Android Apps
![Page 46: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/46.jpg)
Cloudant Sync • Library for iOS and Android • Provides local storage and query API
• Optional sync to Cloudant
• Open-source and free to use
![Page 47: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/47.jpg)
Cloudant Sync • Stores data using SQLite • TouchDB provides MVCC
• Replication to Cloudant over HTTPS
• Cloudant Query API
![Page 48: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/48.jpg)
Pro Forma • Define fields you want to collect • Renders form saving data to
PouchDB
• Replicates data to Cloudant
• Demo https://glynnbird.github.io/proforma/
48
![Page 49: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/49.jpg)
MD • Offline word processor • Saves Markdown documents to
PouchDB
• Replicates data to Cloudant
• Demo http://mddoc.mybluemix.net/
49
![Page 50: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/50.jpg)
Gutenberg • Offline e-book reader • Replicates book list from server
• Each book is a Cloudant database
• Demo http://glynnbird.github.io/gutenberg/
50
![Page 51: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/51.jpg)
www.glynnbird.com • My home page • Cloudant database of articles
• Replicated to PouchDB
• Appcache for offline first
• http://www.glynnbird.com/
51
![Page 52: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/52.jpg)
Volt • Password vault in a Chrome
extension
• Data stored in encrypted in PouchDB
• Optional back to CouchDB/Cloudant
• https://github.com/glynnbird/volt
52
![Page 53: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/53.jpg)
Location Tracker • Stores data locally in PouchDB • Front end built with AngularJS
• Authentication logic built with Node.js
• User interface built with Leaflet
• Replicates location data to Cloudant
• More info: https://cloudant.com/location-tracker/
53
![Page 54: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/54.jpg)
Glynn Bird Developer Advocate, Cloud Data Services [email protected] @glynn_bird github.com/glynnbird
![Page 55: NoSQL on the move](https://reader031.vdocument.in/reader031/viewer/2022022203/58739c401a28ab85438b6afb/html5/thumbnails/55.jpg)
Image Credits
55
• Joan Touzet (@wohali), ASF Member, CouchDB PMC Member <https://twitter.com/wohali/status/595689720933445632>
• Device landscape by Jeremy Keith, on Flickr <https://www.flickr.com/photos/adactio/6153481666>
• NASA New Horizons <https://www.nasa.gov/sites/default/files/thumbnails/image/nh-surface.jpg>
• Pneumatic Central by Sleestak, on Flickr <https://www.flickr.com/photos/dlanod/235990854>
• Mango with section on a white background by bangdoll, on Flickr <https://www.flickr.com/photos/bangdoll/5665235102>
• Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr <https://www.flickr.com/photos/80497449@N04/7417352980>