nosql & json
DESCRIPTION
NOSQL and JSON introductionTRANSCRIPT
NOSQL & JSON
2014/10/12 WU TIEN YANG
JSON
JSON JAVASCRIPT OBJECT NOTATION
• serializing data objects
• Human-readable,useful for data interchange
• useful for storing semistructured data
JSON
• Basic value
• number,string,boolean
• Objects{}
• label-value pairs
• Arrays[]
• list of value
LIVE DEMO JSON EDITOR ONLINE
JSON
Relational JSON
Structure TablesNest,label-value,array
Schema Fix Flexible
Queries simple expressive language XD,widely use
Ordering None Arrays
Implementation Native system NOSQL system
JSON JAVASCRIPT OBJECT NOTATION
• Sets of label-value pairs
• Arrays of values
• Base values from predefined types
JSON fileJSON parse program objects
syntactic error
WHEN JSON MEET PYTHON!
JSON
JSON PYTHON
Object dict
array list,tuple
str unicode
null none
number int,long,float
encoding: python -> json json.dumps()
decoding: json -> python json.loads()
LIVE DEMO
NOSQL
WHY WE CHOOSE NOSQL?
Relational DBMS face the problem!
problem: semi-structure data poor performance
problem: scaling difficult / expensive / impossible ->BIG DATA
NOSQL
• SQL : Traditional relational DBMS
• NOSQL = not only SQL
• MASSIVE!
NOSQL
• Flexibles schema
• Quicker / Cheaper to set up
• Massive scalability
• Relaxed consistency -high performance,availability
NOSQL system V.S Traditional relational DBMS
NOSQL
• Flexibles schema
• Quicker / Cheaper to set up
• Massive scalability
• Relaxed consistency -high performance,availability
NOSQL system V.S Traditional relational DBMS
no declarative query language -> more programming
Relaxed consistency ->fewer guarantees
NOSQL MAPREDUCE FRAMEWORK
• no data model,data stored in files
• specific function
• map(),reduce(),read(),write()
• system provides fault-tolerance,scability
NOSQL MAPREDUCE FRAMEWORK
• Map: divide problem into subproblems
• map(item) -> <key,value>pairs
• Reduce: do work on subproblems,combine result
• reduce(key,lists of value) -> result
NOSQL MAPREDUCE EXAMPLE
• record: User ID,URL,timestamp,additional-info
• task: Count number of accesses for each domain
• map(record) -> <domain,value>
• reduce(domain,lists of value) -> <domain,sum>
Web log analysis
NOSQL KEY-VALUE STORES
• extremely simple interface
• Data model: {key,value} pairs
• Operation : insert , update , delete
• implement : efficiency,scalability,fault-tolerance
• record distributed to nodes base on key
• eventual consistency
MONGODB
MONGODB NOSQL DATABASE
RDBMS MongoDB
Database Database
Table collection
Record/Row Document
Column Field
Primary Key _id
a record = a document
many documents = a collection
LIVE DOME TRY IT OUT !
REFERENCES
• Stanford University -DB introduction to Databases
• python library - storing json
• mongoDB