siriuscon2016 - visualization of inferred versioned schemas from nosql databases
TRANSCRIPT
Visualization of Inferred Versioned Schemasfrom NoSQL Databases
Alberto Hernández Chillón Diego Sevilla Ruiz Jesús García [email protected] [email protected] [email protected]
Cátedra SAES-UMUUniversity of Murcia
Faculty of Computer ScienceUniversity of Murcia
Faculty of Computer ScienceUniversity of Murcia
November 15, 2016
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 1 / 41
About the authors
Member of the Cátedra SAES team since 2014
M.Sc. in Computer Science from the UM
MDE, automatic code generation, NoSQL databases
Associate professor at the Faculty of Computer Science of the UM
M.Sc and Ph.D. in Computer Science from the UM
NoSQL databases, distributed systems, testing
Professor at the Faculty of Computer Science of the UM since 1984
Head of the Modelum Group
MDE, DSL, Software modernization, reverse engineering
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 2 / 41
Index
1 Introduction and context
2 Inference process
3 Schema visualization
4 Conclusions and future work
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 3 / 41
Index
1 Introduction and context
2 Inference process
3 Schema visualization
4 Conclusions and future work
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 4 / 41
Introduction and context
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 5 / 41
NoSQL systems
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 6 / 41
Usefulness of an explicit schema
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 7 / 41
What kind of NoSQL systems? - Schemaless
{" person_id ": "123" ,"type": " Person "," lastName ": "Rush"," firstName ": " Christopher "," address ": "C/Gran Via, 13, Madrid "
},{
" person_id ": "456" ,"type": " Person "," lastName ": " England "," firstName ": " Wayne "," address ": {
" street ": "Av. Pinos , 24","city": " Murcia " }
},{
" person_id ": "789" ,"type": " Person "," lastName ": " Hoover "," firstName ": " Quinton "," address ": " Ronda Norte , 15, Murcia ","age": 35
}
NoSQLDatabase
Data non-uniformityDifferent versions for thesame data
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 8 / 41
What kind of NoSQL systems? - Aggregations
{"_id": "2","type": "book"," title ": " Writing and Querying MapReduce
Views in CouchDB "," publisher_id ": "928672" ," author ":{
"_id": "101" ,"type": " author ","name": " Bradley Holt"," company ": {
"_id": "324" ,"type": " company ","name": "IBM Cloudant "," country ": "USA"}
},},{
"_id": "928672" ,"type": " publisher ","name": "O’ Reilly Media ","city": " Newton "
}
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 9 / 41
Proposed objective
A DataVersity report (2015) indicated essential functionalityrequired in the near future:
Model visualizationCode generation from schemasMetadata management
Our goal is to design and implement a tool which will allow usto visualize NoSQL schemas:
Taking into account concepts such as versions......having in mind the underlying inference process......among other things
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 10 / 41
Index
1 Introduction and context
2 Inference process
3 Schema visualization
4 Conclusions and future work
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 11 / 41
Process overview
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 12 / 41
Inference process (I)
NoSQLDatabase MapReduce
ObjectVersions(JSON)
JSONInjection
JSONModel
JSONMetamodel
SchemaReverse Eng
SchemaModel
ApplicationGeneration
SchemaViewer/
DataValidator/MigrationAssistant
Applications SchemaMetamodel
inst
ance
instance
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 13 / 41
Initial metamodel
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 14 / 41
Polished metamodel (I)
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 15 / 41
Polished metamodel (II)
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 16 / 41
Metamodel elements
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 17 / 41
Schema versions (I)
Schema version for Book_1
...{
"_id": "53","type": "Book"," title ": " Book_1 "," hasContent ":[{
"_id": "64","type": " Content "," chapters ": 3," pages ": 17
}]," hasAuthors ":[{
"_id": "155" ,"type": " Author ","name": " Author_155 "," hasCompany ":[{
"_id": "123" ,"type": " Company ","name": " Company_123 "," country ": " Country_1 "
}]}]
}...
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 18 / 41
Schema versions (II)
Schema version for Book_2...{
"_id": "89","type": "Book"," title ": " Book_2 "," hasAuthors ":[{
"_id": "77","type": " Author ","name": " Author_77 "," hasCompany ":[{
"_id": "61","type": " Company ","name": " Company_61 "," country ": " Country_2 "
}]}]
}...
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 19 / 41
Schema versions (III)
Schema version for Author_2...{
"_id": "99","type": " Author ","name": " Author_2 "," company ": " Company_3 "," country ": " Country_3 "
}...
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 20 / 41
Index
1 Introduction and context
2 Inference process
3 Schema visualization
4 Conclusions and future work
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 21 / 41
Schema visualization
Why using Sirius?Views we need:
General tree viewGlobal schema viewSchema version viewsEntity detail view
Ability to create different viewpoints and navigabilitymechanisms between themAutomatic generation of an embedded editor in the EclipseenvironmentExtensibility
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 22 / 41
Design process
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 23 / 41
Schema visualization - Input model
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 24 / 41
Schema visualization - Tree viewpoint (I)Schema versions grouped by Entity
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 25 / 41
Schema visualization - Tree viewpoint (II)Schema versions grouped by Version
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 26 / 41
Schema visualization - Tree viewpoint (III)List of Entities, Versions and Properties
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 27 / 41
Schema visualization - Global schema (I)Diagram of Entities, Versions and Properties
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 28 / 41
Schema visualization - Global schema (II)Implementation of the Global schema
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 29 / 41
Schema visualization - Schema version (I)Schema version root and its associations
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 30 / 41
Schema visualization - Schema version (II)Implementation of the Schema view
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 31 / 41
Schema visualization - Schema version (III)Variation with embedded direct associations
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 32 / 41
Schema visualization - Schema version (IV)Implementation of the Schema view
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 33 / 41
Schema visualization - Entity detailEntity detail with its Versions associations
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 34 / 41
Navigation between views
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 35 / 41
Index
1 Introduction and context
2 Inference process
3 Schema visualization
4 Conclusions and future work
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 36 / 41
To sum up (I)
About Sirius...4 Built in the Eclipse Modeling Framework4 Easily deployable along with the metamodel into plugins4 Different viewpoints, layers and customization options4 Less development time and easier to extend8 AQL might be quite tricky sometimes when defining actions8 Learning curve for creating complex examples may be too harsh
? Language/model maintenance and evolution? Managing large input models
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 37 / 41
To sum up (II)
About the tool...We developed one of the first approaches for NoSQLvisualization considering versionsTool to visualize NoSQL database schemas and schema versionswith Sirius
Tree viewpoint and global schema viewpointSchema version and entity detail viewpoint
...but there is still a lot of work to do:Polish the NoSQL_Schema metamodelImprove the Sirius viewpoint definitionsImplement the editor functionality
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 38 / 41
Future work (I) - DB alterations
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 39 / 41
Future work (II) - DB migration
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 40 / 41
[email protected] https://github.com/Soltari https://linkedin.com/in/albertohc
Alberto H.C. & Diego S.R. & Jesús G.M. NoSQL Visualization November 15, 2016 41 / 41