enterprise on blockchain let you big data sing€¦ · backend app smart contract artificial...

Enterprise on Blockchain – Let you big data sing

Keynote presentation

by

Professor Dr. Mukesh Mohania

Prepared byE. Chang, S. Green, A. Ghildyal

UNSW@ADFA and Department of Defence7th International Conference on Big Data Analytics (BDA 2019)

Ahmedabad University Gujarat, India18-20 Dec 2019

Content

1. Enterprise Big Data Analytics

2. Why Enterprise on Blockchain

3. Shut your data up or let your data sing

4. Enterprise Blockchain and data migration

5. Examples – Blockchain and AI, let your data sing

6. Enterprise Blockchain – Implementation and future challenges

7. Conclusion - 50 years enterprise data management paradigm

1.1 Enterprise Big Data Analytics

setsBags

Text = Strings

Trees

DocumentsProfiles Folders

Containers

1.1 Enterprise Big Data Analytics

Artificial Intelligence

Big data repositoryBackend

Big Data AnalyticsApplication

Big Data VisualizationFrontend

1.2 Enterprise Data Management


Big data repositoryBackend

Big Data AnalyticsApplication

Big Data VisualizationFrontend

Ledger

1.3 Future Enterprise on Blockchain

8

Memory

Simply

Environment Growing

Business Growing

Data Growing

Databases Growing

Enterprises Growing


Evolution of Enterprise ERP

9

BI



• ETL• Data Mart• Data Warehouses • Business Intelligence

Data Mart, Data Warehouse are the core of ERPAimed at efficiency, productivity, RoI, bottom line, customer satisfaction, etc.

However, it did not contribute to BI!

3 tier and 4 tier architecture

10

BI



Operational Issues1. ERP facilitate Silos

2. Propriety databases,

3. Interoperability is impossible

4. Manual data extraction, error-prone

5. Time consuming to extract data

6. Subject to manipulations and overwrite

7. Results are often rejected

8. unable to keep pace with the business growth


(CFO publishing, “The high cost of change for ERP” 2009; Gary Simon, CODA group publishing FSN, 2016).

11

BI




Enterprise Issue 1. Inflexible and inability to scale2. No single source of truth3. Require Expert to operate4. Heterogeneous data sources, not

for managers or end-users5. High Cost and low RoI6. Vendor in control of Data and DW7. No security, privacy, trust8. Data procurement is difficult

High cost, average $300m per year CFO publishing, “The high cost of change for ERP” 2009; Gary Simon, CODA group publishing FSN, 2016

12

2. Why Enterprise on BlockchainDW and DM has no future!

Interoperability should be forgotten

BI


Issue 1: Inflexible • Expensive One-off service,

build once, last once.• Don’t adapt to the changes

of business processes and policies.

• Requires DW and DM to change, when the business or processes are changed, but only Expert can do the job.

• Long lead time to get the change done

• Overtime, the one-off DW/DM only capture or use in-complete data, not the total data set

• hinder the org or enterprise development

13

BI


• Data lakes

• The end of data silos era

• Single source of truth

DW and DM has no future! Interoperability should be forgotten

14

BI


Blockchain1. Distributed BC Ledger2. Single Source of Truth3. Public and Private4. Smart Contract5. Transparency6. Immutability7. Open Source8. Traceability9. Consensus10. Anonymity11. Trust

Ledger

TrustSecurityRisks Accountability

DW and DM has no future! Interoperability should be forgotten

Traditional ERP/DB

Shut your data up

• Homogenous

• Schema controlled

• Data Relations, normalised

• No horizontal scale out

• High licensing cost

• Lacks high velocity

• Access through joining tables

• Centralised, single machine

Blockchain DB (NoSQL)

Let your data sing

• Heterogeneous

• Schema-less data

• Non-related and de-normalised

• Optimised to horizontal scale No license – pay as you go

• Rapid Growth

• Rapid access through one table

• Distributed and replicated

Traditional DB cannot meet the dynamic business and operational needs

3. Shut you data up or Let you data sing

15

# Traditional Structured DB/ERPSQL, shut your data up

Elastic and Blockchain DBNoSQL, let you data sing

1 Only SQL (Structured) No just SQL, but also Non-SQL

2 Table, Rows and Columns Collections and Documents

3 Fixed columns, cannot scale out Wide and no limit columns, can scale horizontally and vertically

4 Homogenous data type only (numbers inside table)

Heterogeneous (tables, bags, sets, arrays, strings, nested text, documents, images, maps, sound, scans, movies, PDF, excel, etc.)

5 Centralised server storage, single point of failure

Distributed storage (replication), fault tolerant

6 Consistency Consistency, Availability, Partition tolerance

7 Unable to analyse data quickly Real time data analytics

8 Costly to change and upgrade Dynamic upgrade

16

3. Shut you data up or Let you data sing

ERP & Legacy DBsBusiness Operation

Multi-system access

Blockchain ApplicationTrusted Data and Operation

People/Operation, Policy/Process, Workflow, Report/Data Data, Data/Process Flow, Smart Contract, User/System Interfaces

Data Migration

Process Automation

Smart Contract

Roles and Accountability

Target Business Re/Engineering

Target System Re/Engineering

Push data to Blockchain

Old OperationContinue

One Dashboard

DB1 DB2

SaveSave

New OperationNew Business

Business Logic

Run Blockchain in Hybrid Environment

Target Application and Data Migration

Heterogeneous DBs

Blockchain Data Lake

17

4. Enterprise Blockchain and Data Migration

Blockchain - How

Data Models

One of the two top key difference from the traditional DB

18


Data Model – An Order in Traditional DB Models

Above, each box is a table. An order is a one-time operation, however, It has to be stored in 4-8 tables. This is due to the normalisation rules (relational calculus), that we need to split the data into tables.When we try to find a particular order, we need to get the data from several tables, called “joint”, this takes time.

19


Order Table (Tax Invoice)

Tax Invoice #DateTimeSupplier CompanyAddressContact details Item #Item QuantityItem PriceTotal pricePurchaser NameSignatureetc

It is only one table. So, an Order is a No-SQL Table or a ledger. We do not need to have several tables. This is a natural way we do things. When we need to find an Order, we just give the Tax Invoice #, then you have all the information on one query, not several quires to several tables.20

Data Model – An Order in Blockchain Data Model


For Elastic search, we can indicate what we want to search. Eg, by given an item#, I can find which order it is in; or by given a Total price, we can find out who ordered, where it coming from, and the entire information. The Red highlights, are called “Indexing”.

Order Table (Tax Invoice)

Tax Invoice #DateTimeSupplier CompanyAddressContact details Item #Item QuantityItem PriceTotal pricePurchaser NameSignatureetc

An Order is a No-SQL file or a “Document”, just like “Tax Invoice” is a document, or it is one row in a ledger. So, this is a natural way to treat them as a “document”.A collection of the documents or collection of the Tax Invoices, we call it “Collection” of the document, instead of call it a "Table”.

21


Data Model – An Order in Blockchain Data Model

Block and Hash to Blockchain Ledger

Creating the DB is very similar to the traditional DB, instead of only structured tables, you can have structured and unstructured tables. You collect the data and “post” them to the corresponding tables. At the end of each data push session, say each Audit at the end of each day, you create a block and hash them /encrypt them using Blockchain infrastructure, it then stored as Ledger.

A Block is a set of collection of the data you want to store, or say a Block or say chunk of a collection, or say Audit at AMT is a Block, …you can set up the block by time, by location, by operation, by task, etc..

22


Store and Retrieve in/from Blockchain DB

Everything can be stored in NoSQL Databases, and Blockchain works on top of all NoSQL databases. It provides Hash or Encryption for immutability and security. Hash provides us with traceability, we can always use it for verification of the authentication of the data.

The hash, store and retrieve are all build-in by the Blockchain System Development Kit. It is like a TV, you just need to learn the remote control buttons and set it up.

The queries language is still apply, but no need joins of multiple tables, but through Elasticsearch

23


Security, privacy and permission

● The SDK - wallet systems● Wallets contain the public/private keys (identities)

24


Business ExpertEnd-user and Business

Requirement Orchestration

Technical ExpertTechnology Implementation Orchestration

(libraries, open sources, AI, GUI, DB)

Front-end BIDashboardGeo-spatial and MapFlow of InteractionAugmented realityGame based

GUI Orchestration

Smart ContractPolicies alignmentProcesses automationOperations updateResponsibilities and Roles

Business Process Orchestration

Backend AppSmart ContractArtificial IntelligenceMachine LearningPrediction AnalyticsElastic search

Analytics Orchestration

Blockchain

BlockchainData lakeData migrationHyper ledger

Data Orchestration

SecurityData SecurityDB securityApp securityNetwork securityMobile security

Security Orchestration

Step-by-step to Blockchain Platform


Step-by-Step push your data to Blockchain with the Platform

26


Business Efficiency through Blockchain and AI

2

National Vehicle Sampling procedure

5. Example1 – National Vehicle Management

2

SPO

Tier 2

DAA

Tier 3

ANAO

DFG

CJLOG/LAB



2

SPO

Tier 2

Key Risks• ANAO Category findings

• Threat of qualification

• Defence capabilities

• Financial accountability

DAA

Tier 3

ANAO

DFG

CJLOG/LAB



DFG

CJLOG/LAB

ANAO

3

SPO

Tier 2

DAA

Tier 3

SMEInnovation Cell



DFG

CJLOG/LAB

ANAO

3


LedgerLedger

Ledger

Ledger

Ledger

Risk 1

Risk 40+

Emerging risk

CASG

DefenceSustainment

SMEInnovation Cell

Ledger

Log bookImageConfirmation sheet

SmartContract



4

AASPO D1713E 661631643Automobile Sedan Camry Toyota

Blockchain solution – let your data sing


6

NVS MobileApp

• Status Analysis• Status Reports• Error Detection• Problem Solving

Blockchain

• Document Storage• Smart Contracts• Accountability• Accessibility• Visibility • Security

Open-source technology integration

AI

• Data & Text Mining• Automatic Document Reasoning• Automatic Object Detection• Automatic License Plate, Price... Recognition

• Opérationnel Ledger everywhere

• Single source everywhere

• One Enterprise Policy everywhere

• One Enterprise Process everywhere

• One Interface everywhere

• Unified results everywhere



Low CostCapability up,Compliance up,Performance up,DSR up.

Cost down,Error by Value down,Error by Volume down,Discrepancy down,Risks and shame down.

Low Risk

Value for Money

Sustainability

Rapid Update

7



35


5. Example2 – Inventory Pricing Management

36

Blockchain solution – let your data singQuarter 1 results


37


Quarter 2 results


Everybody Jump on It

38

6. Enterprise on Blockchain - Implementation

39

Private Blockchain


40

6. Enterprise on Blockchain - ImplementationPeer vs Hierarchy

41

6. Enterprise on Blockchain - ImplementationPeople and Role are important

42

Tracking Asset Outside and Inside Enterprise


43

6. Enterprise on Blockchain - ImplementationSmart Contract – No change

44

6. Enterprise on Blockchain - ImplementationManaging Smart Contract at both frontend and backend

45


A hybrid environment and co-exist with ERP / other legacy DB

46


Blockchain integration with existing DBs

47


An ordered workflow sequence

Permissioned Blockchain

48


1. Public Key Infrastructure2. Private keys and certificates for multi-channel private Blockchain3. Automated Smart Contract generation4. Multi-stake-holders and multi-Blockchain 5. Limitations of standards and policy

49

6. Future Challenges - Enterprise on Blockchain

1980 1990 2000 2010 2020

Time

Adaptability

50 years of Enterprise Information SystemThe future Business and Enterprises are with Blockchain

50

You should also Jump on It

52

UNSW and ADFA

Comments and Questions to: [email protected]