innovate by - smart city expo world€¦ · world to the digital world, bringing insight into...

Innovate

with

Data &

Analytics

By:

Raj Dalal,

Principal, BigInsights

raj@biginsights.co

Greg Baker

Snr Data Scientist

greg@biginsights.co

1

Emerging Technology in Big Data Analytics

AI/Cognitive

Smart City Progression: IoT and Data

1. Connected: IoT is connecting the physical

world to the digital world,

bringing insight into everything

around us

2. Responsive: Real-time situational awareness

and precise data allow cities &

businesses to personalize

services

3. Predictive: Analytics combine datasets to

predict events and drive proactive

actions that shape and accelerate

outcomes

4. Automated: Recommendations can be routed to

machines to automate and optimize

some city processes and

operations

Innovate with Data

Security / IT Infrastructure Ops

Industry Application FSI * Manufacturing * Retail, * Government * Healthcare *

Telecoms/Media * Utilities

Acquire, Grow & Retain Customers Personalisation * Profitability * Acquisition * Retention * 720 degree view of customer

Optimize Supply Chain * Reduce Fraud * Predictive Maintenance

Next Gen Data Platform

| © Copyright 2015 Hitachi Consulting 5

Internet of Everything: New Solutions

Transports

Vehicles in

restricted lanes,

wrong-way

driving,

speeding, vehicle

classification,

density, and

counting,

tailgating at entry

points

Public

Safety

Loitering,

unauthorized

entry,

predictive

crime analytics

Suspicious

Behavior

Suspicious

gatherings,

disturbances

Border

Protection

Wrong-way

walking,

crossing

forbidden

areas

Public

Transportation

Abandoned/los

t packages,

counting

people,

forbidden

areas

Vandalism

Camera-

tampering and

graffiti

From city and region-wide deployments to vertical applications

CITY DATA EXCHANGE Big Data Information

exchange platform

City of

Copenhagen

A Co-Creation With:

Danish

Capital Region

| © Copyright 2015 Hitachi Consulting 7

5. Copenhagen

Copenhagen scored high on connectivity

and carbon emissions

‘Big Data’ platform for Copenhagen on which data collection, integration and sharing is centralised for the entire city

CITY DATA

EXCHANGE

B2B Marketplace

Public and Private Data

Cloud-Based CDE-as-a-Service

Data Privacy – No Personal Data

City Data Exchange Vision Enables Data Suppliers to find Data Consumers

Data Suppliers • City Open Data

• Transportation / Parking

• Telecom Data

• Sensor Data

• Financial Transactions

• Energy Data

• Water Usage Data

• Event Data

• Weather / Environmental

• Social Media

• Citizens

Data Consumers • City Departments

• Public Authorities

• Retailers

• Property Development

• Property Management

• Transportation and Parking

Providers

• Insurance Companies

• Application Developers

• Consulting Firms

Focus on

Critical

Business

Challenges

Gather all

Relevant Data/

Instrument

everything

Store in Raw

format in central

data platform

Analytical

tools &

models

Refinement of

Hypothesis

Test & Verify

Integrate with

operational

process

BigInsights

7 Step

Methodology

Incremental

Improvement

Building a Data Innovation Team

IT Engineering

Data Science

(Maths/Stats)

Domain

Expertise

(Business

Analysts)

Analytics First Approach : Big Data vs Small

Data

© Copyright 2016, Confidential BigInsights, Tibra

Big Data:

• Too big to fit into a single computer (Usually > 10GB

• Hundreds of millions of data points

Small data:

• Small doesn’t mean unimportant

• Good enough for predictive analytics

Yesterday’s big data problem is today’s small data problem

Open source (free) big data tools

• Don’t feel obliged to work only with today’s big vendors proprietary

solutions – Start in the Cloud to experiment and store data

• Train up your own staff

– Coursera, other online course materials

– Data Science from Scratch (First Principles with Python) by Joel Grus

– We run courses, or provide consulting support to help your staff

• Doesn’t require a PhD, isn’t rocket science

– But PhD / Masters intern programs often like this sort of project

© Copyright 2016, Confidential BigInsights, Tibra

Acquiring big data sets

• Buy lots of sensors (talk to every vendor today)

• Or,

– Piggyback on security cameras

• Count things

• Identify broken infrastructure

– Piggyback on Wifi infrastructure

• Count people

– Audio

• Security cameras often have audio

• Is very cheap to store

– Text analytics

• Social media posts, especially community forums

© Copyright 2016, Confidential BigInsights, Tibra

Technologies Greg will show you

• Python, R

• Jupyter

• Scikit Image and OpenCV (image analysis)

• NLTK (natural language toolkit)

• Sentiwordnet (sentiment analysis)

• Scikit Learn (machine learning and predictive analytics)

• Numpy (numeric python library)

• All zero cost

• Widely supported and used

• Suitable for datasets smaller than <10GB

© Copyright 2016, Confidential BigInsights,

Innovate

with

Data &

Analytics

By:

Raj Dalal,

Principal, BigInsights

raj@biginsights.co

Greg Baker

Snr Data Scientist, BigInsights

greg@biginsights.co

15