innovate by - smart city expo world€¦ · world to the digital world, bringing insight into...
TRANSCRIPT
Innovate
with
Data &
Analytics
By:
Raj Dalal,
Principal, BigInsights
Greg Baker
Snr Data Scientist
1
Emerging Technology in Big Data Analytics
AI/Cognitive
Smart City Progression: IoT and Data
1. Connected: IoT is connecting the physical
world to the digital world,
bringing insight into everything
around us
2. Responsive: Real-time situational awareness
and precise data allow cities &
businesses to personalize
services
3. Predictive: Analytics combine datasets to
predict events and drive proactive
actions that shape and accelerate
outcomes
4. Automated: Recommendations can be routed to
machines to automate and optimize
some city processes and
operations
Innovate with Data
Security / IT Infrastructure Ops
Industry Application FSI * Manufacturing * Retail, * Government * Healthcare *
Telecoms/Media * Utilities
Acquire, Grow & Retain Customers Personalisation * Profitability * Acquisition * Retention * 720 degree view of customer
Optimize Supply Chain * Reduce Fraud * Predictive Maintenance
Next Gen Data Platform
| © Copyright 2015 Hitachi Consulting 5
Internet of Everything: New Solutions
Transports
Vehicles in
restricted lanes,
wrong-way
driving,
speeding, vehicle
classification,
density, and
counting,
tailgating at entry
points
Public
Safety
Loitering,
unauthorized
entry,
predictive
crime analytics
Suspicious
Behavior
Suspicious
gatherings,
disturbances
Border
Protection
Wrong-way
walking,
crossing
forbidden
areas
Public
Transportation
Abandoned/los
t packages,
counting
people,
forbidden
areas
Vandalism
Camera-
tampering and
graffiti
From city and region-wide deployments to vertical applications
CITY DATA EXCHANGE Big Data Information
exchange platform
City of
Copenhagen
A Co-Creation With:
Danish
Capital Region
| © Copyright 2015 Hitachi Consulting 7
5. Copenhagen
Copenhagen scored high on connectivity
and carbon emissions
‘Big Data’ platform for Copenhagen on which data collection, integration and sharing is centralised for the entire city
CITY DATA
EXCHANGE
B2B Marketplace
Public and Private Data
Cloud-Based CDE-as-a-Service
Data Privacy – No Personal Data
City Data Exchange Vision Enables Data Suppliers to find Data Consumers
Data Suppliers • City Open Data
• Transportation / Parking
• Telecom Data
• Sensor Data
• Financial Transactions
• Energy Data
• Water Usage Data
• Event Data
• Weather / Environmental
• Social Media
• Citizens
Data Consumers • City Departments
• Public Authorities
• Retailers
• Property Development
• Property Management
• Transportation and Parking
Providers
• Insurance Companies
• Application Developers
• Consulting Firms
Focus on
Critical
Business
Challenges
Gather all
Relevant Data/
Instrument
everything
Store in Raw
format in central
data platform
Analytical
tools &
models
Refinement of
Hypothesis
Test & Verify
Integrate with
operational
process
BigInsights
7 Step
Methodology
Incremental
Improvement
Building a Data Innovation Team
IT Engineering
Data Science
(Maths/Stats)
Domain
Expertise
(Business
Analysts)
Analytics First Approach : Big Data vs Small
Data
© Copyright 2016, Confidential BigInsights, Tibra
Big Data:
• Too big to fit into a single computer (Usually > 10GB
• Hundreds of millions of data points
Small data:
• Small doesn’t mean unimportant
• Good enough for predictive analytics
Yesterday’s big data problem is today’s small data problem
Open source (free) big data tools
• Don’t feel obliged to work only with today’s big vendors proprietary
solutions – Start in the Cloud to experiment and store data
• Train up your own staff
– Coursera, other online course materials
– Data Science from Scratch (First Principles with Python) by Joel Grus
– We run courses, or provide consulting support to help your staff
• Doesn’t require a PhD, isn’t rocket science
– But PhD / Masters intern programs often like this sort of project
© Copyright 2016, Confidential BigInsights, Tibra
Acquiring big data sets
• Buy lots of sensors (talk to every vendor today)
• Or,
– Piggyback on security cameras
• Count things
• Identify broken infrastructure
– Piggyback on Wifi infrastructure
• Count people
– Audio
• Security cameras often have audio
• Is very cheap to store
– Text analytics
• Social media posts, especially community forums
© Copyright 2016, Confidential BigInsights, Tibra
Technologies Greg will show you
• Python, R
• Jupyter
• Scikit Image and OpenCV (image analysis)
• NLTK (natural language toolkit)
• Sentiwordnet (sentiment analysis)
• Scikit Learn (machine learning and predictive analytics)
• Numpy (numeric python library)
• All zero cost
• Widely supported and used
• Suitable for datasets smaller than <10GB
© Copyright 2016, Confidential BigInsights,
Innovate
with
Data &
Analytics
By:
Raj Dalal,
Principal, BigInsights
Greg Baker
Snr Data Scientist, BigInsights
15