predictive maintenance - architecting a solution with devices, services, big data and predictive...
TRANSCRIPT
Uli HommanChief ArchitectWW Services
Predictive MaintenanceArchitecting a Solution with Devices, Services, Big Data and Predictive Analytics
ARC301
Marc MercuriArchitect MgrApplied Incubation
Mark KottkeArchitectApplied Incubation
Michael EpprechtArchitectModern Apps CoE
Presented in 2014
Session Objective(s): Describe the predictive maintenance scenario and identify relevant technologies in the MS stack.Define architecture patterns core to the end to end scenario
Session Objectives And Takeaways
Vs. Preventative Maintenance
Connectivity benefits the customer AND the OEM
• Remote Monitoring• Power Grid
• Tolling• Traffic• Navigation
• Safety
3rd Party Services
Road
Vehicles• Social Networking
• Connected Devices• Mobile Network
Operator
• Retail• Insurance• Infotainment
Long term opportunity
Note: Illustrative based on potential one percent savings applied across specific global industry sectors.Source: GE estimates
What if… Potential Performance Gains in Key SectorsIndustry
Aviation
Oil & Gas
Rail
Healthcare
Power
Estimated Value Over 15 Years (Billion nominal US dollars)
Segment Type of Savings
Commercial
Gas-fired Generation
System-wide
Freight
Exploration & Development
1% Fuel Savings
1% Fuel Savings
1% Reduction in System Inefficiency
1% Reduction in System Inefficiency
1% Reduction in Capital Inefficiency
$30B
$66B
$63B
$27B
$90B
Alerts, Analytics, Events and Access
Workflow and Business
Process AutomationThroughput and OEEVisibility, Role-based
and Mobile
Innovative approaches for transformation
People Assets InformationProcess
Productive environments
Sustainable Performance
Reliable process
capabilities
Informed decision making
• Familiar interaction with natural user interfaces
• Simple Role-oriented workspace
• Collaborative communication
• Connected • Remote
monitoring• Business
continuity
• Control• Quality• Standardization• Flexibility
• Secure and timely visibility
• Complete, contextual, accurate
• Predictive and actionable
Monitor, mine, manage pattern1. Monitor and collect events2. Mine system events to produce active model (e.g.
fraud detection, preventative maintenance)3. Manage active event stream via event engine
Event Engine
ModelGeneration
Digital Shoebox
3
21
Data is acquired from devices, sensors, applications and people.
Evaluation, storage, and processing of data is done locally
as appropriate.
If the local implementation is a hub and spoke design, data is communicated to a locally connected hub.
Appropriate data is transmitted to a public or private cloud.
The specific data transmitted, how it is transmitted, and timing of transmission is determined by policy. Policy includes considerations of “three Cs” – context, connectivity, and cost.
Services are utilized to deliver the data, store it, and if appropriate, initiate one or more associated data pipelines.
Compute Storage
Analytic pipelines perform analysis and generate insight. The resulting data delivers one of four things –
• Information for Subscribers• Enhancements of Existing Data• Recommend Action(s)• Initiation of Action(s)
Compute Storage Analytics
Pipeline(s) Insight
Insight is delivered to appropriate human, device, and application audiences in the forms of -
• Alerts/Notifications• Reporting• Command + Control• Data Services• Personalized User Experiences
Compute Storage Analytics
Pipeline(s) Insight
A diversity of
Peer-to-Peer
Device-to-Service Service-to-DeviceMachine-to-Machine communication is non-interactive, automated, and bi-directional information exchange in
operational systems, performed between peers or between satellite systems and their supporting backend services.
Connectivity Patterns
Connectivity considerations•
•
•
•••
•
•
•
••••••
••••
••••
•
•••
•
•
•
•
••
•
Common Activities and Composability
Peer-to-Peer
Device-to-Service Service-to-Device
Service-to-Service
Information Exchange Patterns
Telemetry
Information flowing from a device to other systems for conveying status of device and environment
Inquiries
Requests from devices looking to gather required information or asking to initiate activities
Commands
Commands from other systems to a device or a group of devices to perform specific activities
Notifications
Information flowing from other systems to a device (-group) for conveying status changes in the rest of the world
Telemetry Types•
•
••
••
Telemetry Considerations•••
•••
•••
•••
Signal Characterization••
••
••
••
Policy Considerations••
••
••
The “three Cs” will help determine the appropriate telemetry to deliver at any given time.
Architectural BaselineScale Unit
x10,000 devices
Data Analysis Pipeline(s)
Gateway
Filtering and Aggregation
Routing
Control System
ScaleUnit
ScaleUnit
ScaleUnit
x1,000,000 devices
ScaleUnitDC Boundary
Device Identity
and Metadata
Store
Provisioning System
Data
Ser
vice
s
Gateway Core Architectural Components1. Custom Protocol
Gateway2. Telemetry Pump
and Adapters3. Command
Gateway4. Provisioning
Service and Metadata Store
Windows Azure Service Bus Messaging
Custom Protocol Gateway Host
MQTT CoAP … …
Telemetry/Request Router
Notification/Command Router
Adapters Command API Host
Provisioning Service
Device Metadata and Key
Store
HDIn
sight
BizT
alk
Sv/S
rOr
lean
sAz
ure
Stor
age
SQL
SB
HTTP
HTTP
Devices
AMQP
1
2 3
4
Configs
• Telemetry Adapters take ingress data then adapt and deliver it to raw storage, data stores and/or associated technologies.
• Data Pipelines are workflows that process data with the intent of transformation and/or generation of insight.
• Adapters and Pipelines can also be re-entrant, transforming data and publishing back into it into telemetry processor.
Telemetry Adapters and Data Pipelines
Real-time* Analysis• Observe Telemetry “as it
happens”• React to state changes or trends• React to aggregate observations
• Examples• “device input voltage drops below 11V for more than
3 minutes”• “temperature readings from sensors on this floor
average above 23°C for last 10 minutes”• “sensor failed reporting data for 5 minutes”
• Very short reaction time required
fn
Data-At-Rest Analysis• Mine Telemetry through DB
Queries• Find and track trends or maxima• Analyze expected vs. actual behaviors• React to longer term observations• Hoard for future use
• Variety of Data Store Options• SQL/OLAP• Cassandra, Riak• Hadoop/HDInsight
• Store choice depends on what questions you’d like to ask
flt
Data Lakes
Data Lake
• Data volume and velocity growing• Storage is cheap• With data warehouses… • Designed to answer questions you have today• Elements and attributes not needed often dropped / lost
• Data lakes…• Keep all data for future needs – known/unknown• Includes meta-data tags to help find the data you need later• Feeds data pipelines for downstream needs.. including data
warehouses
Supervised LearningA supervised learning algorithm analyzes labeled training data and produces an inferred Function which can be used for mapping new examples
Training Data• Training data consist of a set of training examples• Each example is a pair consisting of an input object and a desired output
level• As real data evolves/changes, the algorithm can be run against new
training data
Sensor X
Age
What’s the likelihood that a machine will fail soon given device age and data received from sensor X?
MaintInterval
Daily Usage
Supervised Learning Examples What is the ideal maintenance intervalbased on an machine’s daily usage?
Inputobject
Desired
Output
“Not Failing Soon”
Classification
“Failing Soon”Classification
Unsupervised LearningFinding hidden structure when you don’t know the answers.Often finding clusters within the data
Examples• Market segmentation analysis• Organizing computing clusters• Grouping web content, e.g. news stories• Social network analysis X1
X2
Data Pipeline ComponentsData Analysis Pipeline(s)
Data
Ser
vice
s
1 2
3
4
5
6
1. Hadoop2. R3. RDBMS/SQL4. NoSQL5. Storage6. Codename
“Passau”7. ASP.NET Web API
7
Project Passau
• Easy data exploration through web-based interface
• No programming required
• Flexible and extensible
Hadoop Options
•
•
•
Resulting Insight Types••
•
•
••
Command/Control• Tell a device, remotely, to execute a
logical or physical activity• “Give me the status of X” • “Roll 2 feet forward”• “Track this object with the camera”• “Fetch firmware update”
• Remote: Control service, handheld device, etc.
• Latency requirements vary, but often “perceptibly imminent”
Data Services
Deliver Using Open Standards••
•
••••
•
•
•
Consuming Insight•
•
•
•
••
••
Case Study - Altran
Altran Overview• Hi Tech Engineering &
Consulting • $2B 2012 Revenue, 20K
Employees• Aerospace, Auto, Energy,
Life Sciences, Media, Rail, Government
Daily Data Analysis for 50K vehiclesSimulate 1 Month of data Ingest Data @50K msgs/secShow results in Windows 8 dashboard
Data Harvester
Architectural BaselineScale Unit
x10,000 devices
Data Analysis Pipeline(s)
Gateway
Filtering and Aggregation
Routing
Control System
ScaleUnit
ScaleUnit
ScaleUnit
x1,000,000 devices
ScaleUnitDC Boundary
Device Identity
and Metadata
Store
Provisioning System
Data
Ser
vice
s
Device Gateway – Partition Topology
• “Master” manages device provisioning and partition deployment/configuration for all or a well-defined subset of partitions (e.g. one continent)
• “Partition” is a set of resources focused on handling data from a well-defined and known defined device population that has been assigned to and configured into the partition through provisioning. Cross-partition distribution of devices is based on solution-specific logic, allocation within the partition is handled by provisioning.
PartitionMaster
Provisioning API
Provisioning Runtime
Partition Repo
Ingestion Topics Egress
Service Bus Standard Protocol Custom Protocol
Device Repo
Access Control
in0000 inFFFF…in0001 in0002
AMQPS HTTPS MQTT Custom Protocol HostProtocol Adapters
diagall dia
gall diagall dia
gall
Telemetry PumpN Instances
Telemetry Adapter
Telemetry Adapter
Telemetry Adapter
Deployment Runtime out000
0 outFFFF…out0001
out0002
s000 1
s000 2
s03E 7 s000 1
s000 2
s03E 7
s000 1
s000 2
s03E 7
s000 1
s000 2
s03E 7
g0000/rte0000
g0001/rte0000
g0000/rte0001
g0001/rte0001
out0
out1
out2
out0
out1
out2
out0
out1
out2
out0
out1
out2
n Groups of m Routers
SB AMQPS
Altran ArchitectureVehicle Data Simulated Vehicle
Data
HDInsightAzure Blob Storage (ASV) Azure SQL DB
Client Data Services
(Worker Role)
Demo
In Review: Session Objectives And TakeawaysDescribe the predictive maintenance scenario and identify relevant technologies in the MS stack.Define architecture patterns core to the end to end scenario
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.