Anant Venkateswaran

Kishor Narang

Tutorial Track 2

A Primer on the Next Generation in Big Data Analytics amp Visualzation and its

applications to the Utility Industry

UU 201 A PRIMER ON BIG

DATA AND ENERGY GRID

ANALYTICS ROAD TO THE

FUTURE

Legal Notice

bull All material contained in this presentation shall not be reproduced or reused in any form without the express permission of the authors

Agenda

Introductions amp Objectives

Big Data amp Analytics ndash An Introduction

Big Data amp Analytics ndash Utility Applications

Big Data amp Analytics ndash Applications in the customer side ndash DR EE Renewables

Future amp Closing Comments

Introductionsbull Name Company Rolebull How are you using Analytics todaybull What are you looking to learn

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

UU 201 A PRIMER ON BIG

DATA AND ENERGY GRID

ANALYTICS ROAD TO THE

FUTURE

Legal Notice

bull All material contained in this presentation shall not be reproduced or reused in any form without the express permission of the authors

Agenda

Introductions amp Objectives

Big Data amp Analytics ndash An Introduction

Big Data amp Analytics ndash Utility Applications

Big Data amp Analytics ndash Applications in the customer side ndash DR EE Renewables

Future amp Closing Comments

Introductionsbull Name Company Rolebull How are you using Analytics todaybull What are you looking to learn

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Legal Notice

bull All material contained in this presentation shall not be reproduced or reused in any form without the express permission of the authors

Agenda

Introductions amp Objectives

Big Data amp Analytics ndash An Introduction

Big Data amp Analytics ndash Utility Applications

Big Data amp Analytics ndash Applications in the customer side ndash DR EE Renewables

Future amp Closing Comments

Introductionsbull Name Company Rolebull How are you using Analytics todaybull What are you looking to learn

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Agenda

Introductions amp Objectives

Big Data amp Analytics ndash An Introduction

Big Data amp Analytics ndash Utility Applications

Big Data amp Analytics ndash Applications in the customer side ndash DR EE Renewables

Future amp Closing Comments

Introductionsbull Name Company Rolebull How are you using Analytics todaybull What are you looking to learn

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Introductionsbull Name Company Rolebull How are you using Analytics todaybull What are you looking to learn

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Objectives

bull The origins of big data its evolution as a key asset and how it is very different from the traditional analyticsBI

bull We will discussndash Key analytics areas within the utility the operational rhythm of

the analytics platform as it is leveraged to deliver sustainable value over time

ndash Challenges that remain when trying to combine the analytics needs of the entire utility into a cohesive energy analytics portfolio

ndash How the industry as a whole is responding to this new set of requirements

bull These points will be supported by actual Utility case studies

6

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Regulatory Perspective

bull Grid 10 ndash 20th Centurybull Grid 20 ndash Smart Grid Development

ndash Sensors and Switches ndash Self-healing

bull Grid 30 ndash Advanced Grid Operations ndash Two-way power flowsndash Customer centric ndash Enhanced data and analyticsndash Extracting value from distributed energy resources

7

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Grid 30 Electric Distribution System

8

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Utility 30 ndash one scenario

9

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

So ndash what does this mean The utility consumer compact is changing

bull Changing customer expectations and actions

bull IT technologies that are cheaper consume less power and are smaller in size

bull More alternatives to solving the same problem

10

Source elpcom

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

The various dimensions of change

11

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

A Roadmap to consider

12

Smart Grid Early Adopters

Mar

ket

Dri

vers

Optimize the grid and Sweat assets

2010 2012 2015 2020 2030

Integrate Distributed Energy

Industry Restructuring

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Areas of Change ndash Lessons from NY REV

bull Distribution Planningbull Distribution Grid Operationsbull Distribution Market

Operationsbull Data Requirementsbull Platform Technologies

13Source hitachicojp

Source Modern Grid Solutions

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Utility areas of consideration

bull Focus on business benefits and not technology for the sake of technology

bull Invest in technology based on business need and where benefits justify the cost

bull Anticipate the technology the costs and benefits to change over time

bull Anticipate and expect new technologies to declare existing solutions obsolete

bull Anticipate the role of Big Data amp how it impacts your entire value chain

bull Train your personnel as well as your customers

bull Anticipate new players to come in and threaten the utility business model and the sources of revenue

14

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Policy Focus for Grid 30

bull Data Sharing and Privacy Protection

bull Depreciation Schedules

bull Distributed System Operator Development

bull DER Aggregation

bull All of it is ndash Big Data

15

ResidentialES System

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Customer EngagementGetting the Price Right

bull The Customer Bill

( + ) Charges for system utilization

( - ) Credits for value of resources

( + ) Taxes and Fees ___________________________________________

= Customer Bill

16

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Big Data amp Analytics

17

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Topics

bull In this section we will coverndash The Big Data the hype ndash is it valid and why Now

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Data Scientist Role

ndash Utility Applications and Use Cases

ndash Big Data and the Cloud

ndash How to pick what is right for you

18

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Global Drive to connect everything

Factories $1950 31

Smart Grid $757 12

Smart Building $349 5

Connected Commercial

Vehicles $347

5

Other $2969 47

Smart Metering $100

Other Smart Grid $657

M2M $6372

M2P $3501

P2P $4519

$14 Trillion Internet of Everything by 2022

$64 Trillion M2M by 2022 Smart Grid 12

$100 B Smart Metering by 2022 bull Smart Grid 12bull M2M ndash 16bull IoE - 07

Global Interoperable Standards will drive future AMI amp DA Architectures and Solutions

Source Cisco IBSG 2013

Market Size in $Billions

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Source Gartner August 2013

bull IoT

bull Mesh

bull Predictive Analytics

Big DataGartner Hype Cycle

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

ldquoBig Data amp Analyticsrdquo ndash Big Questions

bull What is ldquoBig Datardquo

ndash Large datasets

ndash Multiple data sources

ndash ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

bull Is it just the next industry buzz word

bull Why does it matter

ndash Smart Grid deployments

ndash Greater customer needs

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

What are AnalyticsCommon Definitions

bull Analytics is the discovery and communication of meaningful patterns in data Especially valuable in areas rich with recorded information analytics relies on the simultaneous application of statistics computer programming and operations research to quantify performance Analytics often favors data visualization to communicate insight ndashSource Wikipedia

bull Data analytics (DA) is the science of examining raw data with the purpose of drawing conclusions about that information Data analytics is used in many industries to allow companies and organization to make better business decisions and in the sciences to verify or disprove existing models or theories Data analytics is distinguished from data mining by the scope purpose and focus of the analysis Data miners sort through huge data sets using sophisticated software to identify undiscovered patterns and establish hidden relationships Data analytics focuses on inference the process of deriving a conclusion based solely on what is already known by the researcher ndash Source What iscom

22

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

23

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Evolution of Analytics

Source Accenture 24

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

25

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Analytics 10 - Traditional Approach

26

Data Storage

Data Presentation Visualization

Business Processes

Leverage Existing Utility Business Processes

Extraction Validation Cleaning amp Data Delivery

ValidatedData

Reporting Engine

Web Portals

Dashboard

Excel Reports

On-Demand Queries from DBrsquos

Ad-hoc Reports

Heterogenous Data Sources

Meter AMI

SCADA

DMS OMS

GIS

CIS CRM

Non- Utility

Post Big Data Traditional Analytics will continuehelliphelliphelliphellip

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Which Approach Works Better

bull Start big data analytics project with a specific use case or problem to solve

OR

bull Start dumping data to store and analyze later

Short Term vs Long Termhelliphellip

27

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

28

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Analytics 20 ndash ldquoBig Datardquo

bull No single standard definition

bull ldquoBig Datardquo is data whose scale diversity and complexity require new architecture techniques algorithms and analytics to manage it and extract value and hidden knowledge from ithellip

29

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Analytics 20 Big Data - DefinitionsAccording to Wikipedia

bull Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications

bull The challenges include capture curation storage search sharing transfer analysis and visualization

bull The trend to larger data sets is due to the additional information derivable from analysis of a single large set of related data as compared to separate smaller sets with the same total amount of data allowing correlations to be found to spot business trends determine quality of research prevent diseases link legal citations combat crime and determine real-time roadway traffic conditions

30

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Does ldquoBig Datardquo Matter bull The New York Stock Exchange generates about one terabyte

of new trade data per day

bull Facebook hosts approximately 10 billion photos taking up one petabyte of storage

bull Ancestrycom the genealogy site stores around 25 petabytes of data

bull The Internet Archive stores around 2 petabytes of data and is growing at a rate of 20 terabytes per month

bull The Large Hadron Collider near Geneva Switzerland will produce about 15 petabytes of data per year

31

Source Hadoop the definitive guide

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Big Data ndash Key Attributes

bull Volume

bull Velocity

bull Variety

bull Veracity

bull Value

32

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Comparing Analytics 10 vs 20

33

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Data Size Gigabytes (Terrabytes) PetaBytes (Hexabytes)

Access Interactive and Batch Batch

Updates Read and Write Many Times Write Once Read Many times

Structure Static Database Schema Dynamic Schema

Integrity High Low but improving very fast

Scaling Linear Non-Linear

Content Tools RDBMS Centric

VisualizationBig Propreitary

Cloud-Centric Massively ParallelVisualization criticalLittle Propreitary (Open Source)

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Comparing Analytics 10 vs 20

34

Description Traditional RDBMSAnalytics 10

Map Reduce (Big Data)Analytics 20

Business IT Owned typicalTeamsMedium Large Size ProjectsMore Governance

Analytics ndash Owned (IT OT)One ndash Offs as neededQuicker hitsLess Governance

Data Complete DataQuality Centric Structured Data

Mostly Internal DataBusiness Analyst DBA

Missing Partial DataQuantity CentricStructured + Unstructured DataInternal + External DataData Scientist

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Typical Big Data Assumptions

bull Hardware will failhellip

bull Processing will be in batches High throughput vs low latency

bull Applications will have large data sets

bull High aggregate data bandwidth and scale to hundreds of nodes

bull Moving computation is cheaper and more efficient that moving data

35

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Big Data Tools

Just like traditional analytics

the key components are

bull Storage

bull Processing

bull Query

bull Presentation and

Visualization

36

SPLUNK ndash Log files mining

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Big Data - Technologies

37Source George Siemens blog

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Landscape

38Source Forbescom

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Grid Analytics Landscape

39

Simulations

Distribution Analytics

Transmission Analytics

Distributed Energy Analytics

SubstationAnalytics

Outage Analytics

Asset Load Control Management

Meter Analytics

Consumer Analytics

Visualization

C amp I AnalyticsAnalytics Aggregation

Big Data Analytics Infrastructure

40Source DataScienceCentralcom

Sample Architecture

41Source DataScienceCentralcom

Big Data Flow

4242

Sample Data Platform

43Source Hortonworks Data Platform (HP)

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

44

The Utlimate Goal - A Data Enabled Utility Enterprise

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Sample Architecture

41Source DataScienceCentralcom

Big Data Flow

4242

Sample Data Platform

43Source Hortonworks Data Platform (HP)

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

44

The Utlimate Goal - A Data Enabled Utility Enterprise

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Big Data Flow

4242

Sample Data Platform

43Source Hortonworks Data Platform (HP)

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

44

The Utlimate Goal - A Data Enabled Utility Enterprise

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Sample Data Platform

43Source Hortonworks Data Platform (HP)

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

44

The Utlimate Goal - A Data Enabled Utility Enterprise

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

An Evolution of Analytics

bull Proactive and Reactive Business intelligence Reporting

10 Classic Business intelligence amp Data Warehousing

bull Proactive Analytics Reporting

20 Big Data Emerges

bull Next Generation Applications ndash the road to the future

30 Data Enriched Applications

44

The Utlimate Goal - A Data Enabled Utility Enterprise

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

What is Driving ldquo Big Datardquo Value

45

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Big Data - The business Value

bull Let us compare the cost of the toolsndash NoSQL is 50x CHEAPER

ndash Why ndashOpen source origin and momentum VC influence

ndashWith a mission ie make cluster of commodity hardwarestorage scale for particular use

bull INodejs for graph database

bull IIMongoDB for general schema free data

bull IIICassandra on pure java cluster

bull IV Couch for general document management

46Relational DBrsquos and tools can not match the price

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Big Data in the Enterprise

bull Beyond the applications and solutions Utilities must recognize the need for governance and clear roles and responsibilities for any Big Data solution

bull Is a stand alone organization meritedndash Conversely is embedded talent more effective

bull The first step is to identify the stakeholders and responsible parties for the raw datandash This should cut across many internal organizations

including Billing Customer Operationss Engineering Field Operations IT and others

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

IT OT What is this about

A New Yorkerrsquos view of the USA can be compared to ITrsquos view of OT

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

A fresh start

bull What is the motivation Big Data needs to be a joint effort between IT and the Business

bull First steps

ndash Identifying business needs

ndash Discovering what data is available

ndash Identifying and Pairing best tools applications and integration solutions

ndash Documentation with Use Cases

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Who will deliver The Data Scientist

bull Big Data era is also the era of the ldquodata scientistndash Who understands data

ndash Who understands business

ndash Who prepares (processes and cleans) data

ndash Who asks the ldquoRight Questionsrdquo

ndash Who understands the value

ndash Who understands deployment methodology

ndash Who understands the results

50

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

How to become a Data Scientist

bull Conferences

ndash Strata

ndash Data Scientist Summit

ndash CloudCamps

ndash Bigdata conf

bull Practicendash httpwwwquoracomProgramming-Challenges-

1What-are-some-good-toy-problems-in-data-science

51

Data Scientist - Resources

52

We canrsquot forget the Cloud

bull Many Utilities want to own and operate their Information Systems

bull Buthellipndash Capital Availability

ndash Fixed costs

ndash Redundant Expenditures

ndash High Energy costs low CPU Utilization

ndash Unreliable Hardware an issue

ndash Overcapacity (Tech and labor)

53

Big Data amp the Cloud

bull Upgrades require capital investmentbull Efficiency and reliability need improvingbull New technologies are enabling new ways to optimize

energy deliverybull IT solutions are involved and expensive to managebull Aging infrastructure and workforcebull Distributed generationbull Inevitable carbon legislationbull Sense of Urgencybull SLA based

54

Data ManagementThe 10 year digital wave outlook - smarter integrated assets big data social networks

2010

2020

20 of firms donrsquot own any IT assets

Smart portable devices are most common web access method

2015

A second wave of IT transformation is underway

Source Verizon 2010

Cloud virtualised consumerized IT models top priorities

Half of businesses get 80+ of their IT services from the cloud

Late 2019All IT budgets ~100 OPEX

55Source General Electric Company

Emerging Enterprise Cloud Computing Standardsbull OCI bull EC2 APIbull Simple Storage Service (S3) APIbull Windows Azure Storage Service REST APIrsquosbull Windows Azure Service Management REST APIrsquosbull DeltaCloud APIrsquosbull RackSpace Cloud Servers APIbull RackSpace Cloud Files APIbull Cloud Data Management Interfacebull vCloud APIbull Globusonline REST API

But wersquore not done The cloud will continue to evolve

56

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

We canrsquot forget the Cloud

bull Many Utilities want to own and operate their Information Systems

bull Buthellipndash Capital Availability

ndash Fixed costs

ndash Redundant Expenditures

ndash High Energy costs low CPU Utilization

ndash Unreliable Hardware an issue

ndash Overcapacity (Tech and labor)

53

Big Data amp the Cloud

bull Upgrades require capital investmentbull Efficiency and reliability need improvingbull New technologies are enabling new ways to optimize

energy deliverybull IT solutions are involved and expensive to managebull Aging infrastructure and workforcebull Distributed generationbull Inevitable carbon legislationbull Sense of Urgencybull SLA based

54

Data ManagementThe 10 year digital wave outlook - smarter integrated assets big data social networks

2010

2020

20 of firms donrsquot own any IT assets

Smart portable devices are most common web access method

2015

A second wave of IT transformation is underway

Source Verizon 2010

Cloud virtualised consumerized IT models top priorities

Half of businesses get 80+ of their IT services from the cloud

Late 2019All IT budgets ~100 OPEX

55Source General Electric Company

Emerging Enterprise Cloud Computing Standardsbull OCI bull EC2 APIbull Simple Storage Service (S3) APIbull Windows Azure Storage Service REST APIrsquosbull Windows Azure Service Management REST APIrsquosbull DeltaCloud APIrsquosbull RackSpace Cloud Servers APIbull RackSpace Cloud Files APIbull Cloud Data Management Interfacebull vCloud APIbull Globusonline REST API

But wersquore not done The cloud will continue to evolve

56

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Big Data amp the Cloud

bull Upgrades require capital investmentbull Efficiency and reliability need improvingbull New technologies are enabling new ways to optimize

energy deliverybull IT solutions are involved and expensive to managebull Aging infrastructure and workforcebull Distributed generationbull Inevitable carbon legislationbull Sense of Urgencybull SLA based

54

Data ManagementThe 10 year digital wave outlook - smarter integrated assets big data social networks

2010

2020

20 of firms donrsquot own any IT assets

Smart portable devices are most common web access method

2015

A second wave of IT transformation is underway

Source Verizon 2010

Cloud virtualised consumerized IT models top priorities

Half of businesses get 80+ of their IT services from the cloud

Late 2019All IT budgets ~100 OPEX

55Source General Electric Company

Emerging Enterprise Cloud Computing Standardsbull OCI bull EC2 APIbull Simple Storage Service (S3) APIbull Windows Azure Storage Service REST APIrsquosbull Windows Azure Service Management REST APIrsquosbull DeltaCloud APIrsquosbull RackSpace Cloud Servers APIbull RackSpace Cloud Files APIbull Cloud Data Management Interfacebull vCloud APIbull Globusonline REST API

But wersquore not done The cloud will continue to evolve

56

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Data ManagementThe 10 year digital wave outlook - smarter integrated assets big data social networks

2010

2020

20 of firms donrsquot own any IT assets

Smart portable devices are most common web access method

2015

A second wave of IT transformation is underway

Source Verizon 2010

Cloud virtualised consumerized IT models top priorities

Half of businesses get 80+ of their IT services from the cloud

Late 2019All IT budgets ~100 OPEX

55Source General Electric Company

Emerging Enterprise Cloud Computing Standardsbull OCI bull EC2 APIbull Simple Storage Service (S3) APIbull Windows Azure Storage Service REST APIrsquosbull Windows Azure Service Management REST APIrsquosbull DeltaCloud APIrsquosbull RackSpace Cloud Servers APIbull RackSpace Cloud Files APIbull Cloud Data Management Interfacebull vCloud APIbull Globusonline REST API

But wersquore not done The cloud will continue to evolve

56

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Emerging Enterprise Cloud Computing Standardsbull OCI bull EC2 APIbull Simple Storage Service (S3) APIbull Windows Azure Storage Service REST APIrsquosbull Windows Azure Service Management REST APIrsquosbull DeltaCloud APIrsquosbull RackSpace Cloud Servers APIbull RackSpace Cloud Files APIbull Cloud Data Management Interfacebull vCloud APIbull Globusonline REST API

But wersquore not done The cloud will continue to evolve

56

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Analytics Market Overview

bull 47 of utilities already have projects going on in the planning stages

bull 72 think Grid Optimization is their 1 priority

57

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Analytics Investments areas

GTM Research has pegged the value of the global utility data analytics market at a cumulative $20 billion between 2013 and 2020 growing from an annual spend of $11 billion this year to nearly $4 billion by decadersquos end

Sources of Big Data at utilitieshellip

bull Data comes from virtually everywhere throughout the enterprisendash AMI and Metering Systemndash SCADA Platformsndash DMS amp OMS Applicationsndash GISndash CIS and CRM Systemsndash Work amp Asset Management Systemsndash Demand Response Platformshellip ndash Renewable Integration hellipand morehellip

bull Field devicesndash Sensorsndash Metersndash Commsndash DA Devicesndash Sub-Station

bull Social Media and Networksbull Mobile Devicesbull The Utility Back-office Systemsbull Customer Service Marketingbull Customer Data

The advent of Grid Modernization has changed the paradigm from few data sources and multiple consumption sinks to multiple data sources and multiple consumption sinks

Big Data in a Utility Environment

60

Proprietary and confidential Do not copy or distribute without permission from OATI copy2015 Open Access Technology International Inc 61

Business StrategyRegulatory Affairs

Bulk Power Supply

Transmission Ops

Distribution Ops

Customer Services

Utility Applications of Big Data

Long-Term Planning

Operations Planning Operations

Post Operations

ForecastingState

EstimationOperations

OptimizationCustomerMarket Analysis

PerformanceAssessment

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 62

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSec

ndash 035 GB of DataDay

ndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous)

ndash 15 GB of DataDay

ndash 10 No of Simultaneous Users0

05

1

15

2

SCADA AMI

GB of DataDay

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 63

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSecndash 035 GB of DataDayndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous) ndash 15 GB of DataDayndash 10 No of Simultaneous Users

DR ndash DER 10 penetration - mix of capabilities

ndash 5000 PointsSec ndash 69 GB of DataDay ndash 500+ No of Simultaneous Users

0

1

2

3

4

5

6

7

8

SCADA AMI DR-DER

GB of DataDay

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

bull Field devicesndash Sensorsndash Metersndash Commsndash DA Devicesndash Sub-Station

bull Social Media and Networksbull Mobile Devicesbull The Utility Back-office Systemsbull Customer Service Marketingbull Customer Data

The advent of Grid Modernization has changed the paradigm from few data sources and multiple consumption sinks to multiple data sources and multiple consumption sinks

Big Data in a Utility Environment

60

Proprietary and confidential Do not copy or distribute without permission from OATI copy2015 Open Access Technology International Inc 61

Business StrategyRegulatory Affairs

Bulk Power Supply

Transmission Ops

Distribution Ops

Customer Services

Utility Applications of Big Data

Long-Term Planning

Operations Planning Operations

Post Operations

ForecastingState

EstimationOperations

OptimizationCustomerMarket Analysis

PerformanceAssessment

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 62

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSec

ndash 035 GB of DataDay

ndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous)

ndash 15 GB of DataDay

ndash 10 No of Simultaneous Users0

05

1

15

2

SCADA AMI

GB of DataDay

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 63

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSecndash 035 GB of DataDayndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous) ndash 15 GB of DataDayndash 10 No of Simultaneous Users

DR ndash DER 10 penetration - mix of capabilities

ndash 5000 PointsSec ndash 69 GB of DataDay ndash 500+ No of Simultaneous Users

0

1

2

3

4

5

6

7

8

SCADA AMI DR-DER

GB of DataDay

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Proprietary and confidential Do not copy or distribute without permission from OATI copy2015 Open Access Technology International Inc 61

Business StrategyRegulatory Affairs

Bulk Power Supply

Transmission Ops

Distribution Ops

Customer Services

Utility Applications of Big Data

Long-Term Planning

Operations Planning Operations

Post Operations

ForecastingState

EstimationOperations

OptimizationCustomerMarket Analysis

PerformanceAssessment

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 62

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSec

ndash 035 GB of DataDay

ndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous)

ndash 15 GB of DataDay

ndash 10 No of Simultaneous Users0

05

1

15

2

SCADA AMI

GB of DataDay

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 63

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSecndash 035 GB of DataDayndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous) ndash 15 GB of DataDayndash 10 No of Simultaneous Users

DR ndash DER 10 penetration - mix of capabilities

ndash 5000 PointsSec ndash 69 GB of DataDay ndash 500+ No of Simultaneous Users

0

1

2

3

4

5

6

7

8

SCADA AMI DR-DER

GB of DataDay

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 62

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSec

ndash 035 GB of DataDay

ndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous)

ndash 15 GB of DataDay

ndash 10 No of Simultaneous Users0

05

1

15

2

SCADA AMI

GB of DataDay

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 63

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSecndash 035 GB of DataDayndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous) ndash 15 GB of DataDayndash 10 No of Simultaneous Users

DR ndash DER 10 penetration - mix of capabilities

ndash 5000 PointsSec ndash 69 GB of DataDay ndash 500+ No of Simultaneous Users

0

1

2

3

4

5

6

7

8

SCADA AMI DR-DER

GB of DataDay

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Proprietary and confidential Do not copy or distribute without permission from OATI copy 2012 Open Access Technology International Inc 63

Data Velocity and Data Volume

Consider a utility with 500000 customers

D-SCADA 5000 pts 4 sec Scan and 20 change

ndash 250 PointsSecndash 035 GB of DataDayndash 6 No of Simultaneous Users

AMI 15 min resolution - 4 point read

ndash 2220 PointsSec (continuous) ndash 15 GB of DataDayndash 10 No of Simultaneous Users

DR ndash DER 10 penetration - mix of capabilities

ndash 5000 PointsSec ndash 69 GB of DataDay ndash 500+ No of Simultaneous Users

0

1

2

3

4

5

6

7

8

SCADA AMI DR-DER

GB of DataDay

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Big Data Applications ndash Utility Industry

bull Generation

bull Transmission ndash RTO

ndash Markets

bull Imagine if the Enron ldquoCongestionrdquo happened today

bull Distributionndash Fraud Detection Theft Revenue Recovery

bull Retailndash Customer retention patterns

ndash Fraud Detection Theft Revenue Recovery

bull Consumerndash Energy Efficiency Demand Side Management Portals

ndash Behind the meter renewables integration microgrids etc 64

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Distribution Analytics Opportunities

bull Outage management damage assessment outage restorationbull Fault location fault cause identification reliability improvementbull automatic reconfigurationbull Distribution planning load forecasting customer modelsbull Real time system management and optimization new visualization approachesbull Risk assessment applicationsbull Volt Var control distribution efficiencybull Integration of renewables storage demand response distributed generation

microgridsbull Dynamic protection systems dynamic ratingsbull Asset management equipment diagnosticsbull Geographic information system management and accuracy improvementbull Customer integration customer support customer communicationsbull AMI applications

Source EPRI

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics for Distribution

66

bull There are many opportunities for integrating Analytics into a Smart Grid Solutionndash Connectivity Model Improvements

bull Auto-Generating Secondary Circuit Modelsbull Correcting Meter Phasingbull Detecting Transformer Connectivity Problems

ndash Identifying OverloadedStressed Assetsbull Proactive Transformer Replacement

ndash Locating Transformer Voltage Problemsndash Using AMI Data to Detect Theft andor Unmetered Load

bull Unbilled revenuebull Tamper detection amp Irregular Usage Patterns

ndash Improved Fault Locatingbull Using Substation Power Quality and Relay Databull Using Feeder Monitors to Locate Faults and Estimate Cause

ndash Reliability Analysis Storm Analysis and Momentary Analysis

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Connectivity Model Improvementsbull A complete and accurate connectivity model is necessary to achieve many of the

benefits

bull Principally the meter to transformer relationship is needed for

bull Outage Management

bull Transformer Load Modeling and Asset Management

bull A full circuit model is important for

bull Circuit modeling

bull Phasing

bull Voltage Management

bull Most model improvements are done manually when discrepancies are noticed by an operator dispatcher or field crew

bull The Smart Gird and Analytics can offer solutions to automatically identify discrepancies and errors They include

bull Outage amp Event Analysis

bull Outlier Analysis

bull 100 accuracy should always be a goal but recognizing the current state will help drive opportunities

bull Known problem areas can be prioritized and addressed

67

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Outlier Analysis

68

During a routine transformer outage outlier customers are identified and corrections to the connectivity models are made

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Transformer Monitoring

69

bull Utilities use analytics to determine transformer overloads and predict failures based on metering data from the AMR AMI data

bull A program was initiated to investigate daily transformer consumption data (aggregated from meter data) for failures that occurred during winter peak load days

bull Transformers with yr-to-yr load increase of more than 25 (3764 winter 1233 summer) were flagged for investigation

TRANSFORMER 142F5A8D

0

500

1000

1500

2000

2500

3000

1112

007 0

00

1212

007 0

00

112

008 0

00

212

008 0

00

312

008 0

00

412

008 0

00

512

008 0

00

612

008 0

00

712

008 0

00

812

008 0

00

912

008 0

00

1012

008 0

00

1112

008 0

00

1212

008 0

00

112

009 0

00

212

009 0

00

Dai

ly k

WH

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Transformer Load Profilesbull Once a daily transformer load shape is identified it can be used to develop

overall ldquonormalrdquo operating parameters for each device

bull If the loading suddenly exceeds the normal operating envelop an investigation can be initiated to understand why the change in load shape

bull In many cases the equipment may need to be upsized or load relief activities must be performed

70

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Voltage Analysis ndash Feeder Voltage Profiles

71

While the voltage profile is within the Tariff guidelines voltages range from redorange = gt240v to bluegray = lt 240v

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics for Reliability

bull Reliability and Outage Management

ndash Reliability Studies

bull Worst Performing Circuits

bull Common Failure Modes

bull Asset Management

ndash Outage Analysis

ndash Outage Prediction and Precursors Analysis

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

The Evolution of Grid ReliabilityFrom

SCADA only in SS

Very few sensors in network

Poor situational awareness and visibility

Overreliance on Vegetation Mgmt RW widening (H) amp animal guarding

Time Based Maintenance and Repair

Selective Undergrounding (H)

Selective Relocations (H)

Separation of Dispacher and Ops

Manual Switching

Trial and Error ndash Reliance only on ldquoExperience and Gutrdquo

Reliance on outage Calls from customers

To

SCADA in SS and Outside the Fence

Increasing sensors in nw

Expanded real time situational awareness

Automated Circuit Reconfiguration

Fault Location and Anticipation (Line Monitoring)

Volt ndash Var Control

Convergence and better teamwork between Dispatcher and Ops

Advanced tools for simulation emulation and replay replacing trial and error - reliance on ldquoExperience and Gutrdquo replaced with actual data and scenario planning in advance during and assessment after the storm

Integration with AMI improves outage detection and notification

(H) - Relocation Redesign

Undergrounding R W Widening etc

are Hardening measures which result

in greater Grid Resilience 73

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

74

Outage Prediction

bull Last-Gasp and Power-Up Messages

ndash More than 65M Last-Gasp and Power-Up messages are received annually

bull 5 associated with actual outages

bull Others are planned work momentary and power quality events

ndash Many are thought to be outage precursors or predictors associated with devices that are starting to fail

ndash Analytics are being developed to use this information to predict outages before they occur

ndash Examples have positively demonstrated to give advance notice

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics for Customer Operations

bull Theft Detection

bull Call Center

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Daily or Interval Reads

Rate Class Benchmarks

Maintenance Behavior Change or Theft A slowing meter

Confidential ndash All rights reserved by DataRaker Inc

Meter Events amp Analytic Flags

Monthly Electric

Billing Read

76

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Review

bull We discussed

ndash Evolution to Big Data

ndash Big Data Players and the Tools

ndash Big Data Value amp Benefit

ndash Big Data and the Enterprise

ndash Data Scientist Role

ndash Distribution Reliability Outage AMI sample

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Questionshellip

78

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Big Data amp Analytics ndashStrategy Roadmap Architecture

79

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Topics

bull Derive Opportunities to Exploit Integrated Data

bull Develop a Analytical Strategy Aligned to Business

bull Develop the Roadmap of Utility Analytics

bull Integrated Architecture to Exploit Utility Data

80

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Smart Utility - Understanding Data

Landscape A Forward Look at Utility Data

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Proposed Business Areas for Analytics Focus

Distribution TransmissionCustomer

ServiceAMI

Corporate

Services

Trouble Management

Vegetation

Work Management

System Expansion

Asset Operations

Cost and Performance

Safety

Reliability and

Restoration

Distribution Automation

System Operations

Load Forecast

Substation

Maintenance

Line Maintenance

Construction Work

Substation Trouble

Management

Asset Management

Reliability Indicator

Management

Care Center

Operations

Billing Operations

Regulatory amp Customer

SAT Management

Customer Field

Services

Revenue Protection

and Assurance

Account Management

Demand Management

Conservation

Meter Operations

AMI Network

Operations

Meter Switch

Operations

Meter Reads and

Billing Integration

Data Integration for

T amp D

IT

Corporate Finance

Marketing

TBD

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics Architecture - A Holistic Stack

Analytics Architecture - A holistic Stack to Exploit data

CORE INFRASTRUCTURE

Distributed

ServersHPC MPP

In-Memory

Databases

Data Movement

Network

Modeling ETL

Tools

High Resolution

Display

Reporting amp

Visualization

Access Tools

(Mobile

Corporate)

PROCESS COMPONENTS

Special Purpose Data Marts B 2 B Data Exchange B 2 C Data Exchange

Real Time Near Real Time Batch Loads Long Cycle Data Loads

Data Stewardship Data Ownership Data Exploitation Business Case

DATA ACQUISITION AND LOAD

DATA MANAGEMENT

DATA ENABLEMENT

Data Model Data Integration Data Quality Data Access Policy Master Data Management

DATA GOVERNANCE

ADVANCED METHODS AND TECHNOLOGIES

Clustering Regression Factor Analysis Time Series Analysis Spatial Modeling Methods

Algorithmic Modeling Genetic Learning Algorithms Machine Learning and Neural Nets

ADVANCED STATISTICAL METHODS

VISUAL ANALYTICS

CLASSIC BUSINESS INTELLIGENCE

Data Mash-ups Situation AwarenessStructured amp Unstructured

Overlays

Multi Dimensional Visual

AnalysisVisual Filtering Methods

ADAPTIVE DATA METHODS

Canned Reports Adhoc Report Data Exploration ToolsBusiness Charts amp Simple

VisualizationOLAP and Data Mining

The Analytics Architecture Must Include Process Methods Technologies Data and Infrastructure

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Key take awayhellip

Think integrated data to exploit business

opportunities

Adopt an incremental value driven analytic

strategy

Target the ldquorightrdquo maturity level for your

enterprise

Implement data and analytic governance

early on

Executive sponsorship is key to success

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Key take awayhellip

Operational efficiency use cases lend

themselves to show case analytics

Contextually evolve to a ldquorightrdquo enterprise

analytic architecture

6

7

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Questionshellip

86

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Big Data amp Analytics ndash The Customer

87

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Topics

bull The Customer Centric Utility ndash Opportunities

bull Customer Segmentation Engagement Marketing - Some Results

bull The ldquoabilityrsquosrdquo as the foundation for success

bull An ldquoIntegrated Approach to Exploit Data amp Gain Insights

bull Closing Thoughts

88

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

The Customer Centric Utility ndash Opportunities for Big Data Analytics amp Visualization

Smart Utility

bull Customer Marketing

Engagement

Segmentation

bull Behavioural

Analytics Social Media

bull The Business Case

bull DSM (EE DR)

bull DER FiT MicroFIT

bull Renewables DER

Smart Consumer

bull Education amp Empowerment

bull Enrollment amp Participation

bull Events amp Rewards Payments

Trading

bull Increase in EV PHEV DER etc

bull The Connected Home

bull The Connected Enterprise

bull Portal

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

What are we hearing from the utilities

Source Utility Dive ndash 2015 State of the Electric Utility

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Lessons Learned from continuum of prosumer engagement

Start with the customer amp their

behaviour segmentation Social

Analytics Smart Meter Data etc

Customer Centric Product Design

Design Smart Rates Tariffs

Market amp Engage amp Recruit the Customers

in Programs

EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for end to end engagement

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Analyze amp understand What is the most cost

effective tariff program Can we determine its

impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V Evaluate Partners amp Validate

Program Value

Single Portal for Customer for Analytics Visualization

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Analytics Visualization opportunities in the continuum of prosumer engagement

Analyze amp understand impact of social media utility data regulatory partner data customer

data weather data market data etc

Cross reference Efficiency Analytics with Operational

Analytics

Analyze amp understand What is the most cost

effective tariff program Can we

determine its impact on the consumers

What is the impact on the utility

What is the impact on markets

Analyze amp understand Which customers

should we select for a program

How will they respond

Based on this data what is impact to utility customer

market etc

Analyitics to evaluate effect on EM amp V

Evaluate Partners amp Validate Program Value

Portal for Utility Customer for Analytics Visualization

Transactional to Continuous feedback ndash Use of Analytics throughout the life cycle

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

bull The ldquoabilityrdquos

bull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Adoptability Acceptability

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

DONrsquoT LEAVE YOUR CUSTOMER BEHIND

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

97

But how to reach every single customer

Computer Science Data Science Behavioral Science

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

98

Big Data Architecture

Meter Reads

300BCalculationsHour

26MUtility Partners

gt95

Computer Science

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

99

Dynamic Optimization

IDENTIFY PEAKERS RELEVANT CONTENT

RESULTS ANALYSIS

Data Science

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

100

Drive Customer ActionBehavioral

Science

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

101

Case Studies ndash 4 Utilities

PEAK DAY

NOTIFICATION

POST-EVENT

FEEDBACK

PERSONALIZED

ADJUSTMENTS

PRODUCTIVITY

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

102

A day in the life of an event

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

103

Results Positive feedback on

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

104

Results Transformative smart grid economics

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Hydro One

1

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Using a combination of our data coupled with purchased data we built a database of customer

intelligence to help us better understand our customers

2011 Hydro One Customer

Satisfaction Surveys

Hydro One Data

Environics Research Social Values Survey (PRIZM Link)

Hydro One Customer File with Program

Participation and Consumption

Personas

Print Measurement Bureau Survey (PRIZM Link)

Sensible Seniors

Heartland Homeowners

Empty-Nest Enthusiasts

Flourishing Families

Segmentation Methodology

3

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Customer Segmentation Demographics

Comparison

Heartland Homeowners

bull Town and Rural Upper Middle-class

Families with Children

bull Middle-aged married couples (79

are families some with children

present some becoming empty-

nests)

bull Average education about half have

high school or less

bull Average house hold income

Empty-nest Enthusiasts

bull Suburban and Rural Middle-class

Families and Seniors

bull Middle-aged and older married

couples and singles (27 non-family

households 13 of families are lone-

parent)

bull Average education about half have

high school or less

bull Slightly below average household

income

Sensible Seniors

bull Retired Seniors

bull Middle-aged and older singles and

married couples without children

(53 over the age of 55)

bull Average education about half have

high school or less

bull Slightly below average household

income

Flourishing Families

bull Suburban Comfortable Families

bull Middle-aged and younger married

couples with children (48 are

between ages 35-54)

bull Relatively high level of education

(30 university vs 28 Ontario

average)

bull Above average household income

4

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Hydro One Program Enrolment by

Segment

000

100

200

300

400

500

Peaksaver

000

050

100

150

Energy Audit

000

100

200

300

IHD

000

050

100

150

200

250

Fridge Pick-Up

1600

1650

1700

1750

Pre-Authorized Payments

000

200

400

600

800

e-Bill

1000

1100

1200

1300

1400

Budget Billing

1300

1350

1400

1450

1500

1550

1600

My Account

000

010

020

030

040

Summer Savings Program

bull Dissatisfied with pricebull Not price sensitivebull Average image

bull Satisfied with pricebull Slightly price sensitivebull Positive image

bull Satisfied with pricebull Not price sensitivebull Negative image

bull Average Satisfaction with pricebull Price sensitivebull Average image (slightly below)

7

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Social Benchmarking Pilot ndash Overview

Purpose

Drive residential customer behavioural energy savings by leveraging billed usage

data (TOU data where possible)

Method

Achieved through the use of residential customer Home Energy Reports mailed to

100000 selected participants

Personalized reports indicate how a participant customerrsquos energy consumption

compares to others like them (ldquoNeighboursrdquo)

tailored suggestions on how they might best save energy

information on province wide energy efficiency Programs available

9

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation

CDM needed better peaksaver PLUS reg results

Hydro Onersquos first tested its segmentation analysis on this program

The result was the development of a more effective marketing campaign for the peaksaver PLUSreg Initiative

110

Bill Insert ndash Before SegmentationBEFORE hellip

ldquoThe Productrdquo

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Peaksaver PLUS Marketing ndash ldquoBeforerdquo and ldquoAfterrdquo

Segmentation (Contrsquod)

111

Segmentation analysis was used to create marketing samples focused on customer values not the product

Offered a layered campaign of segmented bill inserts direct mail and e-blasts

AFTER hellip

ldquoThe Customerrdquo

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Segmentation Success Soaring Results

Customer enrolment more than doubled with segmentation

112

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Predictability

bull Customer Enrollment amp Resource Data Mgmt

bull Customer Baseline Load (CBL) Computation

bull Curtailable Load Modeling ndash Forecasting

bull Temp Dependency Temporal Availability etc

bull Aggregation by Product and Network Location

ndash CIM Model

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

bull Baselines measure load reduction for DRcurtailment programsndash Difference between actual usage and baseline is defined as the reduction amount

ndash Impossible to ldquoknowrdquo what consumption would have been

Baselining

bull Desired characteristicsndash Unbiased ndash does not consistently under or

over estimate reductions

ndash Accurate ndash provides estimates as close as possible to actual reduction

ndash Transparent ndash easily validated and understood by parties

ndash Avoid Moral HazardGaming ndash Not easily manipulated by loads

bull Impactsndash Wholesale market clearingreconciliation

ndash BillingIncentive payments for programs such as peak time rebates

ndash Analysis of effectiveness of DR programs

Reduction amount

114

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Baseline Methods

bull Existing Methods focus on near term historical data

ndash Use from 1 up to 10 previous days

ndash Typically take simple hourly average of selected historical data

ndash May apply adjustments due to weather pre- and post- event load data

bull Issues with Existing Methods

ndash Accuracy

ndash Subject to gaming especially if events can be anticipated

ndash Adjustments may be misleading (eg pre-cooling)

bull Why not Load Forecasts

ndash Statistically calibrated and proven over many years

ndash Incorporate weather and other factors that influence daily load changes

ndash Based on Aggregate loads ndash not adequate for individual loads

115

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Forecasting

bull A good forecast helps utility justify rate case

bull Enables incorporation into Unit commitment and dispatch

bull Components ndash1) Load Forecasting to feedermeter level

ndash Allow for forecasting for dynamic setsgroups of loads

2) Response (capability) Forecastsndash Direct Load Control Operational forecasts and limits

ndash Dynamic Pricing Behavioral considerations

bull High levels of uncertainty due to historical lack of detailed individual information

ndash 15 min AMI data is an enabler

Need to be able to accurately forecast aggregate diverse noisy resources

Forecasting no longer is just ldquoKWHrsrdquowind solar price DR outage

congestionall of it needs to be rolled inhellip

116

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Smart Forecasting ndash Is a classic Big Data Applicationbull Meter level forecasts ndash Usage Forecastingbull Real-time - re-forecasting amp corrections base on real-time conditionsbull Account for Tariffs DR amp DER demographics etcbull Targeted outcomes

ndash Improving reliabilityndash CVRndash Volt-Varndash Network Model level precisionndash Customer Satisifaction

bull Unstructured data sourcesbull Scalabilitybull Interoperabilitybull Monetization ndash the critical output

Platforms of today can accomplish some of it but more needs to be done

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Availability

bull Moves beyond forecasting into measurement of resource capacity at all times

bull Enabled by two way communications

bull Necessary for closed loop controlndash True incorporation of demand into grid operations

ndash Move from passivereactive to active management of generation AND load

Need to be able to accurately monitor and assess DR resource capability in real time

by location

118

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Asset DR Capability and Availability Modeling

kW L

evel

Time

Typical Loading of an Electric Water Heater Heater Element Capacity Average Load

Heater Load

WHOn WHOff

Average Load Profile

On Duration 5 minFunction of

ndash Asset Type

ndash Customer Class

ndash Time and Day Type

ndash Temperature

ndash Other Factors

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Transient Period Operational Period of Interest

Modeling of Load Pickup Characteristics

bull Loss of Load Diversity after DR operation

bull Load After Pickup will be Greater than Baseline Value (CBL)

bull A Function of

ndash DR Duration

ndash DR Asset Type

ndash Other Factors

bull An Aggregated Value

T0 T1 T2 T3

Dev_Capmax

CBL

DR

Source Ali Ipakchi WECC PresentWECC Variable Generation Subcommittee Meeting

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Dependability

bull Programsndash Notification-Based Programs

ndash Dynamic Pricing ndash TOU CPP etc

ndash Critical Peak Pricing with Control

ndash Direct Load Control

ndash Net Metering

ndash Etc

bull Agreementsndash Customer and Utility

ndash Customer and Service Provider

ndash Service Provider and Utility

ndash Etc

bull Work Flowsndash DR Program Execution Process

bull Mapping to Wholesale Supplyndash Forecasting

ndash Scheduling

ndash Pricing

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

DSM Programs ndash EM amp V

bull Demand-Side Management Used Here to Cover the Complete Portfolio of EE Conservation DR and RE Programs

bull EMampV Applications Cover Program Planning BC Analysis MampV Program Performance Lost Revenue Recovery Regulatory Reporting amp Internal Management Reporting

123

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Issues Faced

bull Lack of Timely and Consistent Data

bull Data Collection Time Consuming and Frustrating

bull Program Tracking is Hit or Miss

bull Hard to Generate Repeatable Reports with Clear Audit Trail

bull Long Time Lag for Verifiable Results

124

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Factors Affecting System Selection

bull How Well can It Manage Data Capture Analysis amp Reporting

bull Is It Modular amp Customizable to Meet Program-Specific Requirements

bull Does it Provide Out-of-the-Box Support for Wide Variety of Programs

bull Does it Force Too Many Compromises on the User

125

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

System Selection Factors (Contrsquod)

bull Can it Function As ldquoData of Recordrdquo

bull Does it Provide Role-Based Access to Data Approvals amp Reports

bull Does it Need Major IT Equipment andor StaffInvestment by they Utility

bull Secure Reliable and Available 247

bull Scale to Match Future Growth

bull How Well Can It Integrate with Other Systems

126

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Utility DSM Program Data

bull Transactions [Customer Premises Measures Rebates]

bull Financial [Budgets Program Expenses]

bull Analytical [NTG Current and Future kW Savings MampV Performance]

bull Reports [Portfolio to Program Performance Dashboards Management Tracking Regulatory]

bull Implementation Contractor Submitted

bull Others MIS Data Weather etc

127

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Heterogeneous Business Processes Data Storage DataData Sources Presentation

Extract

Validate

Clean

and

Delivery Data

Processes

Reporting

Engine

Web Portals

Dashboards

Excel Export

Custom Reports

Validated

Data

Metrics

PM TrackingTools

ContractorReports

External Data

Other Flat Files

System Overview

128How does this transition and how much of this should transition to Big Data

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Benefits of Integrated System

bull Unify Data from Multiple Sources in Consistent Form and Format

bull Free Up PMs to Focus on Planning Design amp Management

bull Rapid Program Launch amp Ramp-Upbull Improve Program Marketing and Performancebull Flexible and Customizable Report Generationbull Track Changes To Program Data And Assured

Repeatability Of Past Reportsbull Consistent Data Source For EMampV Activitiesbull QAQC Checks on Databull Audit Trail for Regulatory Compliance

129

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Case Study Example

bull Before Systemndash Reports Filed 1 Year or Later After

End of Program Year

ndash Repeat Interactions with Program Staff Implementation Contractor to Complete MampV

ndash Unable to Guarantee Reports Can be Repeated

ndash Underlying Data Could Not be Traced Reliably

ndash Could Not Provide Timely Answers to Management on Program Performance

130

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Case Study Example (Contrsquod)

bull After Integrated System Installedndash Real-Time Reporting of Complete Program

Performance with Multiple Metric Analytics

ndash Reports are Repeatable and Auditable

ndash Consistent MampV Data Available Across the Full Portfolio of Programs Covering Multiple States

ndash Support Add-On Modules for Custom MampV Lost Revenue Recovery etc

ndash Provide Valuable Insights for Program Marketing And Performance Evaluation

131

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Dispatchability

bull Linking Demand Side Capabilities to Supply Side Wholesale Capabilities amp Markets

bull Voltage Based Demand Management

ndash CVR ndash Dispatchable but limited with constraints

bull Combination of DR amp DER Storage along with some real-time capabilities is a good option

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Demand Response Impacts (single home)

Average Critical Peak Pricing

(CPP) response from

California pilot (2004)

CPP wPCT ~ 50 peak

reduction (~15 KW per home)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

CPP wPCT

CPP no PCT

Control (no CPP)

PCT = Programmable Communicating Thermostat

Dispatch

Ramp

Coordination

Issues

bull Rebound (post event increases)

bull Coordination - grouping of customers (Network Model CIS etc)

bull Ramp Rate control

Need equivalent responsiveness and precision as generation ndash Not

every DSM load is Dispatchable 133

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Scalability

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Interoperability

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

A sample case study for Energy Efficiency Marketing amp Analytics Platform

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

bull 360 degree view of consumers for marketing planning and analyticsbull Discover and mine relationshipsbull Create highly targeted and individualized marketing programs

The Vision

Enterprise Marketing and Analytics Platform

The How

bull Co-location master data management custom data quality and cleansing rules and more

bull Which allow integration of data from across and outside of the company to create 360 view of consumers

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Summary of Key Benefits

Provides a single platform to house key customer and prospect data sources

Establishes persistent keys across previously disparate data sources

Provides for rapid intake of new data sources (structured and unstructured)

Eliminates todayrsquos data intake and append bottleneck

Empowers Analysts to explore all data elements

Increases processing power for statistical analysis and

Improves recruiting and retention of Data Engineers and Data Scientists

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Sample Solution Architecture

Pow

er C

ente

r B

ig D

ata

Edit

ion

HDFS

Dat

a Q

ual

ity

Big

Dat

a Ed

itio

n

Iden

tity

Res

olu

tio

n

HB

ase

Hive

Map Reduce

Cleansed Files

Individual Household

Informatica Big Data Edition Cloudera Big Data Platform

Vis

ual

izat

ion

s

Big Data Analytics

Extract Load amp Transform

Data Quality ndashCleaning Identity Resolution

CustomerData (CIS+)

SCADA OMSDMS

MDM

GISInp

uts

CRM

AM

External Data (Social Media

Weather etc

Pre

dic

tive

An

alyt

ics

Visualizations

Consumption

Dat

ame

er

Predictive Analytics

Source Global Big Data Conference

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Profitability

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Utility Perspectives

Utility Dive ndash 2015 State of the Electric Utility

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Primary Targets

Hard to Reach

Low Cost

Low Value

Value Determination

High Output

Low Output

Low

Acquisition

Cost

High

Acquisition

Cost

142

Source OGE ndash MF

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

bull The ldquoabilityrdquosbull Adaptability Acceptability

bull Predictability

bull Dependability

bull Dispatchability

bull Scalability

bull Interoperability

bull Profitability

The secret to success lies in these ldquoabilitiesrdquo

All of these have one key link ndash Analytics Big Data amp Visualization

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

DER Analytics

Future and Closing Comments

145

BACKUP SLIDES

146

Future and Closing Comments

145

BACKUP SLIDES

146

BACKUP SLIDES

146