webinar slides: using machine learning and data analysis ... · pdf file§ residenal...

Webinar:UsingMachineLearningand DataAnalysistoImproveCustomer AcquisiEonandMarkeEngin ResidenEalSolar

July20th, 2017

LogisQcs

• ParQcipantsarejoinedinlisten-onlymode. • UsetheQ&ApaneltoaskquesQonsduringthewebinar.We

willhaveQmeforQ&Aattheveryend. Ø ToaskaquesQon:

o ClickQ&Aboxinthegotowebinartoolbar o TypeyourquesQonintheQ&Abox

• ForanytechnicaldifficulQesduringthewebinar, contactthe GoToWebinars HelpDeskat888.259.3826forassistance.

NATIONAL RENEWABLE ENERGY LABORATORY 2

NATIONAL RENEWABLE ENERGY LABORATORY 3

• KiranLakkaraju–SandiaNa0onalLaboratories• Yevgeniy(Eugene)Vorobeychik–VanderbiltUniversity• MichaelRossol–Na0onalRenewableEnergyLaboratory

Presenters

Photos placed in horizontal position with even amount of white space

between photos and header

Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Promo?ngSolarTechnologyDiffusionThroughData-DrivenBehaviorModeling

Co-PI:Dr.KiranLakkaraju,SandiaNa?onalLaboratoriesCo-PI:Prof.EugeneVorobeychik,VanderbiltUniversity

Team:UniversityofPennsylvania,CaliforniaCenterforSustainableEnergy,Na?onalRenewableEnergyLaboratory,VoteSolar

adoption probability

observability

salience

frame of reference

information/framing

income/ economic outlook

adoption by friends/

neighbors

attitudes of friends/

neighbors

attitudes/beliefs

messaging, framing

immediate net costs

long-term net value

economic uncertainty

online “build your PV” tool

subsidies

gifts

Economic Variables

Psychological Variables

Social Variables Computational Predictive Model

informational website

SAND2014-18601 PE

Residen0alPhotovoltaicsandSocCosts§ DOESunShotGoal:Reducecostofinstalledsolarenergy

systemsto$.06/kWhby20203.

§ Residen0alenergyuseis21%ofconsump0onintheUS(asof2013)1.

§ PVpricesgoingdownbut67%oftotalresiden0alsystempricegoingto“soccosts”2:§ Customeracquisi?on~9.2%§ Installa0onlabor~10.5%§ Supplychaincosts~11.7%

§ Ifwecanreducecustomeracquisi0oncosts,wereducecostsofPV.

51. http://www.eia.gov/beta/MER/?tbl=T02.01#/?f=A&start=1949&end=2013&charted=3-6-9-12

2. Friedman, Ardani, Feldman, Citron, Margoli, Zuboy, “Benchmarking Non-Hardware Balance-of-System (Soft) Costs for U.S. Photovoltaic Systems, Using a Bottom-Up Approach and Installer Survey – Second Edition”

3. http://energy.gov/eere/sunshot/mission

OverarchingResearchQues0ons

6

How can we determine residential solar PV adoption tendencies and trends, at the individual and

aggregate level?

GOAL: Develop a model to test and identify incentive policy structures that increase adoption

7

Political Ideology

Social Influence

Climate Change

Attitudes

Behavioral Economics

Incentives

Financing Structure

Retirement

Solar Community Orgs

Group Buy Environmental Ideology

Adoption Data

Taxes

Solar Irradiance

Module type

Inverters

Factors/Variables

adoption probability

observability

salience

frame of reference

information/framing

income/ economic outlook

adoption by friends/

neighbors

attitudes of friends/

neighbors

attitudes/beliefs

messaging, framing

immediate net costs

long-term net value

economic uncertainty

online “build your PV” tool

subsidies

gifts

Economic Variables

Psychological Variables

Social Variables Computational Predictive Model

informational website

Predictive Model

Experimental Data

OverarchingResearchQues0ons

8

Agent-based model (ABM) development through data-driven machine learning

Approach:

Data gathering Predictive Model ABM Modeling

0.00

0.01

0.02

100 200 300 400n

dens

ity

policyCSI Designbest 1x Bgtbest 2x Bgtbest 4x Bgtbest 8x Bgt

Policy Testing

How can we determine residential solar PV adoption tendencies and trends, at the individual and

aggregate level?

GOAL: Develop a model to test and identify incentive policy structures that increase adoption

9

Prof. Howard Kunreuther Ben Sigrin

Dr. Kiran Lakkaraju (PI) Prof. Eugene Vorobeychik (Co-PI)

Lead Organization

Tim Treadwell

Georgina Arreola

Prof. Arthur van Benthem

Prof. Ruben Lobel

Dr. Dena Gromet

Jesse Denver

Kevin Armstrong

Predic0veModel

10

Predictive Model (prob. of adoption)

• Predict probability of anindividual adopting at aparticular time.

• Identify important features.

AgentBasedModel(ABM)

11

ABM simulates emergent behavior (adoption)

Predictive Model

Interven0ons

12

+ Incentive Structure

+ Incentive Structure

ABM allows analysis of policy interventions

Whathavewelearnedfromthedata?

§ Re0rementspursthoughtsaboutadop0on(Survey).§ Monthlybillsavingsisanimportantfactor(Survey).§ Liberalsrespondmorefavorablytoa“reduce”electricity

usagemessage;conserva0vemorefavorablyto“increase”savingsmessage(Experiments).

§ Step-wiseincen0vestructures(liketheCaliforniaSolar

Ini0a0vestructure)donotincreaseadop0onwithincreasedbudgets.MoreusefultogiveawayfreesysteminourABMmodel(ABMsimula0ons).

13

Y E V G E N I Y ( E U G E N E ) V O R O B E Y C H I K

ALGORITHMIC MARKETING

Assistant Professor, EECS Director, Computational Economics Research Laboratory

Vanderbilt University

PROMOTION OF INNOVATION

• Four important questions:• Causality: which variables causally influence adoption of

innovation• Forecasting: how can we accurately forecast adoption

trends, both individual and aggregate• Impact: what is the impact of changes in environment/

policy variables on attitudes, adoption decisions, and trends• Optimization: how to we choose policy to optimally

promote innovation (subject to cost/budget constraints)

PROMOTION OF INNOVATION

• Econometrics, social science research• Causality: which variables causally influence adoption of

innovation• Forecasting: how can we accurately forecast adoption

trends, both individual and aggregate• Impact: what is the impact of changes in environment/

policy variables on attitudes, adoption decisions, and trends• Optimization: how to we choose policy to optimally

promote innovation (subject to cost/budget constraints)

PROMOTION OF INNOVATION

• Computer Science• Causality: which variables causally influence adoption of

innovation• Forecasting: how can we accurately forecast adoption

trends, both individual and aggregate• Impact: what is the impact of changes in environment/

policy variables on attitudes, adoption decisions, and trends• Optimization: how to we choose policy to optimally

promote innovation (subject to cost/budget constraints)

PROMOTION OF INNOVATION

•  Computer Science •  Causality: which variables causally influence adoption of

innovation •  Forecasting: how can we accurately forecast adoption

trends, both individual and aggregate •  Optimize model accuracy in predicting unseen data •  Crucial: embed known causal models and controls •  Constrain models to make economic sense

•  Impact: what is the impact of changes in environment/policy variables on attitudes, adoption decisions, and trends

•  Optimization: how to we choose policy to optimally promote innovation (subject to cost/budget constraints)

PROMOTION OF INNOVATION

•  Computer Science •  Causality: which variables causally influence adoption of

innovation •  Forecasting: how can we accurately forecast adoption

trends, both individual and aggregate •  Impact: what is the impact of changes in environment/

policy variables on attitudes, adoption decisions, and trends •  Optimization: how to we choose policy to optimally

promote innovation (subject to cost/budget constraints) •  Assuming that causality of policy variables is correctly captured

ALGORITHMIC PERSPECTIVE ON PROMOTION OF INNOVATION

adoption model

ALGORITHMIC PERSPECTIVE ON PROMOTION OF INNOVATION

Aggregate adoption forecast

adoption model

ALGORITHMIC PERSPECTIVE ON PROMOTION OF INNOVATION

Aggregate adoption forecast

promotion strategies

…

policy variables

adoption model

Exogenous (environment)

variables

ALGORITHMIC PERSPECTIVE ON PROMOTION OF INNOVATION

Aggregate adoption forecast

promotion strategies

…

policy variables

adoption model

Exogenous (environment)

variables

Budget constraint

ALGORITHMIC PROBLEM: FORECASTING

•  Problem: •  Use prior individual-level adoption data to

develop a model for forecasting individual decisions, and aggregate adoption trends

•  Technical challenges: scalable model calibration, model validation at multiple scales

•  Approach: •  Data-driven agent-based modeling (DDABM) •  Scalable calibration: using machine learning

(ensuring features are causally grounded) •  Validation: cross-validation (measure

effectiveness of individual behavior); forecasting validation (split data along time dimension into calibration and validation sets)

adoption model

DDABM1

•  Individual agent behavior: probability that individual (household) adopts solar PV, as a function of a collection of variables (including policy variables) •  Learn from a large individual-event dataset of past

adoption decisions •  Include social influence: capture interactions among

agents; leverage previous causal modeling method (Bollinger & Gillingham, 2012)

•  Agent-based model: learned agent models instantiated as households, given household/demographic parameters from data

1 H Zhang, Y Vorobeychik, J Letchford, K Lakkaraju. Data-driven agent-based modeling, with application to rooftop solar adoption. International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2015.

DATA

•  Adopters: adoption (incentive reservation) time, installation time, lease vs. own, system size, assessor data (property characteristic: square footage, etc), energy use data for 12 months prior to adoption (obtained from Center for Sustainable Energy)

•  Owners: cost of ownership •  Leasers: cost of leases (based on a subsample of

actual lease contracts) •  Non-adopters: assessor data •  Climate data (solar insolation/geographic location)

DDABM1

1 H Zhang, Y Vorobeychik, J Letchford, K Lakkaraju. Data-driven agent-based modeling, with application to rooftop solar adoption. International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2015.

Model captures actual adoption patterns

calibration validation

Incentives have weak impact on adoption trends

fore

ca

st

ALGORITHMIC PROBLEMS: PROMOTION

Aggregate adoption forecast

promotion strategies

…

policy variables

adoption model

Exogenous (environment)

variables

Budget constraint

OPTIMIZING INCENTIVES2

•  Problem: given a budget and a time horizon, how to optimally choose incentive policy •  Amount of incentive / watt •  MW transition points to lower incentives

Rate($/watt)

Target (megawatt)

r*

1

m*

1

B

mi

2

r*

2

mi

3

ri

2

0.00

0.01

0.02

150 200n

dens

ity policyCSI Designbest 1x Bgtbest 2x Bgtbest 4x Bgt

2 H Zhang, Y Vorobeychik, J Letchford, K Lakkaraju. Data-driven agent-based modeling: framework and application. Journal of Autonomous Agents and Multiagent Systems, 2016.

DOOR-TO-DOOR MARKETING4

•  Door-to-door marketing has been shown effective, but is also quite expensive

•  How can we effectively decide where to focus marketing efforts? •  Target individuals who are likely to

adopt, and who are in the best position to influence others to adopt

•  Traveling to individual households is time consuming; wish to do this efficiently

4 H Zhang, Y Vorobeychik. Dynamic influence maximization under increasing returns to scale. AAAI Conference on Artificial Intelligence (AAAI), 2016.

DOOR-TO-DOOR MARKETING MODEL

31

•  Walk door-to-door along a road network

•  Goal: maximize # of adopters, subject to the constraint that your walk takes at most H hours

social influence

network

road network

4 H Zhang, Y Vorobeychik. Dynamic influence maximization under increasing returns to scale. AAAI Conference on Artificial Intelligence (AAAI), 2016.

ALGORITHMS FOR DOOR-TO-DOOR MARKETING

•  Developed a new greedy algorithm (GCB), adding households to visit based on marginal impact on influence per unit marginal cost of visiting

•  Proved that it’s near-optimal and much faster than state-of-the-art alternatives

32

0 2 4 6 8 10

020

4060

80100

Budget

Influence

GCBGRISK

0 2 4 6 8 10

010

2030

4050

Budget

Run

ning

Tim

e (s

)

GCBGRISK

4 H Zhang, Y Vorobeychik. Dynamic influence maximization under increasing returns to scale. AAAI Conference on Artificial Intelligence (AAAI), 2016.

ALGORITHMIC PROBLEM: DIVIDE BUDGET AMONG MANY MARKETING ACTIVITIES

Aggregate adoption forecast

promotion strategies

…

policy variables

adoption model

Exogenous (environment)

variables

Budget constraint

ALGORITHMIC PROBLEM: DIVIDE BUDGET AMONG MANY MARKETING ACTIVITIES

•  Problem: have a collection of marketing activities, with varying costs and impact on adoption

•  Approach: can formulate as an approximate multi-Knapsack problem •  Developed new algorithms for efficient query strategies

when channel response is represented using simulations (e.g., ABM) Our approach

BOTTOM LINE

• Many algorithmic problems arise in the context ofinnovation diffusion• Modeling and big-data analytics of adoption decisions

(human behavior modeling)• Computational (often, combinatorial optimization) methods

for optimizing marketing decisions

• We made progress addressing some of these in thecontext of solar PV adoption, using individual-levelCSI data for San Diego county• Uncovered and addressed some fundamental

computational problems in the process

WillingnesstoAdoptSolar:Aninves?ga?onintoAddesignMichaelRossol1KimberlyWolske2,AnnikaTodd3

JamesMcCall1,andBenSigrin1July20th,20171Na0onalRenewableEnergyLaboratory2HarrisPublicPolicy,UniversityofChicago3LawrenceBerkeleyNa0onalLab

NATIONAL RENEWABLE ENERGY LABORATORY 37

Customeracquisi0onisasubstan0alpor0onofthecostofPV

Fuetal.2016U.S.SolarPhotovoltaicSystemCostBenchmark

$2.93/W

Moezzietal.In-preparaBonANon-ModelingExploraBonofResidenBalSolarPhotovoltaic(PV)AdopBonandNon-AdopBon

EnvironmentPrimarily:3-5%

Environment+Money:33%

Both,butMoneyoverEnvironment:39%

MoneyandnotEnvironment:6%

OpportunisAc:~20%

NATIONAL RENEWABLE ENERGY LABORATORY 38

H1:Consumerswillrespondmorefavorablytolossframesthangainframes

Ø  Theliteraturesuggeststhatlossesloomlargerthanpoten0alequivalentgains,solossframingshouldbemoremo0va0ng

H2:AdsthatpresentthebenefitsofPVintheneartermwillbemoreeffec0vethanadsthatframethebenefitsinthelong-term

Ø  Theliteraturesuggeststhatpeoplediscountthefutureandthusneartermfinancialbenefitsmaybemoreeffec0ve

H3:AdsthatpresentthebenefitsofPVintermsofnetsavings(i.e.savingsnetofthesystempayment)willbemoreeffec0vethanthoseintermsofgrosssavings(i.e.doesn’tincludesystempayment)

Ø  Priorworkhassuggestedcustomersassumethesavingsaregross,whichunderstatesthepoten0albenefits

H4:AdsthatpresenthighermonetarysavingsfromPVwillbemoreeffec0vethanthosewithlowermonetarysavings.

Ø  Assessesmarketpoten0alasafunc0onofthebillsavingsamount

ResearchQues0ons

NATIONAL RENEWABLE ENERGY LABORATORY 39

•  Surveyso  Adso  Measuresusedo  Demographics

•  Resultso  Gain/LossbyTimeFrameo  Net/GrossbySavingAmounto  RoleofAdSkep0cismo  Roleofpre-exis0ngpsycho-socialconstructs

•  Conclusions

Outline

NATIONAL RENEWABLE ENERGY LABORATORY 40

Survey1:Gain/LossbyTimeFrame

GainFaming LossFraming

SavingsarebasedonaMay2016assessmentoflikelysolarsavingsforresiden0alcustomerinCAorNYwitha5kWsystem.

CA NY

$67eachmonth $28eachmonth

$804eachyear $336eachyear

$20,100overthenext25years $8,400overthenext25years

NATIONAL RENEWABLE ENERGY LABORATORY 41

Survey2:Net/GrossbySavingAmount

Grosssavings:Ambiguous Netsavings:Unambiguous

Amountofsavings:-  Low=$300peryear-  Mid=$625peryear-  High=$925peryear

MidsavingsarebasedonaMay2017assessmentoflikelysolarsavingsforresiden0alcustomerinMAwitha5kWsystem.

NATIONAL RENEWABLE ENERGY LABORATORY 42

•  DependentVariables:o  Likelihoodtorespondtotheado  BasedontheadhowappealingisPV

–  AppealofSolarPanels–  CostbenefitofPV–  FinancialImpactofPV

o  Howskep0calareyouofthebenefitsofPV•  Covariates:

o  Psycho-social–  EnvironmentalNorms–  PerceivedSocialSupport–  CustomerInnova0venessCNS,CUM–  Homein-suitability

o  Socioeconomics–  Age–  Gender–  Educa0on–  Householdsize–  HomeSquareFootage–  SESRung(Income)

SurveyMeasures

NATIONAL RENEWABLE ENERGY LABORATORY 43

Gain/LossandTimeFramingdidnotsignificantlyinfluencetheappealofPVorlikelihoodofrespondingtothead

Study1–Gain/LossbyTimeFrame

LossLoss

GainGain

LikelihoodtoRespondtoAd AppealofPVacerseeingtheAd

NATIONAL RENEWABLE ENERGY LABORATORY 44

HigheramountsframedasnetsavingsincreasedtheappealofPV,butdidnotincreaselikelihoodtorespondtothead

Net/GrossXAmountp=0.02

Study2–Net/GrossbySavingAmount

Net

Gross

Net

Gross

LikelihoodtoRespondtoAd AppealofPVacerseeingtheAd

NATIONAL RENEWABLE ENERGY LABORATORY 45

SurprisinglyNetadsindicategreaterskep0cism

Net/Grossp=0.018

Net

Gross

Loss

Gain

Gain/LossbyTimeFrame Net/GrossbySavingAmount

NATIONAL RENEWABLE ENERGY LABORATORY 46

Results:Psycho-SocialVariablesstronglyinfluenceWTA

LikelihoodtoRespondtoAd Skep0cismabouttheAd

NATIONAL RENEWABLE ENERGY LABORATORY 47

H1:Consumerswillrespondmorefavorablytolossframesthangainframes

Ø  RespondentsrespondedsimilarlytobothgainandlossframesH2:AdsthatpresentthebenefitsofPVintheneartermwillbemoreeffec0vethanadsthatframethebenefitsinthelong-term

Ø  Respondentsrespondedsimilarlyregardlessof0meframeforsavings/losses

H3:AdsthatpresentthebenefitsofPVintermsofnetsavings(i.e.savingsnetofthesystempayment)willbemoreeffec0vethanthoseintermsofgrosssavings(i.e.doesn’tincludesystempayment)

Ø  NetadsresultedingreaterappealforPVbutdidnotleadtoagreaterlikelihoodtorespondtothead

H4:AdsthatpresenthighermonetarysavingsfromPVwillbemoreeffec0vethanthosewithlowermonetarysavings.

Ø  Respondentsrespondedsimilarlyregardlessofadamount

Hypothesis

NATIONAL RENEWABLE ENERGY LABORATORY 48

•  EffortstomakethefinancialbenefitsofPVseemmoreappealingdonotappeartostronglyinfluenceinterestinPV:o  Timeframeforsavingshasliwleeffecto  Gainandlossframingareequallyeffec0veo  Greatersavingsdoesnotincreaselikelihoodtorespondo  Providinggreaterfinancialdetailleadstogreaterskep0cism

•  Insteadpre-exis0ngpsycho-socialconstructsappeartodriveinterestinPVo  Beingpro-environmento  HavingsocialsupporttoadoptPVo  PerceivingyourhometobewellsuitedforPVo  Beinginnova0ve

•  Targetedmarke0nghasthepoten0alforgreatestreturno  Findtheproperaudienceo  Targetadstosaidaudience

Conclusions

NATIONAL RENEWABLE ENERGY LABORATORY 49

Ques0ons?

NATIONAL RENEWABLE ENERGY LABORATORY 50

Ques0ons?