Weekly OpenADE Meeting Notes

Tuesday, February 25, 2014

OpenADE Task Force Topics• Issues Raised and Implementation Questions

– How to use BR=bulkID with application to account and account groupings, as well as, large ThirdParty collections of Authorizations.

– Service Request 83 – including Function Block for optional customer info (service point address, etc.)

– Service Request 84 – having scope selection screen on Data Custodian Site vs 3rd Party site– Service Request 85 – Duplicating TOU and CPP from ReadingType to IntervalReading as in

SEP 2.0– Service Request 86 – Desire to add digital signature to Green Button data to protect

against tamper.• New Resources for OpenADE Exchange requested

– Tariff Model Resource– Customer Information Resource

• Green Button Connect My Data Testing and Certification– Complete function block descriptions– Complete test case requirements

For February 25

• John Teeter raises issue of path vs opaque URIs for REST services for individual and subscription resources– Does the uri give any indication of what will be

retrieved or not?

Some URIs Found In GBDMD Files• URI ::= protocol://hostname:port/datacustodian/espi/1_1/resource/ resource endpoint of the server

• <link rel="self" href="User/9b6c7063/UsagePoint/01"/>• <NS:link rel="self" href="/User/9b6c7063/UsagePoint/01"/>• <atom:link href="User/9b6c7063/UsagePoint/01" rel="self"/>• <link rel="self" href="User/25cd2af5c5f6f693f8f6d62852033843/UsagePoint/01" />• <link rel="self" href="RetailCustomer/10/UsagePoint/01"/>• <link href="RetailCustomer/115973279529374200002937445377/UsagePoint/0" rel="self"/>• <link rel="self" href="RetailCustomer/765786587/UsagePoint/01" />• <link rel="self" href="/User/9b6c7063/UsagePoint/01"/>• <atom:link href="User/9b6c7063/UsagePoint/01" rel="self"/>• <link rel="self" href="/User/9b6c7064/UsagePoint/01"/>• <atom:link href="/User/9b6c7063/UsagePoint/01" rel="self"/>• <link rel="self" href="/RetailCustomer/1/UsagePoint/J2753386"/>• <link href="/v1/User/455/UsagePoint/580" rel="self"></link>• <link rel="self" href="User/9b6c7063/UsagePoint/01"/>• <link rel="self" href="User/9b6c7063/UsagePoint/01"/>• <link rel="self" href="User/9e610ca8441264b3d21cad5b2a13d028/UsagePoint/01" />• <link href="/v1/User/12704625/UsagePoint/4218907" rel="self"></link>• <link href="/User/4685/UsagePoint/67" rel="self"/>• <link rel="self" href="User/00e308198d020442995dea12c013f77a/UsagePoint/01" />• <link rel="self" href="/User/1564408+15644/UsagePoint/0"/>

Opaque URIs• Opaque URIs

– No need to test structure– No need to recognize structure in sw

• Structured URIs– Easier to recognize the links– Easier to validate what you are doing by looking at them– If I have interval block, I know all the possible URIs for that UsagePoint

• Possible Outcomes of OpenADE Discussion?– No structure, support opaqueness– Optional Structure, make structured URIs an optional Function Block– Required Structure, make structured URIs a requirement but allow some

variability – e.g. User versus RetailCustomer– Single Required Structure – defined structure based roughly on

GreenButtonAtomLinks and Authorization documents

Older or other slides

Will build deck with new content over time.

SFTP for Bulk Transfer• Pertinent to the SFTP discussion are the concepts that each Third Party has a defined relationship with the Data Custodian.

– For automated exchange of information about his relationship there is a special Authorization obtained in Use Case #1 (see the Authorization.docx -- http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/referenceMaterial/GreenButtonAuthorization.docx).

– We anticipate that when the Data Custodian has data available, it sends an asynchronous Notification to the Third Party. – This Notification provides URIs of note that it is assumed the Third Party will want to retrieve.

• For the purposes of Bulk transfer, this URI will be:– sftp://hostname:port/DataCustodian/espi/1_1/resource/Batch/Bulk/{bulkId} – where {bulkId} is a unique identifier assigned by the Data Custodian and the balance of the URI is presented in the

ApplicationInformation resource that both parties share (contains all relevant URIs and data for interchange via OAuth etc…). The Third Party would then retrieve the bulk data by using an SFTP client with that URI. This is a straw man concept for discussion on the call. Its advantage is that it in harmony with overall architecture of the Green Button Connect My Data RESTful architecture and simply adds SFTP as a means of transfer when a large data set is to be returned.

• Used to Retrieve the data using SFTP protocols– How to initiate the SSH connection?– What is the role if any of the client_credentials authorization to control access to SFTP enabled resources?

• Discussion – – After authorization of TP, they use Pene test, so what is benefit of access-token?– sftp user:pw, user=<tpname>, password=<tp client-credentials access-token>

• Summary– sftp://hostname:port/DataCustodian/espi/1_1/resource/Batch/Bulk/{bulkId} – sftp user:pw, user=<tpname>, password=<tp client-credentials access-token>

Function Blocks for CMDFunctionBlocks for Green Button Connect My Data Description[FB_3] Core Green Button Connect My Data Core Services[FB_13] Security and Privacy classes HTTPS support[FB_14] Authorization and Authentication (OAuth) Oauth[FB_19] Partial update data IntervalBlocks without full data sets (Ups,MR, …)[FB_31] Core Rest Services Third Party Access to Subscription/Authorization

[FB_32] Resource Level RESTThird Party Access to UsagePoints, MeterReading, … and collections

[FB_33] Management REST Interfaces GET PUT POST DELETE individual resources …

[FB_34] SFTP for BulkOptionally support the SFTP delivery of Bulk for Bulk request

[FB_35] REST for Bulk Support the REST request for Bulk[FB_36] Third Party (Client) Dynamic Registration Use Case 1[FB_37] Query Parameters[FB_38] On Demand Requests Without Notification[FB_39] PUSH model Notification followed by GET[FB_40] Offline Authorization to Complement OAuth[FB_42] Third Party Core REST Services[FB_43] Third Party Management REST Services[FB_xx] Not a Function Block (Implementation Specific) Implementation Specific RESTful API

• Authorization Sequence– Scope– access-token– Refresh-token– resourceUri (the subscription)– authorizationUri – expiration of the access-token and refresh-token– token-type

Proposed CMD Function Blocks

FunctionBlocks for Green Button Connect My Data Description[FB_3] Core Green Button Connect My Data Core Services[FB_13] Security and Privacy classes HTTPS support[FB_14] Authorization and Authentication (OAuth) Oauth[FB_19] Partial update data IntervalBlocks without full data sets (Ups,MR, …)[FB_31] Core Rest Services Third Party Access to Subscription/Authorization[FB_32] Resource Level REST Third Party Access to UsagePoints, MeterReading, … and collections[FB_33] Management REST Interfaces GET PUT POST DELETE individual resources …[FB_34] SFTP for Bulk Optionally support the SFTP delivery of Bulk for Bulk request[FB_35] REST for Bulk Support the REST request for Bulk[FB_36] Third Party (Client) Dynamic Registration Use Case 1[FB_37] Query Parameters[FB_38] On Demand Requests Without Notification[FB_39] PUSH model Notification followed by GET[FB_40] Offline Authorization to Complement OAuthNEED to Discuss[FB_42] Third Party Core REST Services[FB_43] Third Party Management REST Services[FB_xx] Not a Function Block (Implementation Specific) Implementation Specific RESTful API

Draft of API Allocations to FBsFunction Blocks CRUD API URL

[FB_3] Core Green Button Connect My Data GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ReadServiceStatus

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ApplicationInformation/{ApplicationInformationID}

[FB_31] Core Rest Services PUT https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ApplicationInformation/{ApplicationInformationID}

[FB_31] Core Rest Services DELETE https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ApplicationInformation/{ApplicationInformationID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Authorization/{AuthorizationID}[FB_31] Core Rest Services PUT https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Authorization/{AuthorizationID}[FB_31] Core Rest Services DELETE https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Authorization/{AuthorizationID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Batch/Subscription/{SubscriptionID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Batch/RetailCustomer/{retailCustomerID}/UsagePoint[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Batch/RetailCustomer/{RetailCustomerId}/UsagePoint/{UsagePointId}

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerQualitySummary

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerQualitySummary/{ElectricPowerQualitySummaryID}

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerUsageSumary

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/ElectricPowerUsageSumary/{ElectricPowerUsageSummaryID}

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadingID}/IntervalBlock

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadingID}/IntervalBlock/{IntervalBlockID}

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/LocalTimeParameter

[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/LocalTimeParameter/{LocalTimeParameterID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/MeterReading[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/MeterReading/{MeterReadingID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}/MeterReading/{MeterReadin

gID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ReadingType[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/ReadingType/{ReadingTypeID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/Subscription/{SubscriptionID}[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint[FB_31] Core Rest Services GET https://services.greenbuttondata.org/DataCustodian/espi/1_1/resource/RetailCustomer/{RetailCustomerID}/UsagePoint/{UsagePointID}

ScopeTerm ExpansionScope [ FBTerms ], [ ValueTerms ], [ ResourceTerms ];FBTerms “FB=“, { [FBTerm], ”_”} , FBTerm, ScopeDelimiter ;FBTerm “4” | “5” | “6” | “7” | “8” | “9” | “10” | “11” | “12” | “15” | “16” | “17” | “18” | “19” | “27” | “28” | “29”ValueTerms { ( "IntervalDuration=", nonNegativeNumber | namedFrequency),

| ( "BlockDuration=", nonNegativeNumber | namedFrequency), | ( "HistoryLength=", nonNegativeNumber),| ( "SubscriptionFrequency=", nonNegativeNumber | namedFrequency), ScopeDelimiter };

ResourceTerms

{ (“ApplicationInformation,” | “Authorization,” | “UsagePoint,” | “IntervalBlock,” | “MeterReading,” | “ElectricPowerQualitySummary,” | “ElectricPowerUsageSummary,” | “ReadingType,” | “Subscription,” | “LocalTimeParameters,” | (“BulkAccountCollection=”, nonNegativeNumber) | “BR=”, brID), ScopeDelimiter}

ScopeDelimiter “;”namedFrequency “billingPeriod” | “daily” | “monthly” | “seasonal” | “weekly” | nonNegativeNumber digit, { digit };digit 0 | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;Where:

ResourceTerms

The ESPI resource – default is “Subscription”. If a Bulk resource is specified via the “BR” term, the value of the {bulkID} is provided after the equals sign (“=”). There could be one or more terms in this list that express the granularity of notifications about resource changes.

FBTerms The function blocks supported (only data content FBs are listed)ValueTerms These are parameterized termsIntervalDuration This is the minimum default length of an interval in seconds (e.g. 900 for 15 minutes, 3600 for one hour, …)

BlockDurationThis is the length of a block that contains the intervals (based on enumeration of MacroPeriodKind in ESPI above as namedFrequency)

HistoryLength

This is the length of history buffer of records in number of Interval Blocks (e.g. 12 for a year if BlockDuration is “monthly”). Note: this is what the DataCustodian offers; however, the buffer may not be full for transitional metering systems; in these cases less data will be returned until the buffer is full.

BulkAccountCollection

Used where the DC wants to provide for the reporting of multiple UsagePoints in a single Subscription. The number of UsagePoints is represented by the value in the assignment statement – e.g. 4 UsagePoints would be BulkAccountCollection=4.

Green Button Connect My Data Testing and Certification

• Complete function block descriptions– Current:

• [FB_3] Green Button Connect My Data• [FB_13] Security and Privacy classes• [FB_14] Authorization and Authentication (OAuth)• [FB_19] Partial update data

– New?:• Core Rest Services

– GET Batch/Subscription– …

• Resource Level REST– GET PUT POST DELETE individual resources …

• SFTP for Bulk• REST for Bulk• Use Case 1: Client Registration• Query Parameters• On Demand Requests (as opposed to Notification followed by GET)• PUSH model• Offline Authorization to Complement OAuth – should this be outside the scope of standard and testing or standardized?

– No standard isolated way to get the token to a third party without OAuth– On exceptional basis some customers can’t be required to use a web account– Sometime commercial accounts don’t need privacy and want a service provider just to register the data.– Could use Notification service to tell TP about new authorizations made by DC. Out of band how RetailCustomer is identified to the TP– “transitive” model TP gets bulk data from DC and then becomes DC – can this architecture be of help here?– Possible provision by DC of access token for conveyence to thirdparty devoid of customer information. Maybe even encrypted for TP as in software activations:

» “Please provide this to your TP (the text between the ====)» =============================================» ashoiqwherfhdjnvcjq2dhijvkqnvoiikdfv» =============================================“

Questions• retailCustomerID=authorization=subscription

– Corresponds to a single authorization– Results in one or more usagePoints being associated with subscription– Scope=

“FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLength=13;BulkAccountCollection=10”

• Says that the BulkAccountCollection has 10 usage points

Authorization provides two URIs that can be used:resourceUri GET this to retrieve usage data (all UPs)authorizationUri GET/PUT details of AuthorizationNotification is a list of URIsAll nested resources under the UPs are accessible under the single authorization

Service Request 83 – including Function Block for optional customer info (service point address, etc.)

Service Request 84 – having scope selection screen on Data Custodian Site vs 3rd Party site

[85] Time of Use tier indicator alignment with SEP 2.0

Here is a list of topics raised by you all that we will touch on

• Issues Raised and Implementation Questions– How to use BR=bulkID – relates to HD #61– Service Request 83 – including Function Block for optional

customer info (service point address, etc.)– Service Request 84 – having scope selection screen on Data

Custodian Site vs 3rd Party site– Tariff Model Resource

• Green Button Connect My Data Testing and Certification– Complete function block descriptions– Complete test case requirements

How to use BR=bulkID – relates to HD #61

• Application Profiles– BulkID was proposed for large sets of authorizations – One account level authorization on top of service level

accounts – how to do this• Degrees of freedom we have now – can we cover

– Subscription – 1 or more Usage Points• Granularity of a customer authorization

– BulkID• “macro” for a large set of existing authorizations

– Is there another degree needed?

Contributed by Jerry Yip Clarification/confirmation about ESPI standard: Does ‘shared resource key’ referenced in the NAESB

Ratified word doc correspond to Access Token for oAuth? Yes: This is the access token in the new Oauth 2.0 paradigm.

Formal Submission of Application Profile for bulk (vs. batch?) use case as part of GB/GBC Conformance Testing Plan Write up coming to test concept of BulkIDs

Question: (options to address 1 Acct to many SA issue)- Does UUID correspond to usage point (1-to-1 relationship)? Is there passing of UUIDs (as resource terms in Scope section of GBAuthorization) during authorization sequence? (how would 3rd Party know multiple usage points have been authorized via single oAuth sequence/login?)- Can multiple access tokens be issued (1 token per SA) per oAuth session? An Authorization is one access_token How does Third Party get to know the depth of data (how many Ups) are in the authorization

Perhaps an extension of scope string to have numUPs? Request to consider scope selection screens at Data Custodian Portal instead of 3rd party portal (Need

customer to select SAs to share – only Data Custodian has that info) – also minimizes number of redirects (?)

Customer info as optional functional block (atom feed) for authorization (sharing with 3Ps)

John suggests – prep a large multi account data set and test against a reference sw implementation and measure. SFTP and Streaming, compressed and non-compressed method and compare.

=

How to use BR=bulkID with application to account and account groupings, as well as, large ThirdParty

collections of Authorizations

• Establish Use Case Story for Commercial Accounts

• Design Scope String(s) that convey it• Repaint the storyboard with appropriate

content

Application Profile• Per footnote 1, pg 20 of GBAuthorization.doc:

– A “Web Customer” may actually manage more than one “Retail Customer” where “Retail Customer” is an actual “Customer Account”. Thus identifying the specific Retail Customer may be part of the scope selection on both sides. The scenarios in this section refer to the “Retail Customer” for simplicity.

• Suggest: new FB or Application Profile to properly capture this scenario• [FB_31] Web Customer Manages Multiple Customer Accounts

(OR: 3.9 Application Profile)• For GBCMD, this FB/AP contains tests associated with a Web Customer accessing a Data Custodian’s Web Portal to manage multiple

customer accounts. Upon log in to the Data Custodian’s Web Portal, the web customer can manage multiple customer accounts, for which each customer account can represent multiple usage points (for electricity and/or gas). This mostly impacts large agricultural and commercial customer accounts for which a single web customer can represent hundreds to thousands of individual usage points – imagine a franchise manager with multiple branch locations across a data custodian’s service territory.

• In this scenario, the Web Customer should have the ability to authorize, deauthorize and change scope on an individual “usage point” basis and optionally at the larger aggregated web customer or customer account basis. This includes the ability to perform one-time authorization of multiple customer accounts by a single web customer to third party, and any subsequent scope changes (whether on an aggregated or individual basis) – third party acknowledgement/communication of which customer accounts have been authorized, deauthorized or whose scope has changed needs to be determined.

• Notes:– Whether scope selection in this scenario should live on the 3rd party portal vs. the Data Custodian’s portal needs to be determined as well.– Collection has one description or multiple?– What is the scope string for this use case?– Is there a need for a bulkId in this case (maybe not).– New Scope Resource Term= “BulkAccountCollection”– Scope= “FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLength=13;BulkAccountCollection”

• 1/14/2014– To allow the TP to know how many Ups are being provided, suggest Add to BulkAccountCollection a number of UsagePoints

BulkAccountCollection=nnn

UsagePoint Grouping in Commercial Account Management

Service Agreement

Service Point

Meter

Premise

Account

Person

1:n

1:n

1:n

1:n

1:1

HAN Device1:1

BulkId

SubscriptionId

UsagePointId

/web accountVia gui

Scope= “FB=4,5,15;IntervalDuration=3600;BlockDuration=monthly;HistoryLength=13;BulkAccountCollection”