#Build2016

Cortana Extensibility: How Cortana Can Proactively Drive Engagement With Your AppsPaul Bloom – Product LeadKeith Herold – Technical PM Lead

Session Code: B834

Cortana is your truly personal digital assistant working

across all your devices to help you get things done.

Cortana Across your Devices Cortana is available on Windows 10 desktops, tablets, and phones – as well as iOS, Android and Cyanogen phones.

Productive

Personal & trusted

Proactive

Personality

Helps me get things done by completing tasks or bringing me a step closer

Keeps one shared notebook with what she’s learned about me; I can access thenotebook from any Cortana experience and Cortana is always aware of its contents

Always looks out for me (e.g. using contextual awareness and her notebook)

Consistent visual identity ( ), personality traits, voice (e.g. Jen Taylor in the US); Adapts continually to be globally inclusive and culturally relevant

Natural interactionLets me interact using natural language in a consistent and contextual way

Cortana Guiding Principles

Cortana’s Team of Experts

• Apps can register voice commands for users to say to Cortana

• Cortana launches the app to complete the task

• App can use Cortana’s canvas and voice to present feedback to the user

• Available for Universal Windows Apps built with XAML and JavaScript

Voice Commands for Universal Windows Apps

Hey Cortana, order me food with Just

Eat

Hey Cortana, create a station for Hozier on Pandora

Hey Cortana, send a message to Jonas Brandell on Viber

Hey Cortana, Tweet flying into San

Francisco tonight

Nearly 1,000 Apps withVoice

Commands

New: Proactive ActionsProactively suggests actions to users, at just the right time, to drive usage of your apps and websites

Easy to register actions in Cortana portal

Works everywhere Cortana is starting with Windows 10 and Android

Drive Usage of Your AppsYou Register ActionsCortana has Insights

Users Get Things Done, Effortlessly

Expand actions Cortana can help users do with their favorite apps and services

Surface suggestions and execute actions in the appropriate contexts

• Hand-off with context• One-click task

completion • Full delegation

Leave WorkNew in Town

Arrive at Gym

Upcoming EventIn the

MorningFlight will depart

On my way Home

Electricity bill due date

Working Late

Insights and ActionsInsights Actions

Arrive / Leave home

Arrive / Leave work

Meeting during lunch

Working late

Upcoming flight

Upcoming restaurant reservation

At morning / evening

Order food for delivery or pick-up

Make restaurant reservation

Send a message

(Re)schedule /Cancel meeting

On my way home / work

Flight has landed

Bill due date

Upcoming movie reservation

Buy movie ticket

Buy event ticket

Play music Pay bill

Your ActionsUpcoming meeting

Predefined

Actions

Order food with JUSTEAT

Works Wherever Cortana IsStarting with… …in the future

Treat my dog with Petzi

Easy to Register Proactive Actions

No additional code for deep link enabled apps and websites

Register and configure actions at developer.microsoft.com/cortana

Engineering

Cortana has InsightsShe knows facts

It is a weekday morning.The Contoso Café app is installed.

She learns about meI usually leave for work between 8 and 8:15 AM.I use Contoso Café a lot.I work in the City.

She knows my current context

“I am leaving home.”

And offers to act

You tell Cortana which insights to use

You ask Cortana for contextual information

You tell Cortana how to link data & actions

In the morning, I see…Leaving home I consent Contoso

opens

Action prerequisitesTarget deeplink-capable endpointsAppsWebsites

Manage credentialsUser Login/Authorization

Allow for missing contextual informationEffectively ‘optional’

Contextual information may be missingUser may not consent to share informationCortana can still open link, but all parameters will be missing.

Information may not existMeeting with no location.No location signals from dead zones or on a non-mobile device.

Cortana hasn’t learned that informationNew users have less data available.Users may not have told Cortana particular items yet.

Reason #1Reason #2Etc.Plan to work with partial information

Contextual info and app/site flow impactsMost workflows Contextual

information shortens tasks

Contextual data and app/site flow impacts Augment normal flow

with ‘memory’Some info may be missing

Call to Action1. Create voice commands for your Universal

Windows Apps (available now!)2. Request an invite to Developer Preview

for proactive actions:http://developer.Microsoft.com/Cortana• Early access to platform, new features• More direct interaction with development team• Ready-to-release at program exit

LocalesEn-US, en-GB, zh-CN

What to do at //BuildVisit our boothExpo Hall main entrance

Complete the Voice Command Code ChallengeExpo Hall

Proactive Actions and Voice Commands at Code LabsWednesday, Thursday, and Friday - Room 2009

Meet the Cortana ExecutivesThursday, 3:30 – 4:30 - Mainstage Theater

Attend the 200-level SessionThursday, 5-6 pm - Moscone West/2005

Online ResourcesPortal: http://developer.microsoft.com/Cortana

This session deck (when available)

Session 200 deck (when available)

Please Complete An Evaluation FormYour input is important!

or

© 2016 Microsoft Corporation. All rights reserved.