cortana / speech platform - sec.ch9.mssec.ch9.ms/slides/winhec/2_03_cortana_speech_platform.pdf ·...
Post on 30-Jan-2018
223 Views
Preview:
TRANSCRIPT
Cortana / Speech Platform
2
Cortana
truly personal
across all your devices get things done.
Cortana Across your Devices
Cortana Usage is Growing
In 9 months since Windows 10 launched
• Over 270 million active Windows 10 devices
• Over 5 billion questions asked of Cortana
• One million voice questions are directed towards
Cortana on a daily basis
Of these, 15% use the keyword
to initiate
On any given day, 25% of Cortana PC users
use speech
&Hey Cortana
Cortana Drives Customer Satisfaction
intent to recommend
improve customer satisfaction
increase “re-purchase” intent
With Windows 10 Anniversary Update
Nearly 1,000 Apps
withVoice
Commands
Cortana functionality for Windows 10 is available in the following languages and countries
English-US
English-UK
Chinese-China
German-Germany
French-France
Italian-Italy
Spanish-Spain
Geographies
Japanese-Japan
English-India
English-Canada
English-Australia
Portuguese-Brazil
French-Canada
Spanish-Mexico
In plan for Windows 10
Anniversary Update
Cortana with Voice (CwV) Marketing Assets
Icons only
Icons + Titles
Titles and
descriptions
Technical
Requirements
to market
Cortana with
Voice (CwV)
*The “Cortana with Voice” title, icon, description, and associated assets
such as messaging can only be used if all following criteria are met:
1. The device is tested using Microsoft’s “Speech platform tools”
2. The device’s test results meet or exceed the Standard spec
Cortana with Voice
Your truly personal digital assistant who helps you get things done, even
hands-free - just say “Hey Cortana” to get started.
Cortana with Voice
Standard vs. Premium Spec
Windows 10 Anniversary Update & Beyond
Standard • Normal ambient noise level
• 0.5m
• Standard experience
• Cortana with voice (CwV)
• Normal ambient noise level
• 0.5m
• Standard experience
• Cortana with Voice (CwV)
• Additional tests for Voice Activation
(optional)
Premium • More challenging conditions
• <1m• Talk level: 89 dBA• Premium experience
• More challenging conditions
• 4m• Talk level: 99 dBA• Far-field experience
• Additional tests for Voice Activation
*Early draft, subject to change
Cortana with Voice Testing Scenarios
Run the OEM Verifier tool and ensure the driver settings aligns with the design
• Utility - OEMVerificationx86.exe
• Driver reports these key elements correctly
Number of microphones
Microphone geometry
Driver mode supported
Follow the test guidance to validate Cortana with Voice performance
• Validate in 3 scenarios: Quite, Ambient Noise (café and pub), Echo
Parse data and ensure the test result are at or above the provided benchmark
• Utility - OEMScoreUtilityFarAndNearx64.exe
• Calculate the speech accuracy score & troubleshoot the issues
Cortana with Voice Validation Process
Audio effects supported
Exposes Audio Pipeline being used
Default microphone gain
Examples of Test Result
OEM Pipeline Microsoft Pipeline Speech Accuracy Score
Future Planning for CwV Requirements
*Early draft, subject to change
Platform Experience
• Platform providers test voice activation false accepts (FA) and correct accepts (CA) for every supported locale
• These two tests are not needed to be run by OEMs
Release People Scenario Near-field Far-field
0
degrees 50
degrees 0
degrees 50
degrees
2016
Male
Quiet (Private Office, Living Room)
90% 90%
Medium (Coffee Shop, Kitchen)
Music listening in Quiet
Female
Quiet (Private Office, Living Room)
90% 90%
Medium (Coffee Shop, Kitchen)
Music listening in Quiet
Future
Male
Quiet (Private Office, Living Room)
92% 92% 90% 90%
Music listening in Quiet
Medium (Coffee Shop, Kitchen)
Music listening in Medium Ambient
Female
Quiet (Private Office, Living Room)
92% 92% 90% 90%
Music listening in Quiet
Medium (Coffee Shop, Kitchen)
Music listening in Medium Ambient
Children (5-12)
Quiet (Private Office, Living Room)
92% 92% 90% 90%
Music listening in Quiet
Medium (Coffee Shop, Kitchen)
Music listening in Medium Ambient
OEM Experience
OEMs test specific devices for good voice activation accuracy on EN-US and JA-JP. These tests will become quality gates beyond 2016.
OEMs test specific devices for good
speech recognition on EN-US only.
When the Microsoft far-field processing is used the recommendation is to use the microphone array geometry shown above
For 360 degree operation an 8-mic circular array is recommended
For 3rd party solutions use their spec recommendations for microphone array and geometry
Recommended Array for Far-Field
Technical Requirement References
Requirement Detail Reference / Spec
Speech Platform
test tools
Application and
content guidance for
testing
Microsoft Speech Platform Test Tools
http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=59735
Speech Platform Audio Calibration files
http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=57500
v1.1 “Standard”
benchmark
Quiet, Ambient, Echo
scores
Speech Platform Input Device Recommendations v1.2
(§2 – Device.SpeechRecognition recommendations)
http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=58099
https://msdn.microsoft.com/en-us/library/windows/hardware/dn915051(v=vs.85).aspx
v1.1 Test setup Measurement setup Speech Platform Input Device Test Setup v1.1
http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=58099
https://msdn.microsoft.com/en-us/library/windows/hardware/dn915051(v=vs.85).aspx
Test setup for
accessories
Measurement setup Speech Platform Input Device Recommendations – Accessories Support v1.0
http://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=59861
v2.0pre3 Specs Benchmarks and
measurement setup
Speech Platform Input Device Recommendations v2.0pre3
Speech Platform Input Device Test Setup v2.0pre3
https://connect.microsoft.com/site1304/Downloads/DownloadDetails.aspx?DownloadID=60086
• Ensure your drivers have the correct settings (mic geometry, mic gain, number of mics, expose audio pipeline being used)
• Download and test the latest specs and tools
• Provide feedback on Speech Platform v2.0pre3 specs
Call to Action
Online Survey Formhttp://aka.ms/winhecfeedback
Join WinHEC LINE Community @winhec
Download WinHEC presentations here:
https://channel9.msdn.com/Blogs/WinHEC* Gifts are limited. They will be offered at
“a first come, first serve” basis.
top related