ABC CONNECTED CARExploring voice-led interaction and CUIs

ABC R+D

WHAT IS THIS PROJECT DEMONSTRATING? ABC media moments in a future car driver’s morning commute, in a connected car, using conversational user interfaces (CUIs) and voice-led interaction.

ABC Digital Network Research and Development – ABC Connected Car: CUIs

What are CONVERSATIONAL USER INTERFACES?

A conversational interface is any user interface that mimics chatting with a real human - i.e. it allows us to use natural language to find answers, get directions and more.

A conversational UI can take place as a back-and-forth text chat, email or spoken conversation.

“A conversational interface enables humans to converse with

machines (in much the same way we communicate with one another)

in order to create, access, and manage information and to solve

problems.” — MIT

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

THE ERA OF CONVERSATIONAL INTELLIGENCE

Speech recognition

Natural Language Understanding (NLU)

Understanding intent, contextually aware always listening and learning

Artificial Intelligence

+ +

=

1. The interface is a conversation.

2. The conversation is with a smart, connected object or thing, or with a bot or virtual assistant like Siri, Cortana, Alexa or Google Assistant.

3. The smart object or virtual assistant we are conversing with can also

start the conversation. This is a novelty today - but in the future

many objects, devices and apps will initiate conversations with us.

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

Three ways that CUIs will change our lives:

Future, advanced CUIs will allow us to talk “with” them, not “at” them…

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

Your CUI + virtual assistant will know you the way a friend or colleague does> They will use perceptive listening (automatically wake up when you speak)

> They will remember context and past conversations, and detect how you are feeling based on the sound of your voice and even your body language.

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

They are perfect for smart environments and the ‘Internet of Things’

> In smart homes, cars, and workplaces where hands-free mobility is key.

> Smart objects will be EVERYWHERE - many will be too small to have screens.> We’ll increasingly feel less self-conscious about interacting with our voices.

CUIs will become ubiquitous

What’s driving the rise of CUI’s & VOICE INTERACTION?

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

WeChat CUI — micro apps within a conversation timeline

Telegram Quartz News Today’s big virtual assistants are audio bots - Siri, Google

Assistant, Alexa, Cortana Viv

MESSENGER APPS, BOTS, VIRTUAL ASSISTANTS AND ADVANCED AI = MAKING LIFE EASIER

— SMH, April 2016

“There are already 3 billion of us using instant messaging apps, with nearly a billion more expected to come on board by the end of the decade.”

1. Messenger apps are now like “mini webs” - conversations include mini applications. Chatbots are super easy to install and use. These, along with voice-driven virtual assistants such as Apple Siri, Microsoft Cortana, Google Assistant, Facebook M and Amazon Alexa are becoming increasingly intelligent, more sophisticated and more human.

ABC Digital Network Research and Development – ABC Connected Car: The rise of CUIs

Facebook M (currently in beta testing)

Google Allo (ties in with Google

Assistant)

Why open and close a bunch of apps on your phone to get stuff done when you can invoke a text-message-like window and just say what you want done to a chatbot?

Bots + Virtual Assistants + CUIs + Advanced AI = future with no apps?

HOW BOTS WILL INTEGRATE OUR TECH LIVES

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

“Bots will do what Apps hoped to accomplish, but failed to deliver… One day soon, you will have a personal bot that will be able to do things like set up

appointments, help with your do-to’s, check your emails, pull data for you (say your

company's info) and even give conversational updates to your friends, family, and co-

workers. It will integrate your entire digital life in one place, make it easily accessible

and then automatically handle certain tasks for you.

Currently, Facebook and Google are working on creating Master Bots. The Master Bot

is basically a bot that can seamlessly summon services from other bots. For instance

let’s say you wanted to order a burger, currently you have to do a small research

project or use an app…

In the near future, Siri, Alexa, Facebook M or Google’s Master Bot will have access to

all of bot services be able to process orders on the back-end seamlessly. In other

words, you will tell Siri what you want, Siri will find it and give you a few great options

and you will be able to order and pay for it seamlessly.”

— How Chatbots and AI Will Fundamentally Change the Future, 2016

THE INTERNET OF THINGS & THE CONNECTED WORLD

ABC Digital Network Research and Development – ABC Connected Car: The rise of CUIs

1. It makes sense that voice is the ideal operating system of Internet of Things (IoT).Many smart objects and devices don’t have screens and keyboards, and graphical user interfaces aren’t likely to be very practical or easy to use when we start to use content and services seamlessly across smarter environments.Voice control will make the Internet of Things more human.

Only 6slices left

BUY

THE INTERNET OF THINGS & THE CONNECTED WORLD

ABC Digital Network Research and Development – ABC Connected Car: The rise of CUIs

Big tech companies like Apple, Google and Amazon are looking to win the ‘voice wars’ - so that their products and services are the ones we can easily use in all life contexts.

Apple Siri Google Assistant & Google Voice

THE VOICE WARS

ABC Digital Network Research and Development – ABC Connected Car: Defining CUIs

> Amazon - announced the Alexa Fund in June 2015, providing up to $100m in investments to support developers, manufacturers, and start-ups of all sizes who are passionate about creating new experiences designed around the human voice. So far Amazon has sold approximately 3 million Echo devices - the Echo launched in the US in late 2014.

> Apple - has made big investments in voice recognition companies — VocalIQ (Oct 2015), Novauris (April 2014). Apple bought Siri in 2010, launched it in 2011 on phones and in 2012 in selected cars; CarPlay launched in 2014, and Apple announced it would be opening up Siri to developers at WWDC 2016 so that they could connect Siri to their car apps.

> Viv AI - Viv Labs was established in 2012 by Siri’s creators; has $30m investment so far; first demo launched May 2016.

> Google - invested in speech recognition company Mobvoi in Oct 2015; acquired voice and image research firm DNN Research in March 2013; acquired SayNow in 2011; launched Google Voice in 2008 after acquiring GrandCentral; plus big investments in deep learning & machine learning (e.g. bought Deep Mind for $400m in 2014); launched Google Voice Search 2011, Google Now 2012 and Google Assistant 2016; released Android Auto app in March 2015.

> Microsoft - launched voice control for Kinect in 2011; Cortana launched in Windows 10 in 2014; builds prototype of Cortana-powered car in 2015; announced its “Conversation as a Platform” strategy at Build 2016 and bought Wand Labs to add more natural language tech to messaging apps and bots.

> Facebook - is developing it’s virtual assistant M - a beta version launched in August 2015; running FAIR (artificial intelligence research program); acquired Wit.ai in January 2015 to help it’s developers with speech recognition and voice interfaces.

‘CONVERSATION AS A PLATFORM’

ABC Digital Network Research and Development – ABC Connected Car: The rise of CUIs

This is the phrase that has been coined for a key new focus for the five tech giants - Microsoft, Apple, Google, Facebook and Amazon.> people are downloading apps less and less, and want to be able to jump between digital services across devices and spaces more easily. > virtual assistants and bots are gaining traction because they are starting to feel more convenient - combining many services and features all in one place (e.g. media players, games, in-message payments, images, bookings, social engagement).

> when you add in voice-led, conversational interaction to all of this then things start to feel natural and intuitive.

According to comScore, 200 billion searches per month will be done with voice by 2020.

This creates a $50+ billion per year market opportunity around voice search.

As of September 2015, Siri was being used by 52% of iPhone users. By June 2016 , Siri was receiving 2 billion requests per week.

In February 2016 Google reported that voice searches had doubled from 2015-16.

A 2014 Google study revealed that 55% of teens and 41% of adults used voice search more than once a day.

Voice-driven CUIs are PERFECT FOR CONNECTED CAR DRIVERS

ABC Digital Network Research and Development – ABC Connected Car: Why CUIs and voice are appropriate

It’s all about all talk, no buttons - allowing the driver to control and interact with services easily with their hands on the wheel

> these could be complimented with supplementary visual elements - e.g. simple/subtle visual activity on your windscreen — but ultimately drivers need minimal distractions

> CUIs will work with other systems and elements - e.g. wearables, smart elements in the environments you are moving through

Voice interactions are: > fast - we can speak faster than we can type > easy - convenient, hands-free > personalised and context driven - ability to understand a wide context of questions based on prior questions > offer random access - as opposed to hierarchical graphical user interfaces (GUIs)

Virtual assistants will be necessary in connected cars Our future cars will come ready-made with an increased range of functions and distractions = think of the assistants as co-pilots

> in addition to keeping check of your car and how well and safely it’s operating, your assistant will also deliver other kinds of content and services as and when you need during your journey - e.g. traffic, weather, information about environments you are passing through, news, entertainment.

> some cars might come with their own embedded virtual assistant. It seems likely that we’ll prefer to have a single assistant that seamlessly follows us across all contexts of life - however not all car manufacturers are happy to cede control of their systems to a third party.

ABC Digital Network Research and Development – ABC Connected Car: Why CUIs and voice are appropriate

TECH EXAMPLES

Capio CUI chip for cars (and other appliances) - includes facial recognition

Dragon Drive - voice controlled virtual assistant for all parts of life

Android Auto - uses Google’s voice commands

Google Home

Apple CarPlay - includes Siri

Navdy - voice and gesture controlled HUD for your car

Navdy - voice and gesture controlled HUD for your car

Navdy - voice and gesture controlled HUD for your car

Sony Experia Ear - could be a device used in the car in the future

Amazon Echo with Alexa virtual assistant

Amazon Echo links in with Mojio (connected car platform)

Thank you.

ABC R+D

Get in touch: rd-contactus@abc.net.au