1June 1, 2017

Cognitive Computing and Conversational Agents

Rob High, Jr.IBM Fellow, Vice President

Chief Technology Officer

IBM Watson

2June 1, 2017

Memex: Vannevar Bush, “As We May Think”, The Atlantic Monthly, 1945

• A device for people to

compress and store all

of their books, records,

and communications,

mechanized to provide

rapid recall, access

and cross document

linkages…

3June 1, 2017

1970

“In from 3 to 8 years we will have a machine with the general intelligence of

an average human being” – Marvin Minsky

1977

“There is no reason anyone would want a computer in their home” – Ken

Olson, Chairman DEC

4June 1, 2017

DeepQA: Incremental Progress in Answering Precision on the

Jeopardy Challenge: 6/2007-11/2010

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Pre

cis

ion

% Answered

Baseline 12/06

v0.1 12/07

v0.3 08/08

v0.5 05/09

v0.6 10/09

v0.8 11/10

v0.4 12/08

v0.2 05/08

V0.7 04/10

IBM Watson

Playing in the Winners Cloud

5June 1, 2017

With Precision, Accurate Confidence and Speed, the rest was History

6June 1, 2017

Why are Cognitive Systems Important to Modern Life?

Digital information growth is exponential

Understand the intention of human expression

Act as a research assistant to pinpoint relevant information

Amplify human abilities to scale the democratization of expertise

2.5 Exabytes of data produced every day

This is where we

were in 2016

This is where we

will be in 2020

7June 1, 2017

“ a”By 2018 half of all consumers will regularly

interact with services based on cognitive

- IDC FutureScape

8June 1, 2017

Cognitive systems amplify human cognition

9June 1, 2017

What is Cognitive Computing?

• Cognitive systems understand human expressions – textual, verbal, visual

• By reasoning about the actual intention or problem being addressed

• They learn how to recognize patterns of meaning through examples and feedback

• And they interact with humans on their own terms, and in a way that inspires people.

… and do so at enormous scale.

10June 1, 2017

Some common concepts

Analytic Systems use statistical techniques for detecting

patterns within data

• Yield an understanding of historical or current state from which to draw

conclusions

• Can be used recognize trends, or predict unknown information

Natural Language Processing (NLP) is a core function

for parsing and identifying significant words in language

• Uses Annotators to identify significant words

• Increasingly used to extract Entities and Relationships into Knowledge Graphs

• Often rule-based, but a growing focus on statistical techniques

Machine Learning is a class of statistical techniques

that use training data to recognize the correlation

between a set of feature patterns and outcomes

• Enables the prediction of other similar outcomes given a similar pattern of feature

values

• Often associated with the problem of identifying the weak signals in the noise that

indicate meaningful patterns

Cognitive Systems leverage machine learning to predict

meaning in features of human language (spoken,

written, visual) and related forms of human reasoning

• Reasoning strategies often employ high-dimensional feature spaces

• A lot of work goes in to identifying and refining appropriate features

Deep Learning is a rapidly maturing space, based on

neural network techniques, that are taught to find their

own features

• Generally very high dimensional, fed with an arbitrarily high number of primitive

signals and left to sort itself out through training data and reinforcement

Cognitive Solutions leverage a combination of cognitive

system reasoning strategies and other analytic and

classical computing techniques to solve for a problem

• Learn from training data and experience; Interact to the human condition;

Transparent about their rational and confidence; Evolve to improve – Amplify

Human Cognition

11June 1, 2017

The Watson Cognitive Computing Evolution

• 2011: Jeopardy … introduced a Factoid Pipeline for use with general domain knowledge

• 2012:

– Watson Discovery Advisor … leverages the Factoid Pipeline around specific domains to ‘help find the questions you’re not thinking to ask‘

– Healthcare … Utilization Management, Oncology Treatment Advisor, Clinical Trial Manager

• 2013: Watson Engagement Advisor … Introduced a Passage and then FAQ pipeline

• 2014:

– Expanded Watson Discover Advisor … added knowledge extraction and graph visualization

– Introduced Watson Explorer … combined with Watson Content Analytics

– Watson Ecosystem … made the (WEA) Passage and FAQ pipeline available as a service

• 2015: Watson Developer Cloud … dramatically expanded the range of services designed to interpret the human condition,

and made them available on Bluemix

• 2016: Enhancing Human Engagement … focusing on conversational agents, with emotion detection and expression

• 2017: Developer experiences … Cloud-native scaling and developer efficiency

12June 1, 2017

Watson Reference Model

13June 1, 2017

Watson is available as a set of services delivered as APIs in the Cloud

Higher Reasoning Skills

• Conversation

• Discovery

Higher

Reasoning

Skills

ibm.com/bluemix

Can be combined with the 100s of other available services on Bluemix

14June 1, 2017

Conversation

• Add a natural language interfaceto your application to automate interactions with your end users.

• Common applications include virtual agents and chat bots that can integrate and communicate on any messaging platform.

• Model is trained on user-defined intents, entities and dialogs.

• Expanded to recognize the emotion of the user and to respond accordingly.

15June 1, 2017

A cognitive chat-bot is a conversation

A conversation should:

• Engage the user

• Focus on the user’s broader concern

• Build on an idea

• Leave the user inspired and satisfied

16June 1, 2017

A cognitive conversation can occur through many different

channels• Mobile Apps, Web Applications

and IVR telephone calls

• Messaging Platforms

– Facebook/M, Twitter, Twillio, Slack,

etc.

• Personal Assistance Devices

– Amazon Echo, Google Home, Siri,

etc.

• Even, in a collaboration multiple

people in a conference room

Bot APIs

Watson Bot

Framework

Proprietary APIs

e.g. Reserve Table

Business Lo gic

Watson Conversation

Now, imagine you’re in …

18June 1, 2017

Now, imagine you’re in …

19June 1, 2017

Watson Virtual Agent

• WVA is a collection of 105 common, pre-built Conversations

– Account Management– Billing– Device Management– Help / Information– Complaints– Order Management– Payment– Sales– Service Management

• Just configure, add your App, and Go!

20June 1, 2017

Discovery Service

• Entry point into deep reasoning over literary and structured resources

– Entities

– News

– Sentiment

– Time-sequenced

– Co-mentions and Trends

– Related concepts

• Over 300,000 news articles and blogs added daily

• Connect your enterprise data sources

21June 1, 2017

Watson is available as a set of services delivered as APIs in the Cloud

Knowledge Organization Skills

• Watson Knowledge Studio

• Document Conversion

• Retrieve and Rank

Knowledge

Organization

Skills

ibm.com/bluemix

Can be combined with the 100s of other available services on Bluemix

22June 1, 2017

Watson Knowledge Studio

• Enable subject matter

experts and developers to

teach Watson the linguistic

nuances of industries and

knowledge domains

23June 1, 2017

Retrieve and Rank

• Uses machine learning

to improve search

across documents

using natural language

questions

24June 1, 2017

Watson is available as a set of services delivered as APIs in the Cloud Watson Foundational Skills are grouped into four categories

Foundational

Cognitive

Skills

Language• Author

• Concepts

• Dates

• Entities

Speech• Speech to Text

• Telephony Speech to Text

• Keyword Spotting

Vision• Image Classification

• Face Detection and Attribution

• Celebrity Recognition

Empathy• Personality Insights

• Tone Analyzer

• Emotion Analysis

• Text to Speech

• Expressive Text to Speech

• Relations

• Typed Relations

• Sentiment

• Taxonomy

• Text Extraction

• Feeds

• Keywords

• Language

• Microformats

• Publication Date

• Visual Text Recognition

• Similarity Searching

ibm.com/bluemix

Can be combined with the 100s of other available services on Bluemix

25June 1, 2017

Expressive Text to Speech

• Enables speech

that reflects

intended tone

– Expressive SSML

– Voice

Transformation

SSML

26June 1, 2017

Watson Visual Recognition Service

• Image Classification

• Face Detection and Attribution

• Celebrity Recognition

• Visual Text Recognition

• Similarity Searching

27June 1, 2017

I'm a hard working adventurous, very talented man who's been caring and helpful throughout my life, I like to travel, play my guitar, dance, and cook, I love the beach, sailing my boat, and the outdoors.

I raised two great kids and now I'm starting a new chapter in my life.

Thanks.

What I’m doing with my life

Working toward a new goal, keeping fit, helping others, and traveling whenever Iget a chance.

I’m really good at Listening, enjoying the moment, and many other things.

The six things I could never do without

Family, the ocean, intimacy, friends, adventure, music, love.

On a typical Friday night I am

Meeting with friends, listening to a band or playing my guitar, dancing or just staying home with someone special and enjoying each other.

You should message me if

You're looking for a relationship with someone that likes to sail his boat, ride bicycles, travel, swim, go to the beach, listen to music and enjoy everyday pleasures together.

Tone Analyzer Understands and Helps Fine Tune Your MessageUses psycholinguistics, emotion analysis and language analysis to assess Tone

You also need to know how people are feeling …

29June 1, 2017

You also need to know how people are feeling …

30June 1, 2017

Emotional Analysis helps build empathetic systemsUses state-of-the-art machine learning models and feature engineering techniques to predict emotion labels

31June 1, 2017

Personality Insights

32June 1, 2017

Many institutions are already building advantage from

conversational chat-bots• H&R Block – Tax preparation

• Staples/Easy System – Buy on-demand

• Autodesk/Otto – Problem resolution

• The Weather Channel – Personalized weather

• Royal Bank of Scotland – Account management

• 1-800-Flowers/Gwyn – Gift concierge

• Meeka – Wedding Planning

• WayBlazer – Travel conierge

• Treasure Island – Customer service

• Ross – Legal research

• Up All Night – Music concierge

33June 1, 2017

Thank You