Future AI:Autonomous Machine Learning and Beyond

Harri Valpola, CEO

The Curious AI Company

2017-03-20

Handcrafted SW

Handcrafted concepts,

useful in narrow problems

– Perception

Learning

Autonomy

Reasoning

–

–

+

Three waves of AI

Deep learning

Classification and prediction,

lacks object representations

Advanced AI

Autonomous learning and

symbolic reasoning

The current AI boom

Perception

Learning

Autonomy

Reasoning–

Perception

Learning

Autonomy

Reasoning

+

–

+

+

+

+

+

Curious AI solves this

Wave 1: 1980s- Wave 2: 2000s- Wave 3: 2020s-

Adapted from DARPA’s 3-wave model

CAI: Leader in autonomous deep learning

Neural

network

Input

Label Class X

Traditional deep learning

Currently

typical:

Abstractions

provided by

humans

Autonomous deep learning

CAI

autonomous

learning

Error:

2.76%

labels

images

500

570 000

labels

images

70 000

70 000

Standard

deep learning

Error:

2.81%

Google Street View

House Numbers

dataset

WIP: Autonomous control and attention

Currently typical:

Action hierarchies

defined manually with

specific pre-defined

discrete goal types

Example: Boston Dynamics

Atlas works well in some

tasks, but does not learn

With CAI Core:

Autonomously learn

action hierarchies

with model-based off-

policy RL

Hierarchical RL Attention

Currently typical:

Manually make sure

training data is

balanced and relevant

Example: ImageNet

dataset designed to train

well

With CAI Core:

Autonomously focus

attention on most

relevant data

Task planning

Task coordination

Torso

controller

Head

controller

Hands

controller

Torso

subtask

Head

subtask

Hands

subtask

Unsupervised learning: The last frontier?

“We need to solve the unsupervised learning problem before we can even think of getting to true AI. And that's just an obstacle we know about. What about all the ones we don't know about?”

Dr. Yann LeCun

Director of AI Research at Facebook, Prof. NYU

March 2016

Fundamental problem in current wave:Incompatible representations

Easy to represent objects and

structured relations, but discrete

and handcrafted categories

Objects and relations

Real-world problems require both

types of representations but they

are fundamentally incompatible

Neural networks learn, but lack

representational power for objects

and their interactions

Handcrafted software Neural representationsHybrid systems

Neural representations

A NB N DD AN H

1 33 1

2 12 22

3 17 19

4 3 2

5 3

6 15 13

7 2 5

if distance < 100:

cmd = BREAK

else:

if distance >= 800:

cmd =

ACCELERATE

Discrete categories Coding and rules

Segment pixels to objects

Detect object bounding boxes

Unique solution: Neuro-symbolic deep learning

Brain-inspired

neuro-symbolic

representation

See video: tinyurl.com/taggervideo

CAI: Leader in autonomous segmentation

Currently typical:

Humans draw

segment

boundaries

Example: 70k man-

hours spent labeling

the MS COCO dataset

Traditional segmentation Autonomous segmentation

With CAI Tagger network:

Autonomous segmentation, significant boost to

semi-supervised classification

Future AI: Neuro-symbolic reasoning

Current models cannot group features into

objects or represent their interactions

Neuro-symbolic representations will allow

deep networks to represent objects and model

their interactions

“I want to surprise my girlfriend with something special this weekend”

Competitors lack full AI stack

Deep learning

integration

Autonomous

recognition

Autonomous

segmentation

Autonomous

control

Neuro-symbolic

integration

X

X

X

X

X

X

X

X

X X

X

X

X

Technology layers

Core algorithm examples

• Supervised deep learning

• Tree search

Component examples

• Facial expression sensing

• Speech recognition

• Image recognition

• Superficial language

understanding

Application examples

• Driver assistant systems

• Speech user interface

• Machine translation

CAI Core algorithms

• Autonomous learning

• Neuro-symbolic reasoning

Component examples

• Complex scene understanding

• Theory of mind

• Common sense

• Advanced language

understanding

Application examples

• Self-driving car

• Automated accountant

• Customer service

Core

algorithms

Components

Applications

Wave 2: Hybrid systems CAI tech layers in Wave 3

Transition from technology to applications

2015 2016 2017 2018 2019 2020-

CAI Ladder state-of-the-

art autonomous learning

on MNIST and CIFAR

Patent filed

File further patents

Three patents in pipeline

CAI Tagger state-of-

the-art autonomous

segmentation

Patent filed

CAI state-of-the art

autonomous learning

on SVHN

Patent filed

Engage in customer

projects

Currently negotiating with

automotive player,

Huawei, and Visma

Decision point: focus

on one vertical based

on highest value fit

with CAI technology

Main focus: Develop CAI core technology Main focus: Develop applications

Seed round

Develop CAI core technology

Beat established benchmarks

Build team

Round A

Complete CAI core technology

Engage with customers in consulting mode

Expand team and build components team

Later round

Ramp up selected

application, e.g.

digital worker

Timeline for CAI core algorithm development and applications

Component

licensing

CAI Core

Algorithms

Seed Phase Phase A

2014 2015 2016 2017 2018 2019

Autonomous recognition

Classification components

Digital worker

components

Range of components

E.g. complex scene understanding, control,

language, communication, interaction

Autonomous segmentation, control

and attention

Neuro-symbolic reasoning

Selected

application

2020-

Selected application

E.g. virtual workers

Phase B

Thank you

HarriValpola, CEO

The Curious AI Company

harri@cai.fi