towards social robotics on smartphones with simple xyzv sensor feedback

2015/11/29

Marat Zhanikeev [email protected] HCI研@別府

PDF: bit.do/151129

Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback

(dumb) Robotics Today

• most robots today are dumb• if you have all the necessary inputs, no room/need for AI, just use your inputs

• the case of HAL: weight shifting,bioelectricity, relative angles

M.Zhanikeev -- [email protected] -- Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback -- bit.do/151129 2/192/19

From to Smart Robotics• objective 1: there is a need for truely smart/flexible robotics• objective 2: preferably within commodity reach (smartphones)

Increasing specialization

Increasing smart-ness

Most robots today

Social Robotics


Social Robotics Basics

• basic idea: start with a blankrobot and teach it a task◦ humans are trainers

• optimization: reasoning byreinforcement learning

• complexity: reducesearch space via humanfeedback


Social Robotics in Soft- vs Hard-ware• good parallel between soft- and hard-ware social robotics• for higher reliability, vision/recognition in hardware robots should besimplified◦ reliable feedback gestures, possibly even without vision

• context is natural to software but is also important in hardware (more later)

Generic Use

Teaching, Guidance Reasoning

Human Role

Hardware Robots

Wide range of behavior

Reinforcement Learning

YES. Vision, recognition Guide only

Software Automation

Any kind of context

Bayesian Classification

NO Not needed

Guide and decision - maker

Potential for simplification


Social Robotics in Context Management• example of software robot that processes folksonomy bigdata

Robot

(careless) Input

Human Human

{structure}

(pinpoint) Select

Browse (or use otherwise)

Some Knowledge

(folksonomies, knowledge bases, databases, indexes, ontologies, etc.)

(metromaps )

guide, trainer

sources, crowd

Recommendation Bot = Rebot


Social Robotics in Context (2)

0 5.2 10.4 15.6 20.8 26Time/Learning Sequence

0

4

8

12

16

20

Rebo

t Ord

er v

s Use

r Sel

ectio

n

Linear fit (circle bullets)Rebot: New TrainingRebot: Order of Selection in Rebot List

0 5.2 10.4 15.6 20.8 26Time/Learning Sequence

0

4

8

12

16

20

Rebo

t Ord

er o

f Use

r Sel

ectio

n

• starts off really dumn, but gets better with time

• diff between human and robot decisions decreases

• when diff is mostly zero, human trainer can go home


Social Robotics and Blackswans• a useful "show me only the blackswans" application

• useful in : disaster scenarios, literature reviews, etc.• crucial to remove human component as some point

Human judgment

Auto judgement

Folksonomy


Social Robotics and Blackswans (2)

Metromap Classifier

Human

Check Metromap

Fuzzy?

Cold? Hot?

Robot (Automatic Classification)

Bad

Input

No Yes

No

No


Simple XYZV Channel


Model of Robotic Decision Making• on smartphones, easy access to XYZ gravity and V acceleration = XYZV input

• perfect if the same channel can be used for sensor input and human feedback

Model

Human

Robot

Actions

Constraints, logic

Same -channel interaction

Context management


The Single-Channel Objective

Number of distinct data inputs

Reliability

Single-channel

Acceptable level

1

Dumb curve Smart curve


Experiment (1)

• attach smartphone to one ankle• detect up-the-stairs steps• use human feedback to learn whichdetections are wrong


Experiment (1) Feedback

1. smartphone notifies detected steps viavibration

2. human punishes smartphone for baddetections (negative feedback)

• stroke/hit your smarphone when it is wrong/bad

• jerk/shake a leg to loosen it up

• both are natural interactions for humans

• positive feedback is harder (so far)


Experiment (1) : Resultsnegative mirror = moving average“Stroke” feedback x

yzv

“Jerk” feedback

Event detected

StepStroke

StepJerk


Experiment (1) : Other Results

• about 100 experimental sessions

• 60% are FPs, 30% are FNs• within the FPs, in 90% of them stroke/jerk feedback was detected

◦ 3s timeout, detect unusual XYZV signal right after vibration◦ vibration itself is too small to register in XYZV


Experiment (2) Context

• current work in progress

• idea: establish context first, then detect steps• experiment: walk with a smartphone in yourpocket


Experiment (2) Context (2)xyzv

flat walking• horizontal/flatwalking

xyzvwalking up the stairs

• walking up thestairs


That’s all, thank you ...


towards social robotics on smartphones with simple xyzv sensor feedback

Technology