today s agenda - college of computing & informaticssalvucci/courses/cs630-f14/... · 2014. 11....

1 CS 630: Cognitive Systems. Dario Salvucci, Drexel University.

Lecture 9: Applications

CS 630: Cognitive Systems. Dario Salvucci, Drexel University. 2

Today’s agenda

■  We’ve spent lots of time writing cognitive models in cognitive architectures

■  Many of the models have been “small” — aimed at specific domains

■  So do the architectures scale up? Are they useful in the real world? –  Intelligent tutoring –  Driving


Observation

■  The “Two-Sigma” Effect

–  i.e., the average tutored student does as well as the top 2 percent of classroom-taught students

Students who receive one-on-one instruction perform two standard

deviations better than students who receive traditional classroom instruction.


Intelligent Tutoring Systems

■  Computer tutors that recognize and respond to student behavior and inferred knowledge

■  Advantages –  a teacher for every student? not feasible.

a computer tutor for every student? sure! –  facilitates “human-like”, personalized instruction –  enables practice, practice, practice on possibly

large (or infinite) database of problems –  but also can answer questions, provide guidance


Intelligent Tutoring Systems

■  Disadvantages / challenges –  not a human! can’t (yet) sense emotion, etc.

(is this always bad though?) –  when/how to guide student from passively

absorbing knowledge to actively putting it to use –  when/how to correct errors


ITS components

■  What makes up an ITS?

Pedagogical Knowledge

Student Knowledge

Domain Knowledge


Model Tracing

■  Model tracing = relating observed actions with hidden cognitive states

■  In essence, “think” along with the student and keep track of cognitive state

■  From a given point in time... –  simulate all possible “thought” sequences

(i.e., all possible firings for productions) –  determine which sequence best matches the

actions observed from the student –  make the best matching sequence the current “estimated” cognitive state

■  (What is the “cognitive state” in this case?) CS 630: Cognitive Systems. Dario Salvucci, Drexel University. 8

Model Tracing

■  Main components of the approach... –  Student model: a production system that

represents the necessary skills –  On-path actions: recognize correct actions by the

student based on the student model –  Off-path actions: if student performs an off-path

action, steer back to solution path –  Error feedback: provide student with bug and

help messages to steer toward a solution

■  Each component embodies certain assumptions & is non-trivial to build


Model Tracing

production firing

conflict resolution

Cognitive Model

Observed Actions


Knowledge Tracing

■  Sounds similar, but very different from model tracing

■  Knowledge tracing = maintaining estimates on how well students know certain skills –  the “skill-o-meter” –  example: algebra word problems

identifying units

entering the givens

writing the expression


■  Include “buggy” rules in our model ■  Include help to go with these rules if fired

Handling errors

“buggy” rule

Cognitive Model

Observed Actions


Field tests: LISP tutor

■  Step-by-step tutor with “structured editor” –  top-down approach to coding

(defun create-list (num) <<BODY>>) (defun create-list (num) (let (<<INITIALIZATIONS>>) <<BODY>>)) (defun create-list (num) (let ((count 1)) <<BODY>>)) ... etc.


Field tests: LISP tutor

■  Results –  mini-course at CMU with LISP tutor –  two groups in course, with & without tutor –  group with tutor took 30% less time to complete a

sequence of prescribed lessons –  AND group with tutor scored 43% = 1 std dev

higher on the post-test! •  learned faster, and the knowledge stayed with them!


More tutors in the field

■  Geometry tutor –  14 extra points (out of 100) for tutored students –  actually, only 1-on-1 tutoring; 2-on-1, only 4 pts

■  Algebra tutor –  no differences for tutoring –  problems: interface differences, truancy

■  Carnegie Learning –  company spun-off from this and other research –  tutors in several (mostly mathematical) areas

with integrated curriculum (textbooks, etc.) –  now serving 845 school districts = 340,000

students nationwide!!


Algebra word problem tutor


Driving & Driver Distraction

■  Driver distraction is one of today’s most discussed topics in research and public circles alike

■  There are many layers to this discussion... –  a scientific understanding of driving & distraction

•  theoretical developments -- our primary focus •  experiments that quantify distraction -- our secondary

focus

–  legislative efforts •  e.g., bans on handheld cell phones?

–  public-relations efforts •  how can we get people to stop being distracted?



■  Example: GMAC study, 2006 (>5000 people)

Driving is the New “Down Time”

American drivers engage in a variety of distracting behaviors, like chatting on a cell phone, sending text messages, e-mailing friends, selecting songs on iPods, applying makeup, changing clothes and reading.

Eating and talking on a cell phone are by far the most common activities (42% eat and 40% chat on cell phones).

Younger drivers aged 18-24 who are accustomed to always being “plugged in” have the most mentions for every distracting situation while driving:• Eat – 62%• Talk on a cell phone – 71%• Send text messages – 24%• Select songs on an iPod – 20%• Apply makeup – 8%• Change clothes – 8%• Read – 4%• Send e-mails – 1%



■  How can cognitive models help? –  contribute to the scientific knowledge base

•  generally: multitasking behavior •  specifically: driver behavior under distraction •  more specifically: particular devices

–  serve as core simulation engines for computational design tools

•  e.g., rapid prototyping & evaluation for new devices •  e.g., inferring and tracking driver intentions


Driving as a Single Task

■  Two-level steering with near & far points

€

˙ ϕ steer = k f ˙ θ far + kn ˙ θ near + kIθnear



■  Corrective steering validation (Hildreth et al., 2000)

–  Vehicle drifts off course (driver couldn’t control) –  Then driver corrects the vehicle’s course



■  Corrective steering –  varying initial vehicle angle (measured from

straight down the road)



■  Corrective steering –  varying initial vehicle speed



■  ACT-R model procedural steps



■  Lane change: Switch, start à end lane



Drive

Control

Perceive control variables

Update steering, acceleration/braking

Monitor

Note other car in random lane

With probability Pmonitor

Decide

Look at or recall car in other lane in front

Look at or recall car in other lane in back

Decide to change lanes if clear

If lead car is too close





■  Curve negotiation

Human Model

SteeringAngle

LanePosition



■  Lane changing

Human Model

SteeringAngle

LanePosition



■  Human driver...



■  Model driver...


Driving & Dialing

■  Cell-phone use can affect… –  lane keeping: ability to maintain central position –  speed control: ability to maintain safe distance –  general attention: ability to see/react to hazards

■  Driving & dialing a phone –  four dialing conditions

•  full-manual: ON + 7 digits as xxx-xxxx + SEND •  speed-manual: ON + speed number + SEND •  full-voice: ON + say 7 digits + repeat + END •  speed-voice: ON + say phrase + repeat + END

–  interference of manual vs. voice dialing?


Driving & Dialing

■  Model (full-manual dialing)


Driving & Dialing

■  Time to dial a phone number

0

1

2

3

4

5

6

7

8

9

10

Full-Manual Speed-

Manual

Full-Voice Speed-

Voice

Dia

ling T

ime (

s)

Dialing Only

While Driving

0

1

2

3

4

5

6

7

8

9

10

Full-Manual Speed-

Manual

Full-Voice Speed-

Voice

Dia

ling T

ime (

s)

Dialing Only

While Driving

Human Model


.0

.1

.2

.3

DrivingOnly

Full-Manual

Speed-Manual

Full-Voice

Speed-Voice

Late

ral V

eloc

ity (

m/s

)

.0

.1

.2

.3

DrivingOnly

Full-Manual

Speed-Manual

Full-Voice

Speed-Voice

Late

ral V

eloc

ity (

m/s

)

Driving & Dialing

■  Lateral velocity (vehicle stability)

Human Model


Driving & Dialing

■  When exactly do drivers switch tasks? –  more data from a study of 10-digit dialing...


Driving & Dialing

■  When exactly do drivers switch tasks? –  more data from a study of 10-digit dialing...


Driving & Sentence Span

■  What about if the secondary task has no resource conflicts with the primary task? –  driving: steering & braking in response to lead car –  secondary task: “sentence span”

•  hear sentences of form “X does Y” –  e.g., “The boy brushed his teeth” or “The train bought a newspaper”

•  judge each sentence for sensibility •  memorize last word of 5 sentences, then repeat back

■  Model of sentence span borrows components of earlier model (Lovett, Daily, & Reder, 2000)



■  Model



■  Results


Driving & Memory Rehearsal

■  Eliminating visual and/or motor demands does not eliminate effects on driving! –  e.g., conversation can affect performance –  but such tasks still use aural/vocal resources

■  What about a “purely” cognitive task?: Driving & memory rehearsal –  memory task: encode, rehearse, recall 5/9 digits –  driving task: drive for 20 sec

•  drive during the rehearsal phase only!

–  braking task: brake when lead car brakes



■  Experiment structure



■  Model



■  Lateral deviation



■  Brake response time


Other Distractions

■  iPod distraction –  interaction with an iPod is also (surprise)

distracting •  using an iPod is at least as distracting as dialing a cell

phone •  using an iPod is even more distracting than watching a

video!!

–  how could we model iPod use? •  this is more difficult than cell-phone dialing because of

the different perceptual-motor requirements •  e.g., how would you model a thumb wheel?

how would you model the vision needed for scrolling? •  we don’t really know this yet


Other Distractions

■  Distractions were recently discussed at a DOT Distracted Driving Summit in 2009

■  Here’s Tom Dingus (Virginia Tech)...


CS 630: Cognitive Systems. Dario Salvucci, Drexel University. 49 CS 630: Cognitive Systems. Dario Salvucci, Drexel University. 50

The Engineering Side of Distraction

■  Engineering questions on our plate… –  how can we design less distracting devices? –  how can we evaluate devices w.r.t. distraction?

■  One attempt: Distract-R –  Idea to Evaluation in four steps…

•  (1) Prototype Interfaces •  (2) Demonstrate Tasks •  (3) Specify Drivers •  (4) Analyze Results


Distract-R Demo


Env

LK

LC-R

...

...

...

LC-L

LC-L

Another Application: Recognition

■  Instead of prediction, do the complement: recognize behavior –  similar to model

tracing in tutors

■  Focus: Inference of driver intentions


The Big Picture

■  Cognitive architectures are not only fun, they’re useful

■  Many directions to go with this work, theoretically & practically

today s agenda - college of computing & informaticssalvucci/courses/cs630-f14/... · 2014. 11....

Documents