Innovation, Environment and Expertise

Kenneth KotovskyDepartment of Psychology Carnegie Mellon University

Introduction

Goal: to link three somewhat disparate ideas to make an argument about creative invention or innovation, its contexts and development:1) Some research done in collaboration with Jonathan Cagan on the

engineering design process and its environmental context, on the assumption that a deeper process understanding enhances our chances of creating both process and educational interventions to stimulate creative innovation.

2) Some classic findings with regard to the acquisition of expertise. 3) A movement in higher education that can be put in service of the above.

1. Understanding the process: problem solving search within multiple representations

(with Jonathan Cagan)

Basic model applied to individual, group and computational design agent process

Stimulating the Design Loop(with Jarrod Moss & Jonathan Cagan )

What allows environmental input to be assimilated?Typical Experiment

Remote Associate Test (RAT) problems (fox peep man) holeAnswers for both new and previously unsolved problems presented in

intervening task

RATProblems

(20)

RATRepeated + New

(20)

Lexical Decisiontask(50)

Moss, J., K. Kotovsky, and J. Cagan, “The Influence of Open Goals in the Acquisition of Problem Relevant Information”, Journal of Experimental Psychology: Learning, Memory, and Cognition, Vol. 33, No. 5, pp. 876-891, 2007.

Effect of hint on solution time (solved problems)

0.00

5.00

10.00

15.00

20.00

25.00

New Previously Unsolved

Type of problem

So

luti

on

tim

e i

n s

ec

on

ds

No Hint

Hint

Experiment 2: Hint Timing

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Impasse No Impasse

Type of Impasse

Pro

port

ion

Cor

rect

No hint

Hint

fixation No fixation

Conclusions

• Hints more effective in presence of open goals.• People carry unsolved problems with them and are

implicitly sensitized to relevant environmental cues.• Timing matters-hint effective after exploration & dev.

of open goal but before fixation (protocol studies).• Method for impacting the design process by delivering

useful hints, perhaps even automatically.

Transfer to Engineering Design(with Ian Tseng Jarrod Moss & Jonathan Cagan)

How do open goals and hint assimilation affect engineering design?

Generate as many time-keeping devices as possible using a list of 14 household objects.

Explored early/late and close/distant hints

Tseng, I., Moss, R. J., Cagan, J. and Kotovsky, K. Overcoming Blocks in Conceptual Design: the Effects of Open Goals and Analogical Similarity on Idea Generation. 2008 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE August 3-6 2008, New York.

Presented Hints

Distantly Related Information - Three Devices

Similar Information - Three Clocks

Effects of Open Goals

Functionally Distinct Designs

0 1 2 3 4 5 6 7 8

Control

Clocks-Before

Devices-Before

Devices-During

• Examples are influential in extending both the range and novelty of solutions, with devices (distant/varied examples) most effective

• Open goals facilitated recognizing analogies in distantly related information

Transfer to Group Design & Tracking Representations

(with Katherine Fu & Jonathan Cagan)

This study examined:

• How engineering design teams converge to a common understanding of a design problem and its solution using LSA to track representation development.

• Effect of good and bad examples on convergence and qualityof produced solutions.

• The problem was to design an automatic peanut shelling device for a poor isolated village.

Fu, K., Cagan, J. & Kotovsky, K. Design Team Convergence: The Influence of Example Solution Quality. Proceedings of the ASME 2009 International Engineering Design Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2009 August 30 - September 2, 2009, San Diego, California, USA

Results: Convergence within Groups and Convergence to Final Design Solution

Poor example produced decreased convergenceGood example condition and control not significantly differentAll increase over sessions: example effect is immediate and lasting

AVERAGE LEVEL OF SEMANTIC CONVERGENCE WITHIN GROUPS;ERROR BARS SHOW ± ONE STANDARD ERROR

AVERAGE LEVEL OF SEMANTIC CONVERGENCE TO FINAL DESIGN SOLUTION DESCRIPTION TEXT;

ERROR BARS SHOW ± ONE STANDARD ERROR

Results: Quality of Final Designs

Good example produced best quality solutions.Poor example produced lower quality solutions.Convergence and quality not totally equivalent.

AVERAGE QUALITY OF FINAL DESIGN SOLUTIONS; ERROR BARS SHOW ± ONE STANDARD ERROR

Emerging Conclusions From Work on Design Process

The design environment or context matters:– Hints, both cs and ucs and other people have an impact– Representations can be tracked and do converge– Timing matters

Representation tracking potentially allows for a fine-grained analysis of multiple issues in creative invention:

-- Brainstorming, -- Team composition effects, -- Expertise effects,-- Monitoring impact of environmental input.

Other work: – Automate the generation of fruitful examples– Develop a model of the control systems by which designers can optimally

control their activity and resist fixation– Turn this developing process knowledge into teachable-learnable skills.

2. Expertise: A long time coming

• Hayes & others: ten year rule for world class expertise in many domains (music composition, music performance, chess, etc).

• Simon & Gilmartin : On the order of 50,000-100,000 memory chunks (for chess expertise).

• Ericsson: targeted deliberate practice (not just amount) leads to expertise.

• Expertise development involves many cognitive skills and functions: memory & strategy (ex. chess, problem-solving), perception (chess) and representation and knowledge (physics, engineering).

Simon H. A. & Gilmartin, K. (1973). A simulation of memory for chess positions. Cognitive Psychology, 5, 29 -46.Chase, W. G., & Ericsson, K. A. (1982) Sill and working memory. In G. H. Bower (Ed.), The psychology of learning and motivation, (Vol. 16). New York: Academic Press. Kotovsky, K., Hayes, J.R. & Simon, H. A. (1985). Why are some problems hard? Evidence from Tower of Hanoi, Cognitive Psychology, 9, 52-76.Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) Categorization and representation of physics problems by experts and novices, Cognitive Science, 5, 121-152.Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In W. G. Chase (Ed.), Visual information processing. New York: Academic Press. Moss, J., K. Kotovsky, and J. Cagan, “Expertise Differences in the Mental Representation of Mechanical Devices in Engineering Design”, Cognitive Science, Vol. 30, No. 1, pp. 65-93, 2006.

Power Law of Practice

• Skill learning often seems to asymptote relatively quickly over training trials.

• In actuality, keeps increasing, with additional increments in performance requiring exponentially more practice.

Time = Practice –b log (Time) = -b log (Practice)

Generating Geometry Proofs

Neves, D. M., & Anderson, J. R. (1981) Knowledge Compilation: Mechanisms for the automatication of cognitive skills. In J. R. Anderson (Ed.) Cognitive Skills and Their Acquisition. Hillsdale, NJ: Erlbaum.

Time to Perform Mental Addition

Crossman, E. R. F. W. (1959). A theory of the acquisition of speed-skill. Ergonomics, 2, 153-166.

3. The Broader Context: Educational Implications

• If expertise requires vast time and results in continuous cognitive improvement in many diverse areas, why not innovative engineering design as well—particularly as we come to better understand the specific processes and skills via empirical study?

• This is somewhat in contrast to short-term focused creativity training courses.

• Focus on in-situ, learning by doing & concomitant training of skill and knowledge often within the milieu of higher education.

The Carnegie Commission Boyer Report-Reinventing Undergraduate Education:

a Blueprint for America’s Research Universities

• Educate through research, making research-based education the standard.• Beginning in the freshman year.• Inquiry-based courses should allow for joint projects & collaborative efforts.• Internships can turn inquiry-based learning into practical experience.• Combining a group of students with a combination of faculty and graduate

students for a semester or a year of study of a single complicated subject or problem.

• Doesn’t focus on the issue of creativity.• Does have a tilt toward universality (whole school approach).

The Carnegie Plan

• An older version of some of the main ideas of the Boyer Report• Generated by a forward thinking former president of my own

university, Robert Doherty, who was president from 1936 to 1950. • The Carnegie Plan provided a well-rounded "liberal/professional"

education in the context of an engineering school.– Emphasis on being able to do things in addition to knowing things – Students taught to apply fundamental knowledge to solve practical problems – Forerunner to today's focus on an interdisciplinary, problem-solving oriented

university curriculum.

• Enhanced by Herbert Simon with focus on science of design and incorporation of meta-cognitive skills.

Example: Engineering Education

Some Carnegie Plan-like engineering curricular examples that in combination can lead toward expertise.

• First year engineering courses: design, not analysis.• Research internships over multiple years.• Senior design and Integrated product design courses • Focus on multiple interdisciplinary perspectives with faculty from three

colleges and diverse students.• Company sponsorship and involvement directed toward real-world

challenges and resultant patents which are often obtained.• Necessary focus on innovation and creative solutions.

(Similar approach in psychology curriculum but issue of creativity)

Conclusions

• The basic argument is that creative invention and innovation is at least in part a skill set that students can acquire expertise in if-• we understand it at a process level, • we are clear about the desired behavior and our goals of teaching it,• Students are trained and motivated to do it via deliberate training,

coaching and sustained (motivated) effort over long periods of time• we all maintain a continued focus on creative solutions rather than

simply skilled performance.

The policy recommendation is that part of what do in this area is to not only foster research on creative processes but also the translation of the results of that research to methods for teaching invention and innovation.

“Mediocrity, It takes a lot less time, and most people won't notice the difference until it's too late."

“Underachievement, The tallest blade of grass is the first to be cut by the lawnmower.”

The Larger Context

Acknowledgements

Lauren BurakowskiSteven BendersDhruv DattaPasha Gill

FundingNSF under grants DMI-

0627894 and BCS0717957

People