Can Students and Faculty Learn to Respond to Challenges Facing Engineering Education?

Jeff FroydDirector of Academic Development

Texas A&M University

Purdue University

6 October 2003

"There is an incredible evolution of learning or education as almost the sole source of competitive advantage in an economy that has changed so much."

Howard Block, Managing Director

Banc of America Securities

An investment-bank and brokerage subsidiary of Bank of America.

Challenges in Engineering Education

Challenges Challenge of lifelong learning Challenge of problem solving Challenge of engineering design Challenge of transfer Challenge of conceptual

understanding

LASSI SCALE ENGR111(Mean)

CVEN349(Mean)

Significance*

Skill Component

Information Processing 60.68 60.29 0.930Test Strategies 64.33 63.27 0.794Selecting Main Ideas 55.18 59.29 0.342Will Component

Anxiety 60.52 67.12 0.147Attitude 42.47 34.56 0.080Motivation 63.30 59.29 0.397Self-regulation Component

Concentration 61.31 54.56 0.144Self-testing 52.47 37.92 0.006Study Aids 60.24 45.04 0.005Time Management 55.23 47.65 0.134

“Although the data suggest a slight upward upward trend, the trend proved not to be statistically significant based upon an analysis of variance (ANOVA). Thus the cross-sectional study did not find evidence of an increase in readiness for self-directed learning, even for students in the later semesters who are taking elective courses and their capstone courses.”

Litzinger, T., Wise, J., Lee, S., and Bjorklund, S. (2003) Assessing Readiness for Self-directed Learning, Proceedings, ASEE Annual Conference

Lifelong Learning at Penn State

Self-Directed Learning Readiness Survey (SDLRS), Guglielmino & Associates, http://www.guglielmino734.com/prod01.htm, March 2003. 27

Challenge of Problem Solving

“Despite individual professors’ dedication and efforts to develop problem solving skill, “general problem solving skill” was not developed in the four years in our undergraduate program. Students graduated showing the same inability that they had when they started the program. Some could not create hypotheses; some misread problem statements. During the four-year undergraduate engineering program studied, 1974-1978, the students had worked over 3000 homework problems, they had observed about 1000 sample solutions being worked on the board by either the teacher or by peers, and they had worked many open-ended problems. In other words, they showed no improvement in problem solving skills despite the best intentions of their instructors.”Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem Solving Program,” Journal of Engineering Education,

Challenge of Problem Solving Ineffective approach #1. Give the students

open-ended problems to solve; This, we now see, is ineffective because the students get little feedback about the process steps, they tend to reinforce bad habits, they do not know what processes they should be using and they resort to trying to collect sample solutions and match past memorized sample solutions to new problem situations.

Challenge of Problem Solving Ineffective approach # 2: Show them how you

solve problems by working many problems on the board and handing out many sample solutions This, we now see, is ineffective because teachers

know too much. Teachers demonstrate "exercise solving". Teachers do not make mistakes; they do not struggle to figure out what the problem really is. They work forwards; not backwards from the goal. They do not demonstrate the "problem solving" process; they demonstrate the "exercise solving" process. If they did demonstrate "problem solving" with all its mistakes and trials, the students would brand the teacher as incompetent. We know; we tried!

Challenge of Problem Solving Ineffective approach #3: Have

students solve problems on the board Different students use different

approaches to solving problems; what works for one won't work for others. When we used this method as a research tool, the students reported "we learned nothing to help us solve problems by watching Jim, Sue and Brad solve those problems!"

Challenge of Problem Solving Through four research projects we identified

why and how these and other teaching methods failed to develop process skills and which methods were successful in developing the skills

Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching Problem Solving Skills," Annals of Engineering Education, 1, 1, 238-243.

Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching Problem Solving" CHEM 13 News no. 155 (1985).

Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science Teaching, 22, 312-314.

Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College Science Teaching, 23, 57-58.

Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J. College Science Teaching, 23, 157-158.

Challenge of Engineering Design

The literature is filled with positive comments from students, instructors, and industrial sponsors who have participated in capstone design courses. The vast majority of participants feel that the course benefited all involved.The nature of capstone design courses, however, often leads to a purely subjective evaluation with little or no “hard evidence” of actual benefits. Born, for example, does not attempt to prove the value of senior level design courses. He simply states that he is convinced from his experiences that such courses are valuable. Other educators have similar “feelings” as to the relative costs and benefits of capstone design courses.

Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education

Challenge of TransferResearches posed this problem to people

"Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the health tissue?"

Challenge of TransferConsider the following story

"A small country was ruled from a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely, since the dictator need to move his troops and workers to and from the fortress. However, any large force would detonate the mines. Not only would this blow up the road, but it would also destroy many neighboring villages. It therefore seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his army into small groups and dispatched each group to the head of a different road. When all was ready he gave the signal and each group marched down a different road. Each group continued down it road to the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator."

Challenge of Transfer After the subjects read and summarized this

story, they were asked to solve the tumor problem under the guise of a separate experiment.

Given the clear analogy, you might think that performance would be near ceiling. Surprisingly, only 30% of the subjects offered a convergence solution.

Moreover, when these same subjects were given the suggestion that they should use the General story, 80% provided a convergence solution.

This finding demonstrates that half the subjects could apply the General story to the tumor problem when they were instructed to but did not do so on their own.

Challenge of Conceptual Understanding Force Concept Inventory

Signals and Systems Concept Inventory

Heat Transfer Concept Inventory

Materials Concept Inventory

Hestenes Force Concepts Inventory ResultsHistogram of Number of Courses vs.

Their Normalized Gain ( )N

um

ber

of

Co

urs

es(6

2 C

ou

rses

En

rolli

ng

654

2 S

tud

ents

)

Interactive EngagementInteractive EngagementTraditionalEngagementTraditional

Engagement

Bin Centers for Normalized Gain (%)From: R. Hake, Am. J. Phys. 66, 64-74 (1998),

or see hitchcock.dlt.asu.edu/media2/cresmet/hake/

100%0%Pretest Posttest<g>

<p>

<g><p>

x 100

FIPE‘98-99

FIPE‘96-97,‘97-98 CC class,

Spring ‘99

FIPE‘99-00

FIPE‘99-00

Hestenes, Wells and Swackhamer, The Physics Teacher 30, 141 (1992); Revised edition available at http://modeling.la.asu.edu/R&E/Research.html

One of the Questions from the Force Concept Inventory

Hestenes, Wells and Swackhamer, The Physics Teacher 30, 141 (1992); Revised edition available at http://modeling.la.asu.edu/R&E/Research.html

Signals and Systems Concept Inventory (SSCI) Continuous-time SSCI Discrete-time SSCI Analysis of the 174 CT-SSCI exams revealed a

normalized gain between pre- and post-test scores of 0.22 +/- 0.07 which is consistent with the results Hake reported for other concept inventory studies of traditional lecture courses.

http://ece.gmu.edu/~kwage/research/ssci/

Heat Transfer Concept Inventory

Another student was confident of “energy balances!” but asked “Should I understand the derivation of the heat diffusion equation and where each term comes from?” The heat diffusion equation is an energy balance equation.

One topic that is typically covered in depth over a one- to two-week period in a heat transfer is the use of fins to increase heat transfer. All of the students expressed uncertainty about this subject.

They appear unable to connect to what they know about thermodynamics (i.e. heat cannot be transferred without a temperature difference) and apply it to the problem at hand (two surfaces at the same temperature).

Materials Concept Inventory Jointly developed between Arizona State

and Texas A&M Preliminary results show normalized

gain of about 0.2 in lecture classes and 0.4 in a class with some active learning

Both developers are disappointed in the pre- to post-gains shown by their classes and are actively revising their approaches to teaching

C I Questions - MCI Example

b*

3 spoonfuls of salt are stirred into a glass of water, leaving a spoonful of water-saturated salt on the bottom. If a small amount of salt is stirred in, the change in concentration of the salt in the solution is given by curve: _______

Pre-test - ASU 39%, TAMU 39%; Post-test -TAMU 66%

Critical concept - Solubility limits in phase diagrams

Student-Generated Distracters

After a piece of copper wire from a hardware store is heated it becomes softer. This is because:

a. The bonds have been weakenedb. It has fewer atomic level defects * c. It has more atomic level defectsd. The density is lowere. There’s more space inside crystal lattice

Student-generated distracters are underlinedCritical concept is role of defects in metal strength and in

metal processing.

a 39% 18%

*b 8% 29%c 6% 21%d 9% 4%e 48% 28%

Pre Post

C I Questions - MCI Example

A “Favorite” Misconception from the MCI

Nickel, and probably most other metals, cannot exist as a gas, but only as a solid or a liquid.

Nickel can exist as: ____a) solid onlyb) liquid onlyc) gas onlyd) liquid or solid onlye) liquid or solid or gas *

ASU Pre

TAMU Pre

TAMU Post

a 5% 6% 2%

b 0% 0.4% 0%

c 0% 0.4% 0%

d 46% 57% 43%

e 49% 36% 53%

Change in Engineering Education Initiating and Sustaining Change

Resistance

Leadership

Culture

Learning, Assessment, Evaluation

Initiating and Sustaining Change “It matters to me.”

Personal mastery and learning “It matters to a network of my peers.”

Knowledge management Communities of practice

“It matters to constituents and stakeholders.” Faculty and curriculum development Technology transfer and knowledge

management

Resistance Resistance is inevitable and

comprehensible Responses to resistance

Ignore Steamroll Understand and address Anticipate, understand, and address

Possible Sources of Resistance to Change

“Whatever changes we make, we must be sure to teach the fundamentals!”

Possible Sources of Resistance to Change

“Why do we need to change? Our graduates are being hired.”

Leadership Leadership takes place every day. It

cannot be the responsibility of the few, a rare event, or a once-in-a-lifetime opportunity.”

Heifetz, Ronald and Donald Laurie, “The Work of Leadership,”

Leadership is too important to be left in the hands of the few people near the top of the organizational hierarchy.

Leadership for Change Change is hard work. Leadership begins

with values Intellectual leads

physical Real changes takes

real change Leadership is a team

sport Expect to be surprised

Today competes with tomorrow

Better is better Focus on the future Learning from doing Grow people Reflect

Sullivan and Harper, Hope is not a Method

What is the organizational culture?

The culture of a group is a pattern of shared basic assumptions that the group learned as it solved problems … that has worked well enough to be considered valid, and therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems.

Artifacts

Visible structures and processes

Espoused Values

Strategies, goals, philosophies

Basic Underlying Assumptions

Unconscious beliefs, perceptions, thoughts, feelings

Changing Culture

“You cannot create a new culture. You can immerse yourself in studying a culture ... Until you understand it. Then you can propose new values, introduce new ways of doing things, and articulate new governing ideas. Over time, these actions will set the stage for new behavior. If people who adopt the new behavior feel that it helps them ... The organizational culture may embody a different set of assumptions, and a different way of looking at things ...”

Edgar Schein, in Senge, Peter, The Dance of Change

Changing Culture

“… culture is the result of all the daily conversations and negotiations between the members of an organization. They are continually agreeing (sometimes explicitly, usually tacitly) about the ‘proper’ way to do things and how to make meanings about the events of the world around them. If you want to change a culture you have to change all these conversations—or at least the majority of them.”

(Richard Seel, 2000)

Questions?

References Fowler, D., Maxwell, D., Froyd, J. (2003) Learning Strategy Growth Not What Expected After

Two Years through Engineering Curriculum, Proceedings, ASEE Annual ConferenceAbstract As the pace of technological development continues to increase, consensus has emerged that undergraduate science, technology, engineering and mathematics (STEM) curricula cannot contain all of the topics that engineering professionals will require, even during the first ten years of their careers. Therefore, the need for students to increase their capability for lifelong learning is receiving greater attention. It is anticipated that development of this capability occurs during the undergraduate curricula. However, preliminary data from both first-year and junior engineering majors may indicate that development of these competencies may not be as large as desired. Data was obtained using the Learning and Study Skills Inventory (LASSI), an instrument whose reliability has been demonstrated during the past fifteen years. The LASSI is a ten-scale, eighty-item assessment of students’ awareness about and use of learning and study strategies related to skill, will and self-regulation components of strategic learning. Students at Texas A&M University in both a first-year engineering course and a junior level civil engineering course took the LASSI at the beginning of the academic year. Improvements would normally be expected after two years in a challenging engineering curriculum. However, data on several different scales appears to indicate that improvements are smaller than might be expected.

ReferencesLitzinger, T., Wise, J., Lee, S., Bjorklund, S. (2003) Assessing Readiness for Self-

directed Learning, Proceedings, ASEE Annual ConferenceConclusions The goals of this study were to determine whether students’ readiness for self-directed learning increases as they proceed through an undergraduate engineering program and to determine whether capstone courses increase their readiness for self-directed learning. A related goal was to determine if there are any gender differences in readiness for self-directed learning. The cross-sectional study showed no statistically significant increase in SDLRS scores. This result suggests that most courses that students take in the undergraduate engineering programs do not ask them to undertake tasks that increase their readiness for self-directed learning. No gender differences were present in the cross-sectional study. Although some of the data show an increase in SDLRS scores between a pre-test and post-test in capstone courses in Mechanical and Electrical Engineering, the results failed to support the hypothesis that the complex problem solving required in these courses would lead to increases in readiness for self-directed learning. The lack of a statistically significant increase in SDLRS scores is of concern in an era when lifelong learning is more critical than ever for engineers. The results of the present study will be extended in future work to increase the sample size in the pre-test/post-test study and to document carefully the type of activities that the capstone courses require the students to undertake

ReferencesWoods, D. et al (1997) “Developing Problem Solving Skills: The McMaster Problem

Solving Program,” Journal of Engineering Education, Abstract This paper describes a 25-year project in which we defined problem solving, identified effective methods for developing students’ skill in problem solving, implemented a series of four required courses to develop the skill, and evaluated the effectiveness of the program. Four research projects are summarized in which we identified which teaching methods failed to develop problem solving skill and which methods were successful in developing the skills. We found that students need both comprehension of Chemical Engineering and what we call general problem solving skill to solve problems successfully. We identified 37 general problem solving skills. We use 120 hours of workshops spread over four required courses to develop the skills. Each skill is built (using content-independent activities), bridged (to apply the skill in the content-specific domain of Chemical Engineering) and extended (to use the skill in other contexts and contents and in everyday life). The tests and examinations of process skills, TEPS, that assess the degree to which the students can apply the skills are described. We illustrate how self-assessment was used.

ReferencesDutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of

Literature on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of Engineering Education

Abstract Senior project or Capstone-type courses have existed at engineering schools for many years. Capstone courses provide student engineers the opportunity to solve real-world engineering projects, and have been highly regarded as important learning activities. A survey of Capstone courses in engineering departments throughout North America was conducted in order to understand current practices in Capstone education. This study was conducted for presentation at the 1994 Advances in Capstone Education Conference, held at Brigham Young University, which brought together engineering educators interested in improving Capstone experiences. This conference was sponsored by ASEE, ASME, SME, and NSF. Response to the study was very high, with 360 departments from 173 schools responding to the survey. Survey results were categorized into five major areas of interest: Profile of the Respondents, Course Description Information, Faculty Involvement in Capstone Education, Project Information, and Industrial Involvement in Capstone Education. Graphs provide the results of responses to survey questions.

ReferencesMedin, D., Ross, B., Markman, A. (2001)

Cognitive Psychology, third edition, Fort Worth: Harcourt College Publishers, 470-471

Gick, M., Holyoak, K. (1980) Analogical problem solving. Cognitive Psychology, 12, 306-355

Gick, M., Holyoak, K. (1983) Schema induction and analogical transfer. Cognitive Psychology, 15, 1-38

ReferencesHake, R.R. (1988) "Interactive-engagement vs traditional methods: A six-thousand-

student survey of mechanics test data for introductory physics courses," Am. J. Phys. 66, 64- 74, http://www.physics.indiana.edu/~sdi/ajpv3i.pdfAbstract A survey of pre/post test data using the Halloun-Hestenes Mechanics Diagnostic test or more recent Force Concept Inventory is reported for 62 introductory physics courses enrolling a total number of students N = 6542. A consistent analysis over diverse student populations in high schools, colleges, and universities is obtained if a rough measure of the average effectiveness of a course in promoting conceptual understanding is taken to be the average normalized gain <g>. The latter is defined as the ratio of the actual average gain (%<post> – %<pre>) to the maximum possible average gain (100 – %<pre>). Fourteen "traditional" (T) courses (N = 2084) which made little or no use of interactive-engagement (IE) methods achieved an average gain <g>T-ave = 0.23 ± 0.04 std dev). In sharp contrast, forty-eight courses (N = 4458) which made substantial use of IE methods achieved an average gain <g>IE-ave = 0.48 ± 0.14 (std dev), almost two standard deviations of <g>IE-ave above that of the traditional courses. Results for 30 N = 3259) of the above 62 courses on the problem-solving Mechanics Baseline test of Hestenes-Wells imply that IE strategies enhance problem-solving ability. The conceptual and problem-solving test results strongly suggest that the classroom use of IE methods can increase mechanics-course effectiveness well beyond that obtained in traditional practice.

References Halloun, I., Hestenes, D. (1985)

The Initial Knowledge State of College Physics Students, Am. J. Phys. 53, 1043-1055

Halloun, I., Hestenes, D. (1985) Common Sense Concepts about Motion, Am. J. Phys. 53, 1056-1065

More information about the Force Concept Inventory is available at http://modeling.la.asu.edu/R&E/Research.html

ReferencesWage, K., Buck, J., Welch, T., Wright, C. (2002) The Signals and Systems Concept

Inventory, Proceedings, ASEE Annual ConferenceAbstract This paper describes the development of continuous-time and discrete-time signals and systems concept inventory exams for undergraduate electrical engineering curricula. Both exams have twenty-five multiple choice questions to assess students’ understanding of core concepts in these courses. The questions require little or no computation, and contain incorrect answers that capture common student misconceptions. The design of both exams is discussed, as are ongoing studies evaluating the exams at four campuses. Preliminary results from administering the continuous-time exam as a pre-test and post-test indicate a normalized gain of 0.24 ± 0.08 for traditional lecture courses, consistent with reported results for the Force Concept Inventory exam in lecture courses for freshman physics.

ReferencesJacobi A., Martin, J. Mitchell, J., Newell, T. (2003) A Concept Inventory for Heat

Transfer Proceedings, Frontiers in Education Conference, http://fie.engrng.pitt.edu/fie2003/papers/1192.pdfAbstract Students enter courses in engineering with intuitions about physical phenomena. Through coursework they build on their intuition to develop a set of beliefs about the subject. Often, their understanding of basic concepts is incomplete and their explanations are not “correct.” Concept Inventories are assessment tools designed to determine the degree to which students understand the concepts of a subject and to identify the bases for misunderstandings. A cooperative effort between faculty at the Universities of Wisconsin and Illinois has been undertaken to develop a concept inventory for heat transfer. The process initiated with student identification of the conceptual problems rather than with faculty perceptions of student misunderstandings. Students then explored areas of conceptual difficulty and phrased questions that would test understanding of the concepts. Students working together with faculty developed a concept inventory for heat transfer. The presentation will report on the experience with using student groups and the resulting concept inventory.

ReferencesKrause, S., Decker, J., Griffin, R. (2003) Using a Materials Concept Inventory to

Assess Conceptual Gain in Introductory Materials Engineering Courses, Proceedings, Frontiers in Education Conference, http://fie.engrng.pitt.edu/fie2003/papers/1160.pdfAbstract A Materials Concept Inventory (MCI) has been created to measure conceptual knowledge gain in introductory materials engineering courses. The 30-question, multiple-choice MCI test has been administered as a pre and post-test at Arizona State University (ASU) and Texas A & M University (TAMU) to classes ranging in size from 16 to 90 students. The results on the pre-test (entering class) showed both “prior misconceptions” and knowledge gaps that resulted from earlier coursework in chemistry and, to a lesser extent, in geometry. The post-test (exiting class) showed both that some “prior misconceptions” persisted and also that new “spontaneous misconceptions” had been created during the course of the class. Most classes showed a limited, 15% to 20%, gain in knowledge between pre and post-test scores, but one class, which used active learning, showed a gain of 38%. More details on these results, on differences in results between ASU and TAMU, and on the nature of students’ conceptual knowledge will be described.

References Senge, P., Kleiner, A., Roberts, C.,

Roth, G., Ross, R., Smith, B. (1999) The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations, Double Day

References Mauer, R. (1996) Beyond the Wall of

Resistance: Unconventional Strategies That Build Support for Change, Bard Press, see also http://www.beyondresistance.com/index.html

References Heifetz, R., Laurie, D. (1997). The work of

leadership. Harvard Business Review, 75(1), 124-134.

References Sullivan, G., Harper, M. (1996)

Hope is Not a Method, New York: Random House

References

Seel, R. (2000) “Culture and Complexity: New Insights on Organisational Change,” Organisations & People, 7(2) 2-9, see also http://www.new-paradigm.co.uk/culture-complex.htm

McMaster Problem-Solving Skill Sets Self-awareness, self confidence, self-

management Personal problem solving

Well-defined problems Ill-defined problems

Interpersonal and group skills Self-assessment Change management Lifelong learning skills

MPS Self-awareness, Self-confidence, Self-management Skills

Stress management Time management The unique you Managing anger Awareness Self assessment Personal enrichment

MPS Well-defined Problem Solving Skills

Awareness Strategies Analysis

(classification) Analysis (consistency) Creativity Drawing diagrams Engaging in the

problem Defining the stated

problem Getting unstuck

Broadening perspectives

Defining real problems Creating the look back Exploring the problem Tactics and heuristics Criteria Decision-making Learning skills Stress management Time management The unique you

MPS Ill-defined Problem Solving Skills Mission Vision and goals Trouble shooting Coping with ambiguity Optimum sloppiness and successive

approximation Project management

MPS Interpersonal and Group Skills

Assertiveness Asking questions Listening Interpersonal skills Group skills Chairperson skills

Conflict resolution Giving and receiving

feedback The unique you Managing anger Stress management

MPS Lifelong Learning Skills Knowledge structure Tacit or experience knowledge Learning skills Self-directed learning/PBL The unique you

MPS Change Management Skills Managing change Stress management Managing anger

MPS Self-assessment Skills Self assessment Personal enrichment

Attributes of Effective Leaders Inner drive/energy: necessary to initiate

and sustain leadership of change over extended periods of time.

Intellectual capacity: necessary to listen to input from diverse sources and synthesize vision and strategy

Integrity: necessary to synthesize vision and strategy that benefits the organization first and the individual second

Mental/emotion health: necessary for self-confidence and interpersonal skills

Kotter, John P., A Force for Change: How Leadership Differs From Management,