mentorship and performance assessment of design … · 1 kaizen is a japanese wo rd f or cont...

MENTORSHIP AND PERFORMANCE ASSESSMENT OF DESIGN TEAMS IN

TRANSPORTATION-RELATED PROJECTS

FINAL REPORT

DECEMBER 2005

Budget Number KLK314

N05-04

Prepared for

OFFICE OF UNIVERSITY RESEARCH AND EDUCATION

U.S. DEPARTMENT OF TRANSPORTATION

Prepared by

NATIONAL INSTITUTE FOR ADVANCED TRANSPORTATION TECHNOLOGY

UNIVERSITY OF IDAHO

Edwin Odom; Steve Beyerlein

DISCLAIMER

The contents of this report reflect the views of the authors,

who are responsible for the facts and the accuracy of the

information presented herein. This document is disseminated

under the sponsorship of the Department of Transportation,

University Transportation Centers Program, in the interest of

information exchange. The U.S. Government assumes no

liability for the contents or use thereof.

1. Report No.

2. Government Accession No.

3. Recipient’s Catalog No.

4. Title and Subtitle

Mentorship and Performance Assessment of Design Teams in

Transportation Related Projects

5. Report Date

December 2005

6. Performing Organization Code

KLK314

Author(s) Edwin Odom; Steven Beyerlein

8. Performing Organization Report No.

N05-04

9. Performing Organization Name and Address

National Institute for Advanced Transportation Technology

University of Idaho

10. Work Unit No. (TRAIS)

PO Box 440901; 115 Engineering Physics Building

Moscow, ID 838440901

11. Contract or Grant No.

DTRS98-G-0027

12. Sponsoring Agency Name and Address

US Department of Transportation

Research and Special Programs Administration

13. Type of Report and Period Covered

Final Report: September 2002-

August 2003

400 7th Street SW

Washington, DC 20509-0001

14. Sponsoring Agency Code

USDOT/RSPA/DIR-1

Supplementary Notes:

16. Abstract

A new type of design laboratory known as Mindworks© has been created to support just-in-time learning about machine design and local manufacturing capabilities that are frequently used in capstone design projects. This laboratory contains design artifacts, quick references, manufacturing videos, and mathematical models for solving common machine design problems. This unique space for learning and practicing design is documented extensively at www.webs1.uidaho.edu/ele/mindworks. Mindworks resources have evolved through project work in a required Machine Design class as well as elective classes in Lean Manufacturing and Advanced Machine Design. Mindworks is used extensively for drivetrain and engine design by students engaged in transportation related senior design projects.

17. Key Words

Manufacturing; education and training; -personnel

performance; production.

18. Distribution Statement

Unrestricted; Document is available to the public through the National

Technical Information Service; Springfield, VT.

19. Security Classif. (of this report)

Unclassified

20. Security Classif. (of this page)

Unclassified

21. No. of Pages

19

22. Price

…

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

MENTORSHIP AND

PERFORMANCE ASSESSMENT

OF DESIGN TEAMS IN

TRANSPORTATION-RELATED PROJECTS

Authors: Edwin Odom; Steven Beyerlein

TABLE OF CONTENTS INTRODUCTION........................................................................................................................ 1

APPROACH AND METHODOLOGY....................................................................................... 2

FINDINGS; CONCLUSIONS; RECOMMENDATIONS........................................................... 6

APPENDIX .................................................................................................................................. 8

Mentorship and Performance Assessment of i Design Teams in Transportation Related Projects

INTRODUCTION

Idaho Engineering Works (IEW) at the University of Idaho is formed of a diverse group of

graduate students whose purpose is to develop an environment that fosters professional and

technical excellence by mentoring year-long capstone design projects. Each year a well-trained,

collaborative, and highly-reflective cohort of graduate students is formed to support design

education. This team is developed through directed study courses, team projects, personal

reflections and monumental technical and interpersonal challenges. Over the last ten years, the

IEW has been successful in facilitating delivery of hardware that exceeds expectations of

industry customers, shortening time required for large-scale design projects, and developing

new manufacturing capabilities within the Mechanical Engineering Department. Many of the

projects are vehicle related and are supported by NIATT as well as the College of Engineering.

A central role of IEW members is to provide guidance to the senior design teams enrolled in

Capstone Design. Those involved in the UI Capstone Design must clearly identify the goals of

the project, make design decisions based on analysis, and provide a functioning hardware

prototype compliant with design specifications. Experience has shown that ME seniors often

need substantial shop training in order to complete their projects. This often leads to extra

fabrication time, reducing the availability of shop facilities for other design teams. IEW

mentors provide machine shop training as well as assistance with client communication, project

management, and conflict resolution.

The demanding task of mentoring senior design teams takes leadership, management, and

proficiency with state-of-the-art tools. In order to better educate mentors in these skills, a

semester long graduate seminar was created. This seminar also allows IEW members to share

and reflect on actions taken with individual teams to find collective solutions for common

problems. The NIATT project described here grew out of the success of previous IEW

seminars and the desire to expand our legacy of design resources, hardware solutions, and

training experiences that are handed down from one generation of students to the next.

Mentorship and Performance Assessment of 1 Design Teams in Transportation Related Projects

APPROACH AND METHODOLOGY

In today’s world of rapidly changing transportation systems, state-of-the-art approaches to

design and just-in-time methods for learning relevant tools, techniques, and technologies are in

great demand. For many organizations, especially universities, this problem is accentuated by a

large annual turnover of those who participate in research and development. An approach to

knowledge transfer that integrates physical, virtual, and human elements is likely to be most

effective with a broad spectrum of learner/practitioners. This begins with programs for training

mentors in new technology areas. It can be enriched through special environments where

knowledge resides in an easily accessible format. Finally, instructional and decision-making

systems need to undergo continuous refinement based on real-time performance measurement.

The objective of this project was to develop resources that support each of these areas. These

include: (a) design and implementation of an intensive, three-week lean manufacturing short

course for graduate mentors as well as a subset of the upcoming senior class, (b) establishment

of a Mindworks© laboratory for just-in-time machine design and manufacturing learning, and

(c) a capstone design assessment system built around key performance rubrics.

Our latest design for the three-week lean manufacturing short course is included in the

Appendix. The course stresses the following lean concepts: value stream mapping, visual

workplace design, 5S, batch size reduction, standardized work, quality at the source, cellular

flow, pull/Kanban (an orderly and efficient flow of materials throughout the entire

manufacturing process), design failure mode effects analysis, and continuous improvement.

The course also stresses the following shop skills: design for manufacturability, shop safety,

efficient use of resources, cleanliness, location of tools, drawing package formulation, proper

tolerancing, lathe operation, milling machine operation, and welding.


We developed an annual Kaizen event1 where teams of two to three students identify a specific

need for elevated design/fabrication practice by engineering students, create a resource to

advance this practice, and receive peer review prior to its finalization. The final project features

multiple manufacturing approaches to the same design problem.

This past year, student teams explored three fundamentally different approaches to A-arm

design: welded members, machined billet, and composite members. Teams developed detailed

drawing packages, manufacturing plans, manufacturing fixtures, cost analysis, and a system-

level DFMEA. A Projects’ Day at the end of course allowed teams to celebrate their

accomplishments in the CAD lab/shop and critique work products by other teams who

employed similar as well as different methods.

Successful engineering project work begins with a thorough understanding of client needs,

however work can be compromised if it is not followed by thoughtful detailed design. High

quality detail design work involves integration of several practitioner roles: analyst, problem

solver, achiever, and self-grower. Because of the varied nature of problems encountered in

different project environments, time constraints, and quality expectations placed on these

projects, practitioners need just-in-time resources to accelerate project learning and to promote

better project decision-making.

Machine design and modern manufacturing processes were selected as areas where just-in-time

resources could have great leverage. Physical artifacts, posters, web pages, and videos were

created and assembled in a new laboratory environment known as Mindworks for this purpose.

1 Kaizen is a Japanese word for continuous improvement—it is a method that strives toward perfection by

eliminating waste. Kaizen Events are commonly referred to as a tool that:

1) Gathers operators, managers, and owners of a process in one place

2) Maps the existing process (using a deployment flowchart, in most cases)

3) Improves on the existing process

4) Solicits buy-in from all parties related to the process

[http://www.isizsigma.com/dictionary; accessed 12/7/05]


Junior users enrolled in machine design and various elective courses were engaged in

researching and organizing specific contributions to Mindworks. This approach insured that the

resources were attractive and useful to the intended audience of apprentice engineers.

Project work in capstone design courses can be visualized along a project axis as well as a

learning axis as shown in the figure below. The project axis extends from thoughtful definition

of design requirements in conjunction with project stakeholders to detailed solution concepts

that can be manufactured efficiently and tested against key specifications. The learning axis

includes individual competence and continued learning as well as team processes that ensure

positive team dynamics and productivity.

Four assessment tools were created to help instructors and mentors track project development

and design learning in an environment where multiple student projects are underway at the

same time. These support data collection for program accreditation as well as formative

assessment. Copies appear in the Appendix.


A design review form was created to provide feedback on project conceptualization and

definition of requirements. Used in conjunction with Axiomatic Design methods adapted for a

capstone course context, design decisions and their implications for detailed design can be

communicated more clearly to all stakeholders. A design report form provides feedback on

project documentation that allows third-party engineers to replicate the solution and efficiently

continue development work. A design team performance rubric measures individual

engagement in team activities and project contribution. A logbook assessment form was created

to provide feedback on personal competence and professionalism.


FINDINGS; CONCLUSIONS; RECOMMENDATIONS

Shop mini-projects, curriculum materials, training activities related to our lean manufacturing

short course have been significantly upgraded through this work. This short course integrates

principles of lean manufacturing with use of CAD, metrology, milling machines, lathes, and

welding equipment in a prototyping environment. Part of the course involves a day-long factory

simulation workshop. Based on end-of-course surveys, significant professional growth was

noted in knowledge of lean concepts and modern manufacturing practices.

A new type of design laboratory, known as Mindworks, was created through this work. This

laboratory contains design artifacts, quick references, manufacturing videos, and mathematical

models for solving common machine design problems. Mindworks continues to evolve through

project work in a required Machine Design class and has been used extensively by nearly all

capstone design teams in the last two years. This unique space for learning and practicing

design is documented extensively at www.webs1.uidaho.edu/mindworks.

Since the initiation of this project, capstone design products have continued to evolve in

complexity, mechanical and electrical engineering departments have implemented an

interdisciplinary course structure, and the average level of industry funding has steadily

increased. A complete archive of projects is maintained at http://seniordesign.engr.uidaho.edu.

The design infrastructure spawned by the lean manufacturing course and the Mindworks

laboratory have been instrumental in bringing about these changes. Transportation projects that

have directly benefited from efforts associated with work include:

Ultracapacitor System for FutureTruck vehicle

Hybrid Electric Drivetrain for Future ruck vehicle

Airbag Pleating and Folding Station for Autoliv

Airbag Initiator Teststand for Takata

Frame Optimization for Formula SAE vehicle

Gear Drive for Formula SAE vehicle


http://www.webs1.uidaho.edu/mindworks

http://seniordesign.engr.uidaho.edu/

Composite Body for Formula SAE vehicle

Tuned Intake System for Formula SAE vehicle

Direct Injection 2-Stroke Engine for Clean Snowmobile

Direct Drive System for Clean Snowmobile

Exhaust System for Clean Snowmobile

This work contributed to four refereed pedagogy papers. These were published over the last

several years in Proceedings of the Annual Meeting of the American Society for Engineering

Education. These include: Role of Axiomatic Design in Teaching Capstone Courses (2005),

Measuring Added-Value Using a Team Design Skills Growth Survey (2005), Development and

Use of an Engineering Profile (2005), and Capstone Design Course and Assessment of ABET

EC 2000: A National Survey (2004).

Three successful donation requests/proposals were stimulated by this work. The first was a

generous cash donation from Schweitzer Engineering that resulted in a four-axis, Haas CNC

Mill along with related tooling valued at $50,000. The second was a new Haas CNC lathe and

related tooling valued at $30,000. The third was a $50,000 grant from the National Science

Foundation to support University of Idaho collaboration with Washington State University,

Seattle University and Tuskegee University in the development of transferable assessment and

evaluation instruments for capstone design courses.


APPENDIX

Lean Manufacturing Short-Course Syllabus (Summer 2005)

Course Goal: Generate working understanding of Lean Manufacturing terms and practices.

Develop fundamental capabilities in ME machine shop revolving around simple prototypes.

Week 1: Introduction to Lean Thinking and ME shop

Day Lecture Lab HW

5/16 Course Overview

Traditional Manufacturing

Shop safety tour

Shop orientation

Mindworks tour

Read Joe’s Garage

5/17 Lean Manufacturing

Lean Terminology

Lean Tooling

Metrology Equipment

Part Inspection

Drawing Standards

Dimensions/Tolerances

Mill Project Drawings

Define vocabulary terms

in Joe’s Garage

Identify five aspects of ME

design suite that you can

relate to lean manufacturing

(1/2 page on each)

5/18 Kaizen Events

Kaizen project assignments

Group A: Mill Project

Group B: Lathe Project

Read and take notes on

Autoliv pamphlet

Identify resources for

Kaizen projects

5/19 Homework Discussion

Value Stream Mapping

Group A: Mill Project

Group B: Welding

Evaluate Commons Food

Court from lean perspective

5/20 8:00-5:00 TechHelp Lean

Manufacturing Workshop

8:00-5:00 TechHelp Lean

Manufacturing Workshop

Describe five lean concepts

reinforced by workshop and

five new concepts that were

introduced


Week 2: Apply lean concepts in shop, CAD lab, and kaizen projects

Day Lecture Lab Homework

5/23 Homework Discussion

Managing Quality

Logbook Review

Group A: Welding

Group B: Mill Project

Kaizen project

5/24 Review of Kaizen Projects Group A: Lathe Project

Group B: Mill Project

Kaizen project

5/25 Six Sigma Optional exercises:

Rapid Prototyping

CNC Threading

MasterCAM Tutorial

Tramming a Head

Kaizen Project

5/26 7:30-5:30 Plant Tour

Advanced Input Systems

Bayshore Systems

7:30-5:30 Plant Tour

Advanced Input Systems

Tour Questions

5/27 Discuss Plant Tours

Design for

Manufacturability

Kaizen Projects Due

A-Arm Project Kick-Off

Composite Design

Bolted Design

Welded Design

Machining from Billet

A-Arm Project

Drawing Package

Manufacturing Plan

Hardware Prototype

Cost Report

DFMEA


Week 3: Mini-Project Competition and Documentation

Day Lecture Lab Homework

5/31 Cost Estimating

Manufacturing Plans

A-Arm Project A-Arm Project

6/1 DFMEA A-Arm Project A-Arm Project

6/3 Review of projects A-Arm Project Logbook Review Form

6/18 Course kaizen session

Logbook, Final Project,

and Design Report due

at end of day

A-Arm Project Course evaluations

Grading:

Notebooks/Homework 25%

Kaizen Project 30%

Final Project 45%

Drawing Package/Report (15%)

Product Quality (15%)

Shop Safety & Practices (15%)


DESIGN REVIEW ASSESSMENT FORM

Project: Date:

Check status: ____ Underclassman _____ Engineering Senior _____ Engineering Graduate/Client

A. Please score the design team’s presentation using each of the criteria and scoring scales given

below.

Your Scoring Scale Score 1 3 5 ORGANIZATION AND FLOW COMMENTS:

Order and flow unclear, may be confusing; time

usage ineffective

Order and flow clear, parts fit into whole; time used

well in general

Order and flow smooth, clear; time usage very

good to achieve purposes

LANGUAGE COMMENTS:

Language not refined; terms may be misused; voice

unsteady

Language and use of terms proper; voice clear

and steady

Language shows knowledge; very good

word choice; persuasive

CONTENT COMPLETENESS COMMENTS:

Presentation lacks some key

information needed by client

Presentation includes most key information

needed by client

Presentation includes all key information needed

by client

CONTENT RELIABILITY COMMENTS:

Information is uncertain and

causes doubt about its reliability

Information appears correct and reliable; proof

lacking

Information proven reliable; key data,

sources, derivations given

VISUAL AIDS COMMENTS:

Visuals crude, have some errors, are

generally readable; mixed impact

Visuals generally attractive, have no serious

errors, have positive impact

Visuals attractive, excellent quality; no

errors; achieve desired impact

LISTENING/ RESPONDING COMMENTS:

Interaction with audience limited;

not effective in Q/A

Maintains some rapport with audience; generally

effective in Q/A

Excellent rapport with audience; very effective listening, answering Q/A

CONFIDENTIALITY COMMENTS:

No awareness or regard for client confidentiality

Aware of need to respect client confidentiality; casual observance

Clearly knows and ensures appropriate client confidentiality


TEAM CREDIBILITY COMMENTS:

Team shows lack of expertise, inability to perform credibly

or effectively

Team has most necessary expertise, moderate ability

to manage project and personnel

Team exhibits all vital expertise, key skills,

poise; able to manage projects, personnel well

CONCEPT QUALITY COMMENTS:

Team partially understands needs of clients; concept

not viable, does not address some

important criteria*

Team understands main technical and nontechnical needs of clients; concept plausible, addresses most

crucial requirements, constraints*

Team fully understands diverse needs of clients,

society; concept innovative, viable,

satisfies all requirements and constraints*

ECONOMIC ISSUES COMMENTS:

Generally ignores cost issues;

concept not cost-effective

Discusses primary cost concerns; concept has economic plausibility

Effectively judges costs, benefits; concept has

economic viability

* Incorporate engineering standards and realistic constraints that include most of the following considerations: economic; environmental; sustainability; manufacturability; ethical; health and safety; social; and political.

B. Other comments or suggestions for improvement (use other side if needed):


DESIGN REPORT ASSESSMENT FORM

Project: Date:

Your Scoring Scale Score 1 3 5 BACKGROUND COMMENTS:

Identifies most basic client needs for product;

acknowledges few existing resources that may influence the development of a solution

States problem context relative to both client and the

state of technology within society; aware of relevant

literature, patents, and existing products

Describes and analyzes problem context in terms

of clients’ needs and societal/global issues; thoroughly analyzes relevant literature,

patents, and products

DESIGN REQUIREMENTS COMMENTS:

Few design requirements defined; most are loosely

defined, performance-related, qualitative; few or none

based on documented client needs’; broader

considerations* neglected

Defines important design requirements based on primary and secondary

clients; addresses technical and non-technical requirements and

constraints*; includes many measurable requirements

Skillfully defines comprehensive design requirements based on

needs of clients and stakeholders; addresses

system-level and life-cycle requirements and

constraints*; all are measurable

CONCEPTS CONSIDERED COMMENTS:

Limited number of useful concepts; do not reflect

knowledge of state-of-art; little creativity

Useful concepts for components and overall;

reflect knowledge of state-of-art for at least some parts;

moderate creativity

Many useful concepts for components and overall;

reflect knowledge of state-of-art for all

components; significant creativity

CONCEPT SELECTION COMMENTS:

Vague process to select concepts; little record of

decision making process; poorly-defined criteria

Rational, documented process to select concepts; clear measurable criteria in making design decisions

Quality client-focused process for selection;

fully documented; clear quantitative and

qualitative criteria

PRODUCT ARCHITECTURE COMMENTS:

Product features lack client-focus; performance not linked

to design requirements; no integration

Product evidences client-focus; meets key design

requirements; some system integration

Product delights client; fully meets design

requirements; components skillfully integrated into whole


ECONOMIC ANALYSIS COMMENTS:

Vague estimates of product costs; does not consider

other business issues

Reasonable estimates of costs and value to client;

markets identified for product

Reliable estimates of life cycle costs and benefits

to client; business potential well-defined

CASE FOR CONTINUATION COMMENTS:

Project strengths unclear; serious risks to project

completion or to business viability

Project strengths and risks identified; reasonable

potential for project success and for business value

Compelling case for success of project; risks managed; clear, strong

business potential

OVERALL REPORT QUALITY COMMENTS:

Conceptual design is incomplete or meets few

design requirements; little or no evidence justifying

product financially; report is incomplete, unattractive, can

be misunderstood, has distracting errors

Sound conceptual design meets most design requirements; some

indication that product will be economically feasible; report is complete, understandable,

attractive, few flaws

Innovative and competitive conceptual design meets all design requirements; credible

evidence that product will be financially successful; report is very complete, very clear, compelling

CONCEPT QUALITY COMMENTS:

Team partially understands needs of clients; concept not

viable, does not address some important criteria

Team understands main technical and nontechnical needs of clients; concept plausible, addresses most

crucial requirements, constraints

Team fully understands diverse needs of clients,

society; concept innovative, viable,

satisfies all requirements and constraints


DESIGN TEAM PERFORMANCE RUBRIC Commitment: Identifies with the vision. Provides leadership for others to realize the vision.

Produces functioning hardware/software that exceeds customer expectations. Enjoys

assuming even ill defined roles in a team environment. Can be counted on to organize and

carry out almost any work assignment with a high level of quality. Provides timely and well-

received feedback to self, team members, customer, and advisor/mentor. Regularly makes,

rereads, and reflects on logbook entries. Devotes considerable energy to leadership and self-

improvement.

Enrollment: Sees the benefits and wants the vision. Willing to be a good soldier in realizing

the vision, provided that someone else provides the leadership.

Produces functioning hardware/software that meets all customer expectations. Enjoys

assuming a well-defined role in a team environment. Can be counted on to accomplish most

assignments with good quality. Interacts frequently with customer and mentor. Regularly

makes logbook entries.

Formal Compliance: Sees some of the benefits of the vision, but skeptical of the magnitude

of personal changes required. Willing to do what is necessary within one’s job description.

Produces functioning hardware/software that meets most customer expectations. Praises

team, customer, and advisor/mentor solidarity in public. Privately criticizes other’s performance

but assumes no responsibility for negative results. Motivated to do the bare minimum on

documentation.

Oppositional: Sees few, if any, benefits of the vision. Requires constant supervision to do

what is expected. Vocally is not on board.

Produces hardware/software that meets some customer expectations. Displays fearful or

angry relations toward team members, customer, and advisors/mentors in public. Must be

prompted by a supervisor to complete most tasks. Maintains personal documentation only

under duress and resents doing it.

Apathy: Cannot see the vision and is unable to help realize the vision. Is often tardy or

absent.


Produces only untested, underdeveloped paper designs. Is indifferent toward team members,

customer, and advisors/mentors. Devotes no time or energy to personal documentation.


Logbook Quality Measure (cut out and paste in logbook)

Assessee:_______________Assessor:_______________Date:_______________

Daily Activities

Goals: ___vague ___multiple & divergent ___focused & strategic

Roles: ___implicit ___generic ___well-defined

Tasks: ___none cited ___sequenced & logical ___ remove bottlenecks

Meetings: ___scattered notes ___notes w/o discussion ___extensive notes w/discussion

Client Focus: ___no contact info ___some feedback noted ___feedback analyzed

Design Development

Research: ___sparse notes ___relevant notes ___detailed notes & sources

Modeling: ___none/inappropriate ___underdeveloped ___results & discussion

Illustrations: ___none ___basic w/o discussion ___detailed w/discussion

Decisions: ___random ___highlighted ___justified

Purchasing: ___no cost data ___costs inventoried ___vendor comparisons

Project Review

Strengths: ___none cited ___basic awareness ___detailed knowledge

Improvements: ___none cited ___basic awareness ___implementation planning

Lessons: ___none cited ___valuable to team ___valuable to other teams

General Housekeeping

Entries : ___sporadic ___on demand ___spontaneous

Organization: ___haphazard layout ___readable by author ___adds value to others

Headings: ___none ___generic ___informative

IP Issues: ___pencil w/erasures ___pen & dates ___pen & dates & witnessing


Logbook Review Process: Data Collection: check all descriptors that apply based on 5 minute skim.

Data Analysis: Identify your two best entries. Explain what makes each of these exemplary.

Describe two improvements you want to implement before the next round of logbook review.

Outline the benefits you hope to achieve and the action plan you will implement to realize these

improvements.

Peer Feedback: Have at least one other person review your logbook. Have them

do their own scoring and share their ideas about your best entries and areas for improvement.

Look for best practices/insights you want to adopt for yourself.

Instructor Feedback: Submit your logbook to your instructor with personal and peer review

forms/discussion attached. Identify specific areas where you would like feedback.


mentorship and performance assessment of design … · 1 kaizen is a japanese wo rd f or cont...

Documents