engineering education conference - spring 2009 increasing assessment effectiveness in a time of...
Post on 27-Dec-2015
215 Views
Preview:
TRANSCRIPT
EngineeringEducation
Conference - Spring 2009
Increasing Assessment Increasing Assessment Effectiveness in a Time of Effectiveness in a Time of
Decreasing BudgetsDecreasing Budgets
Ken Stanton
EngineeringEducation
PhD in Engineering PhD in Engineering Education at Virginia TechEducation at Virginia Tech
First cohort began Fall 2008
16 total students Coursework in
engineering, education, and engineering education
Teaching and research assistantships available
Graduate Recruitment Days every March and October– travel grants available!
www.enge.vt.edu
EngineeringEducation
Introduction – The Introduction – The ProblemProblem
ABET EC2000 standards are a lot of work for engineering departmentsPurposes: assessment and accountabilityProblems: some faculty less familiar with
assessment, requiring more and more of programsSolutions: faculty development, assessment
experts
Economic downturn affects education budgetsProblem: difficult to implement the solutions above
Enrollment, and therefore class size, increasesTime and resources limited
Any solutions: must be efficient, use what we have, be simple and easy to implement
EngineeringEducation
Introduction – Proposed Introduction – Proposed SolutionSolution
Address the need to increase assessment effectiveness from three angles1. Utilize lean assessment techniques2. Implement a process that maximizes
intrinsic motivation3. Take advantage of efficiency of assessment
methodology
EngineeringEducation
Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment
Goal of assessment
Teaching toward learning
objectives
Assignment created for
student
Student completes
assignment
Grade and feedback given to
student
Typical process for an assignment
EngineeringEducation
Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment
Teaching toward learning
objectives
Assessment created for
student
Student completes
assessment
Feedback given to student, may
be graded
Collect and analyze evidence of learning
Use to improve teaching and assessments
Typical assessment process
EngineeringEducation
Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment
Definitionsimple, efficient, and effective assessment
techniquesbased in the principle of lean manufacturing
productive and simple methodsstill attain useful data
MethodsSome existing
Angelo and Cross (1993)
Some modifiedSome novel
EngineeringEducation
Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment
Technique: Time Requirement:
Purpose: Description: Notes:
Rubrics (for various activities)
MED;LOW if shared development
Categorize important assessment and grading criteria; set up structure to summarize performance; provide clear expectations to students
Important criteria are categorized and assigned point values in a table, then performance levels (e.g. novice, expert) described
Many resources online for pre-defined or custom rubrics; coordinating with other faculty can decrease time investment and increase consistency of course/material
One-minute problem summary
LOW Evaluate student understanding of assigned problems and underlying concepts
At the end of a problem, the student takes one minute to explain what major concepts were used and what was learned
Good to prompt students with leading question to answer in problem summary
Use to assess teaching and learning
EngineeringEducation
Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment
Technique: Time Requirement:
Purpose: Description: Notes:
Individual project reports (in team project)
MED Assess and improve individual report-writing abilities
When team reports are required, have each student submit individual report at least one week preceding
Reviewing 3-5 times as many reports can be time consuming; a rubric should be used and specific aspects of reports focused on
Oral exams LOW and can reduce time
Assess oral abilities and allow for deeper probing of key topics; mimic business meeting
Students present and discuss project instead of written exam/project, one-on-one with instructor; oral questions provide richer evidence of knowledge and learning; grade is given before leaving meeting
A form of authentic assessment, abilities are tested in a more realistic manner that closely reflects real-world; time can be less than administering and grading exams/ projects
EngineeringEducation
Innovative Solution – Innovative Solution – ImplementationImplementation
These assessment methods alone do not solve the problem
Consider motivation theory in implementation
Self-Determination Theory (SDT) by Deci & Ryan (2000) has 3 needs 1. Competence – confidence, self-efficacy2. Autonomy – choice that aligns with identity /
values3. Relatedness – connection to others identify with Satisfaction of these leads to internalization of
motivation – move from extrinsic to intrinsic
EngineeringEducation
Innovative Solution – Innovative Solution – ImplementationImplementation
CompetenceLacking due to inexperience with assessmentLean assessment techniques – simple, effectiveFaculty helping each other
AutonomyChallenged by requirements being handed downHelped somewhat by flexibility of ABET criteriaUse assessment to address faculty’s teaching
concerns
RelatednessCan take opportunity to connect with students
through feedback
EngineeringEducation
Innovative Solution – Innovative Solution – ImplementationImplementation
1. Review program educational objectives (PEOs), program learning outcomes, and course outcomes established for the program and course(s) of interest.
2. Interview faculty for input on assessment and what they would like to improve in their teaching and students’ performance. Review current assignments and assessments being used in their course(s) of interest.
3. Identify lean assessment techniques thatA. are good for measuring the outcomes from #1 B. meet goals that faculty name as important to themC. can complement/replace existing assignments and
activities in the course(s).
EngineeringEducation
Innovative Solution – Innovative Solution – ImplementationImplementation
4. Briefly illustrate how the lean assessment techniques match the faculty member’s goals and can be used alongside or in place of existing course materials.
5. Assist with familiarization and use of the assessments, if necessary, and encourage sharing the expectations and motives with students.
Likely to be effective seeing as studies found faculty generally hold positive views of assessment
EngineeringEducation
Innovative Solution – Innovative Solution – Assessment Assessment Methodology Methodology
Many times we develop courses by deciding material then assessmentsLess efficient and less productive
Wiggins & McTighe (2005) coined “Backward Design”Work from learning objectives to ways to
measure them (assessments) to how to teach itEnsures efforts are goal-oriented and
assessments can be used for accreditation
Using this process can increase efficiency of teaching and assessment efforts
EngineeringEducation
Introduction – Proposed Introduction – Proposed SolutionSolution
Address the need to increase assessment effectiveness from three angles1. Utilize lean assessment techniques2. Implement a process that maximizes
intrinsic motivation**3. Take advantage of efficiency of assessment
methodology
EngineeringEducation
Future workFuture work
Dissertation studyFrom a motivational standpoint, how do
engineering faculty members engage in assessment of student learning?
Also look at ways to integrate assessment into the culture of teaching and learning
EngineeringEducation
THANK YOUTHANK YOU
QUESTIONS?QUESTIONS?
top related