designing valid assessments of student learning rodolfo rincones cetal workshop # 1 enhancing...
Post on 25-Dec-2015
216 Views
Preview:
TRANSCRIPT
Designing Valid Assessments of Student Learning
Rodolfo Rincones
CETaL Workshop # 1
Enhancing Student LearningUTEP Library - Blumberg Auditorium
September 3, 2009
Outline
Context of assessment in higher education
Purpose of assessment
Linking learning and assessment
Types of assessment
Activity
What we know ( 5 minutes)
What we want to know ( 5 minutes)
What we learned (at the end 5 minutes)
Context of Higher Education
Decreasing public funding sources
Tight budgets
Expanding pressures to do more with less
Escalating tuition
Public trust
Increase productivity
Accountability
Context of Higher Education
Factory-production model based on semester credit hours, certification, transfer, articulation, and “student success”
Limited records that reflect actual student learning outcomes
Paradigm shift from teaching to learningDonohue-Lynch, B. Assessment with 21st Century Tools
http://www.carnegiefoundation.org/perspectives/sub.asp?key=244&subkey=2140
Are we assessing student learning to appease a growing tide of legislators and government executives who increasingly question the full impact of colleges and universities on student learning?
or
Are we assessing student learning to improve our own educational practices, curriculum choices, and instruction?
McKitrick, Sean , Assistant Provost for Curriculum, Instruction, & Assessment at Binghamton University (SUNY)
Undergraduate education in research universities requires renewed emphasis on a point strongly made by John Dewey almost a century ago: Learning is based on discovery guided by mentoring rather than on the transmission of information.
Inherent in inquiry-based learning is an element of reciprocity: Faculty can learn from students as students are learning from faculty
The Boyer Commission on Educating Undergraduates in the Research University, 1998
Today’s Learning
Today Learning in the 21st Century
Curriculum
“Curriculum is the formal and informal content and process by which learners gain knowledge and understanding, develop skills, and alter attitudes, appreciations, and values under the auspices of that school.” (Keller, 2004)
Although shared knowledge is an important component of a university education, no simple formula of courses can serve all students in our time.
Integrated educational experience
Collaborative learning experiences
Promote analysis, synthesis, and evaluation
The Boyer Commission on Educating Undergraduates in the Research University, 1998
Creating the learning environment
Teaching
Involves developing the knowledge, skill, mind, character, or ability of others. It “means not only transmitting knowledge, but transforming and extending it as well.”
“Dynamic endeavor involving all the analogies, metaphors, and images that build bridges between the teacher’s understanding and the student’s learning. Pedagogical procedures must be carefully planned, continuously examined, and relate directly to the subject taught.” (Boyer 1990)
Learner-Centered Teaching
How to tie teaching, curriculum, and objectives of learning rather than to the content delivery alone. Focuses attention on what the student is learning, how the student is learning, the conditions under which the student is learning, whether the student is retaining and applying the learning, and how current learning positions the student for future learning. (Weimer, 2002)
Purposes of educational assessment
Assessment is a comparison of achieved results to intended goals (Ewell, 1985)
TraditionalDiagnose students’ strengths and weaknesses
Monitor students’ progress
Assign a grade
Determine program and institutional effectiveness
Purposes of educational assessment
Formative - to aid learning
Summative - for review, transfer and certification
Summative - for accountability to the public
Reasons for Assessing
Shupe, D. (2006). Toward a higher standard: the changing organizational context of accountability for educational results, ON THE HORIZON, Vol. 16, No. 2 2008, pp. 72-96.
Student Learning Outcome
Student learning outcomes should refer normally to competencies or attainment levels reached by students on completion of an academic program
Courses must define two specific sets of learning objectives
Traditional knowledge and skills associated with the course
Abilities addressed in the course. (Keller, 2004)
Abilities
Communication
Analysis
Problem Solving
Valuing in Decision-Making
Integrated Developmental Transferable
Social Interaction
Global Perspectives
Effective Citizenship
Aesthetic Responsiveness
Learning Assessment
What is meant by “assessment” often varies greatly—embracing everything from job placement, through student satisfaction, to self-reported gains in skill and knowledge on the part of students and former students. (Ewell, 2001)
Types of Measures
Direct Measures- provide clear and compelling evidence of what students are learning
Course-embedded assessments, including written work and presentations scored using a rubricScores on locally designed tests and competency exams accompanied by test “blueprints” describing what is being assessedScore gains between entry and exit on tests, competency exams and writing samplesRatings of student skills in the context of class activities, projects and discussionsPortfolios of student workScores on nationally- norm instruments (Ewell, 2001)
Types of Measures
Indirect Measures-Provide signs that students are probably learning, but it is less clear exactly what they are learning.
Student grades Student evaluations and ratings of the knowledge and skills they have gainedStudent or graduate satisfaction with their learning in general education competenciesResults of nationally-norm surveys (Ewell, 2001)
Assessment Evaluation Criteria
ReliabilityConsistency with which assessment produce measures whatever it is measuring
ValidityReflects the defensibility of score-based inferences made on the basis of an educational assessment procedure
Assessment Evaluation Criteria
Absence-of-BiasDegree to which assessments are free of elements that would offend or penalize examinees on the basis of examinees’ gender, ethnicity or other characteristics (Popham, 2002)
Learning Objectives
Define a course in terms of the outcomes the instructor expects students to achieve
ComponentsA description of what the student will be able to do (verb)
The conditions under which the student will perform the task
The criteria for evaluating student performance
Importance of Learning Objectives
Selection of content
Development of an instructional strategy
Development and selection of instructional materials
Construction of tests and other instruments for assessing and then evaluating student learning outcomes
Efficient use of instructional time
Ongoing improvement and student feedback
Example
Objective: Given a set of data the student will be able to compute the standard deviation.
Condition - Given a set of data
Behavior - the student will be able to compute the standard deviation.
Criterion - the number computed will be correct
Course Assessment andEnhancement Model
Combs, K., et al.(2008). Enhancing curriculum and delivery: linking assessment to learning objectives. Assessment & Evaluation in Higher Education, Vol. 33, No. 1, pp. 87–102.
Your turn….
Connecting curriculum elements
Thank you!!!
top related