evaluation methods and findings of the field-tested learning assessment guide
DESCRIPTION
Evaluation Methods and Findings of the Field-tested Learning Assessment Guide. Sue Daffinrud Olga Lucia Herrera. The Motivation for the FLAG. Motivated by needs of faculty who wanted centralized place to learn about already-verified assessment techniques and tools (Third Annual NISE Forum) - PowerPoint PPT PresentationTRANSCRIPT
Evaluation Methods and Findings of the Field-tested
Learning Assessment Guide
Sue Daffinrud
Olga Lucia Herrera
The Motivation for the FLAG
Motivated by needs of faculty who wanted centralized place to learn about already-verified assessment techniques and tools (Third Annual NISE Forum)
Does the FLAG satisfy part of this need?
FLAG Structure
Five major sections Getting Started Matching Goals to CATs CATs Searchable Tools Resources
FLAG target audience
Instructors who:
teach freshman and sophomore-level science, math, and engineering courses
and are interested in learning about new
approaches to assessment
FLAG Evaluation
Does the FLAG website fulfill some of the assessment needs of its target audience?
Formative and Summative Evaluation
Identifying target audience sample
Selected subset of SME disciplines Biology, chemistry, engineering, mathematics,
and physics Selected range of institutions
By Carnegie classification Associate of Arts, Baccalaureate, Masters,
Doctoral, and Research Region of country
Identifying target audience sample
Selected faculty through their participation in some initiative which has student learning as its focus. bioQUEST, Worskhop Biology, OCEPT Chemistry Coalitions Engineering Coalitions, EESP Project NExT Workshop Physics
50 faculty who have demonstrated an interest in “learning, not teaching”
Bio Chem Engr Math Phys
Assoc 2 2 2 2 2
Bacc 2 2 2 2 2
Masters 2 2 2 2 2
Doctoral 2 2 2 2 2
Research 2 2 2 2 2
Contacting participants
Called and emailed faculty requesting their participation
Participation involved: Browsing through FLAG for 30 minutes Providing feedback in a 30 minute interview
High participation rate
Interviews
Participants were asked about: Their current teaching practice How they learn about new
assessment/teaching techniques Impressions of FLAG, both specific and
general Whether would recommend FLAG to others
Data Gathering Status
Have interviewed 32 faculty Will interview the remaining 18 in
December
Preliminary findings
Demographics of respondents
Technical issues
Themes of Use and Need
Dissemination
Demographics
Range of teaching experience (2-22 yrs) Critical of the value of lecture Goals: improving students’ interest in science Role: facilitator for their learning Info on assessment gained through
conversations with fellow faculty, conferences in their discipline
Few “surf” the web
Technical Issues – Navigation patterns
Participants spent between 30 minutes to 4 hours in the FLAG, most about an hour.
Many moved “left to right”
Technical Issues
Almost all found the web site Easy to read Structured well Could find what they needed Enjoyed being able to download documents
Suggestions Provide more obvious description of site structure Provide better navigation instructions
Themes of Use/Need - Overall
Overall response: Positive
Almost all respondents had already or had plans to recommend the site to other faculty, faculty developers, or other groups
Themes of Use/Need - Sections
Has teaching component Matching Goals to CATs Primer
Availability of links to other web sites Annotated bibliography References
“Empty” sections create expectation Searchable tools “Experts” by state
Themes of Use/Need - CATs
Time is a factor in browsing and in applicability First page of CATs (why, how, what’s
involved) People selected:
CATs which were familiar to them CATs which best fit their goals and time
Themes of Use/Need – CATs (cont)
People want a complete picture Like the case study approach When and how to use the CAT What to do with the information How to get students “on board” Clear and useful examples
People want “tools” from their discipline to varying degrees
Dissemination of FLAG
Evaluation AS dissemination Hope to find out how best to reach people
like our sample Discipline-specific professional meetings Workshops People on campus who do assessment Email to appropriate email lists
Final Notes
Submitted abstract for presentation at AAHE 2000
Write monograph or paper Provide more formative information
to Tony To ILT group as they develop site