classroom teaching observations presentation 9-20-18 · planning classroom teaching observations...
TRANSCRIPT
PlanningclassroomteachingobservationsGuidanceforfacultyandstaff
OfficeofAssessment,TrinityCollegeSeptember2018
Ifyou’dliketofollowalong:http://bit.ly/teaching_obs
Ifyou’dliketofollowalong:http://bit.ly/teaching_obs
ThePowerPointandaudio(ifacceptablequality)willbepostedonourwebsitewithinaweek.
https://assessment.trinity.duke.edu/assessment-roundtable
Soundevidence
Meaningfulreflection
Culture of continuous improvement
Soundevidence
Meaningfulreflection
ClassroomobservationsCourseevaluations
Evidenceofstudentlearningoutcomes
Self-evaluationsHabitsandpracticesofreflection
Culture of continuous improvement
Weunderstandstudentlearningthrough:TestsSurveys&evaluationsRubric-scoredprojectsInterviews
Weunderstandinstructionaldeliverythrough:
Surveys&evaluationsTeachingobservations★
Weunderstandstudentlearningthrough:TestsSurveys&evaluationsRubric-scoredprojectsInterviews
Outcomesassessment
UnderstandingthedegreetowhichstudentsarelearningUnderstandingwhatstudentsknowandcandofollowingalearningexperience:
• Basiccontentknowledge• Comprehension&application• Analysis&evaluation
Programevaluation
UnderstandinghowaprogramoperatesUnderstandingtheinputstolearning:
• Areourfacilitiessufficient?• Arelecturesandreadingsaligned?
• Areinstructorseffective?
Outcomesassessment
UnderstandingthedegreetowhichstudentsarelearningUnderstandingwhatstudentsknowandcandofollowingalearningexperience:
• Basiccontentknowledge• Comprehension&application• Analysis&evaluation
Programevaluation
UnderstandinghowaprogramoperatesUnderstandingtheinputstolearning:
Outcomesassessment
UnderstandingthedegreetowhichstudentsarelearningUnderstandingwhatstudentsknowandcandofollowingalearningexperience:
• Basiccontentknowledge• Comprehension&application• Analysis&evaluation
• Areourfacilitiessufficient?• Arelecturesandreadingsaligned?
• Areinstructorseffective?
Whymightanindividualordepartmentdesireclassroomteachingobservations?
Toensurestudentshave
effectivelearning
experiences
Toimproveyourownpractice
Toseehowanewapproachto
teachingisworking
Becausestudentscannotevaluateallaspectsofteaching
Tounderstandcourseevaluationresultsbetter
Torespondtopossibleproblems
intheclassroomAsameansofmentorship
betweenexpertandnovice
Tospurdialogueamongfaculty
Asaregularpartofthepeer
reviewprocess
INDIVIDUAL GROUP
Whatareyourobjectives?
Whatbroughtyouhere?Whatdoyouwanttoaccomplish?
Summative Formative
IntrospectionLocalizeddialogue
Internalimprovement
Annual/APTreviewsReportingoutsideunit
ModelsHybridizationispossible!
Characteristics Advantages Challenges
Department-facilitated
WHO:InstructorswithinthedepartmentWHAT:Rotateandreciprocallyevaluateteachingacrosscoursesandfacultyinthedept
Observationswillbealignedwiththemission&outcomesofthedeptShouldresultinconsensusaboutteachingexcellenceInformsprogramevaluationandself-study
RequiressubstantialinvestmentacrosstheunitMustsetorderofpriorityforcoursesorinstructorsInstructorsmayfeelvulnerable;maybelessinvestedifnotinvolvedinprocess.
Externalobserver(fromanotherinstitutionordepartment)
WHO:Individualobserver(s)unaffiliatedwiththeprogramWHAT:Planwithinstructorinadvancetheobjectives,format,rubric,desiredinformation.
Objectivity:Personalrelationshipsdon’tinfluencetheevaluationWell-trained,sufficientexperiencewithrubricsResultsmaybemorereliableandvalid
Observerdoesn’tknowtheprogramorpersonnelObservermaynotknowthefieldDifficulttoimplementacrossadepartment
Peer-to-peer(reciprocal)
WHO:Individualinstructorswhowanttoreflectontheirteachingpractice,independentofdepartmentaleffortsWHAT:Joinsmallgroupoflike-mindedcolleaguestoreciprocallyobserve/discuss
ConversationalLowrisk;emphasizesformativedevelopmentofallpartiesInsightsfromotherfieldsandteachingmethods
PossiblydivergentconceptionsofteachingexcellenceObservermaynotknowthefieldInformationdoesn’tfeedbacktothedepartment
ModelsHybridizationispossible!
Characteristics Advantages Challenges
Peer-to-peer(reciprocal)
WHO:Individualinstructorswhowanttoreflectontheirteachingpractice,independentofdepartmentaleffortsWHAT:Joinsmallgroupoflike-mindedcolleaguestoreciprocallyobserve/discuss
ConversationalLowrisk;emphasizesformativedevelopmentofallpartiesInsightsfromotherfieldsandteachingmethods
PossiblydivergentconceptionsofteachingexcellenceObservermaynotknowthefieldInformationdoesn’tfeedbacktothedepartment
https://www.facultyfocus.com/articles/faculty-development/teaching-squares-cross-disciplinary-perspectives/àhttps://learninginnovation.duke.edu/faculty-opportunities/connect-with-other-faculty/
“Ateachingsquareconsistsoffourfacultyfromdifferentdisciplineswhovisiteachother’sclasseswithinatwo-to-three-weekperiod.Aftertheclassroomvisits,thefourgatheraroundcoffeeoramealtodiscusstheteachingobserved.Theintentionofthesquareisnottocriticizeeachother’steaching.Rather,it’stogatherideasondifferentteachingapproachesthatmightbeusedinone’sownclasses.It’sanopportunityforfacultytoreflectontheirownteachinginlightofcolleagues’teachingexamples.”
ModelsHybridizationispossible!
Characteristics Advantages Challenges
Department-facilitated
WHO:InstructorswithinthedepartmentWHAT:Rotateandreciprocallyevaluateteachingacrosscoursesandfacultyinthedept
Observationswillbealignedwiththemission&outcomesofthedeptShouldresultinconsensusaboutteachingexcellenceInformsprogramevaluationandself-study
RequiressubstantialinvestmentacrosstheunitMustsetorderofpriorityforcoursesorinstructorsInstructorsmayfeelvulnerable;maybelessinvestedifnotinvolvedinprocess.
Departmentprovidesleadership,structure,andresourcestoenablesystematic,intentionalteachingobservations.Createandreinforcethemessagethatintrospectionandpeerreviewarepartofthedepartment’sculture.Thisisnotapunitiveapproachtoteachingpractice.It’suptothefacultyanddepartmentalofficerstodeterminewhatformatworksbest.Butit’snotnecessarilytop-down:Requiresdialogueandconsensusaboutexcellentteaching.
SpecificthoughtquestionsfordepartmentchairsandprogramofficersWillobserversbeadministratorssuchaschairsorprogramdirectors,seniorfaculty,orteachersatalllevelswhoarepairedtoobserveoneanother'scourses?Willyouinvitefacultyfromoutsideyourprogramortheinstitution?Ifso,whom?Willtheresponsibilityofobservingbevoluntaryormandatory?Willthefacultywhodoformativeobservationsbethesameasthosewhodosummativeobservations?Betransparentabouthowwillfeedbackbeused.Forinformalfeedback?Forteachingreviews?
Adaptedfromhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspx
ModelsHybridizationispossible!
Characteristics Advantages Challenges
Externalobserver(fromanotherinstitutionordepartment)
WHO:Individualobserver(s)unaffiliatedwiththeprogramWHAT:Planwithinstructorinadvancetheobjectives,format,rubric,desiredinformation.
Objectivity:Personalrelationshipsdon’tinfluencetheevaluationWell-trained,sufficientexperiencewithrubricsResultsmaybemorereliableandvalid
Observerdoesn’tknowtheprogramorpersonnelObservermaynotknowthefieldDifficulttoimplementacrossadepartment
Notwhollyseparatefromordistinctfromtheothermodels.Simplymeansbringinginobjectivityfromoutsidetheunit,usuallyintheformofanexternalcolleagueorexpertwithinthediscipline.Mayrequireadditionalcoordinationifexternalobserversareaskedtoutilizerubrics,feedbacktools,schedules,etc.thatarepartofthedepartment’sstructuredapproachtoteachingobservations.
StructuringobservationsEachdepartmentwillhaveitsownapproach!Commonpracticesinclude:
Timing:3-4timespercourse,distributedacrossthesemester,andtouchinguponauthenticrepresentationsofthecourseexperience.
Selectinstructorsand/orcourses:Cyclicalbasis,perhapsbasedonexistingreviewschedules.You’llhavetoprioritizesomeindividualsandcourses,anddeterminefairdecisionrules.
Selectionofobservers:2colleaguesfromwithintheprogram,possibly+1fromoutsidetheprogram.Determinewhichobserver(s)bringabalanceofcontentknowledgeandteachingexpertise.Isserviceasobservervoluntaryorrequired?
Usingarubric/feedbacktool:It’shelpfultohaveastandardizedfeedbacktool,butwhichcanaccommodatedifferencesincoursetypes(e.g.,labsandlectures).Develop,share,solicitfeedback,revise.Likeallrubrics,ittakestimetodevelopconsensusamongindividualswhohaveastakeineffectiveteachingevaluations
Discussionandreporting.Promptdeliveryoffeedback,includingspecificsuggestionsforteaching,isessential.Howthisoccursisuptothedepartment.Instructorsshouldconsiderincludingintheirteachingportfoliossummariesofhowtheyuseobservationalfeedbacktodeveloptheirteachingpractice.
http://bit.ly/teaching_observation_toolshttps://assessment.trinity.duke.edu/course-evaluation-codes-trinity-college
Commondimensions
ofobservational
tools
Format,sequencing,organizationoftheclass? Rapport
betweeninstructor&students?
Studentengagementwithlearning
tasks?
Relevanceoflearning
evaluations?
Accuracyofcontent?
Relevanceofsupportingmaterials&resources?
Equityintheclassroom?
Suitabilityanduseofinstructional
tech.?
Typesofrubric/feedbacktoolsExamplescollectedhere:http://bit.ly/teaching_observation_tools
Structuredcategories,withmostlyfree-response
• https://stanford.app.box.com/s/fe73bkt0cwxd7mr4z6kme2wt3fi7aa84• http://www.academic.umn.edu/provost/peer_review/documents/
observation_protocol.docx• http://www.fctl.ucf.edu/FacultySuccess/FacultyDevelopment/ClassObservations/
contents/criteria_for_peer_observation.pdf• https://wiki.ubc.ca/images/7/75/CTLT_SGIF_Form_March_2016.pdf
Structuredcategories,withdefinedrubricratings
• https://www.augusta.edu/pamplin/documents/peerevalteaching_rubric.pdf• http://tdop.wceruw.org/Document/TDOP-2.1-Users-Guide.pdf
Checklist • http://www.celt.iastate.edu/wp-content/uploads/2017/07/examplechecklistclassroomobservationform-.docx
Specialty:Learner-centricrubric
• https://cetl.kennesaw.edu/sites/default/files/resources/Learner%20Centered%20Rubric%20for%20Classroom%20Observations.pdf
Specialty:Distanceed. • https://www.westga.edu/~distance/roblyer32.html
Specialty:STEM • http://www.cwsei.ubc.ca/resources/COPUS.htm• https://www.lifescied.org/doi/full/10.1187/cbe.14-02-0023
Speciality:Humanities • https://utop.uteach.utexas.edu/sites/default/files/utop-instrument-humanities-august2014_0.doc
Mappingobservationaltoolsacrossclassroomvisits
http://www.ben.edu/faculty-staff/ctle/fac_resoåurces/forms_teaching.cfm
Reviewingotherinstructionalartifacts
http://www.academic.umn.edu/provost/peer_review/
Whatchallengesalreadycometomind?
• Observationsmakeinstructorsfeelvulnerable.Protecttheiragency,andrecognizetheircommitmenttoself-development.
• Well-plannedobservationsrequireaninvestmentoffacultytime.
• Teachingwhileobservedcanfeelunnatural,inauthentic.
• Theremightbealackofconsensusaroundwhatconstituteseffectiveteaching……whichcanfrustrateparticipantsand…impedethedevelopmentofacommonevaluativerubric.
• Iftheprocessisnotwell-calibrated,theresultsmaybelessreliable.Plantotriangulate:useobservationalevidencewithothermeasuresofteaching
Typicalchallengestoacknowledge
Concern Advicetotheobserver Advicetotheobservedinstructor
AdvicetoChairs&otherofficers
Instructorvulnerability
Recognizetherearemultipleeffectiveapproachestoteaching,anddifferencesincomfortwithinstructionaltechnologies.Collaborateonthefocusoftheobservation,goals,areasofconcern.Usethemtoguidefeedback.Balanceconstructivecriticismwithgenuine,specificpraise.
Beopentodiscussingyourteachingchallengesaswellasstrengths.Activelyparticipateinthedevelopmentoftheobservationprocess,especiallyfeedbacktools.
Recognizethosewhovoluntarilyparticipateinteachingobservationsandwhousefeedbacktodeveloptheirprofessionalpractice.Nurtureacultureofconstructiveintrospectionanddialogueaboutteachingexcellence.
Time&labor Planfor3-4visitsacrossthesemester.Balancethoroughnessofobservationsovertimewithlaborrequirements.Recognizethatagoodrubricorchecklistmaytakemultipleiterations.
Acknowledgetheinvestmentoffacultytime.Providereleasetimetodevelopmentgroup,ifpossible.
Authenticity Beasunobtrusiveaspossiblewhenvisitingface-to-faceclassestoputboththestudentsandtheinstructoratease.
Trynottoadjustyourteachingstyleandmethodstothepresenceoftheobserver.Feedbackshouldbebasedonyourauthenticpractice.
Communicatethatteachingobservationsemphasizeformativeimprovement.Reinforceideathateveryinstructorbringsuniquestrengthstotheclassroom.
Consensus Activelyparticipateinthedevelopmentoftheobservationprocess,especiallyfeedbacktools.Beinclusiveofdifferentstyles,assumptions,experiences,andlevelsofcomfort/confidencewithteachinginnovations.
Invitediscussionofexpectations,concerns,andgoals.Clarifypurposeoftheobservationandhowresultswillbeused.Conveneworkinggrouptodevelopfeedbacktool.
Reliability&usefulness
Focusyoursummaryremarksandfollow-updiscussiononspecificareasforimprovement.Offerrelevantstrategiesthattheinstructorcantryinthenearfuture.
Savematerialstoincludeinyourteachingportfolio.Makespecific,focusedplanstoadjustelementsofthecourseoryourteachingapproach.Askforhelpinterpreting/blendingobservationalfeedbackwithcourseevaluationfindings.
Ifimplementedacrossthedept.,encouragecalibrationofobservationsandfeedback.Providetraining.Integrateprofessionaldevelopmentopportunities.Useobservationsinconjunctionwithcourseevaluations.
Adaptedfromhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspx
Adaptedfromhttps://ctl.yale.edu/ReflectiveTeaching
ObservationsareonesourceofevidenceThereareotherwaysfacultycanreflectonandenhancetheirteachingpractice.• Reflectionjournals• Video-recordedteachingpractices• Teachingportfolio• Mid-andend-of-semesterevaluations
https://assessment.trinity.duke.edu/course-evaluation-codes-trinity-college
Mid-termcourseevaluations:
Anothertypeofevidence
https://assessment.trinity.duke.edu/midterm-assessment-strategies
Mid-termcourseevaluations:
Anothertypeofevidence
Mid-termcourseevaluations:
https://assessment.trinity.duke.edu/midterm-assessment-strategies
Anothertypeofevidence
Lastnotes:Evaluatingunionizedfacultyhttp://admin.trinity.duke.edu/finance/non-regular-rank-union-faculty/self-assessment-portal
ConsultresourcesonFacultyAffairswebsiteForexample,http://admin.trinity.duke.edu/faculty-affairs/policies-procedures?qt-faculty_affairs_policies=4#qt-faculty_affairs_policies
DepartmentAssessmentPortfolio:ProgramObjectives
Programevaluation Outcomesassessment
https://learninginnovation.duke.edu/faculty-opportunities/connect-with-other-faculty/https://learninginnovation.duke.edu/faculty-opportunities/art-and-science-of-teaching/assessing-your-teaching/https://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Pages/teaching-observations.aspxhttps://resources.depaul.edu/teaching-commons/teaching-guides/reflective-practice/Documents/Classroom-Observation-Checklist-AustinCC.pdfhttps://cft.vanderbilt.edu/guides-sub-pages/peer-review-of-teaching/https://www.facultyfocus.com/articles/faculty-development/teaching-squares-cross-disciplinary-perspectives/https://ctl.yale.edu/Observation-Protocols-Teaching-Inventorieshttp://www.crlt.umich.edu/resources/peer-review
Webresources
Selectbibliography★ Arreola,R.A.(2007).Developingacomprehensivefacultyevaluationsystem:Aguidetodesigning,building,and
operatinglarge-scalefacultyevaluationsystems.AnkerPublishingCompany.Bernstein,D.(1996).Adepartmentalsystemforbalancingthedevelopmentandevaluationofcollegeteaching:A
commentaryonCavanagh.InnovativeHigherEducation,20(4),241-247.Carter,V.(2008).Fivestepstobecomingabetterpeerreviewer.CollegeTeaching,56(2),85-88.Cavanagh,R.(1996).Formativeandsummativeevaluationinthefacultypeerreviewofteaching.InnovativeHigherEducation,
20(4),235-240.★ Chism,N.,&Chism,G.(2007).Peerreviewofteaching:Asourcebook(2nded).Bolton,MA:AnkerPub.Francis,D.(1995).Thereflectivejournal:Awindowtopreserviceteachers'practicalknowledge.TeachingandteacherEducation,11(3),
229-241.Mager,D.,Kazer,M.,Conelius,J.,Shea,J.,Lippman,D.,Torosyan,R.,&Nantz,K.(2014).Development,implementationand
evaluationofapeerreviewofteaching(PRoT)initiativeinnursingeducation.InternationalJournalofNursingEducationScholarship,11.
Seldin,P.,Miller,J.,&Seldin,C.(2010).Theteachingportfolio:Apracticalguidetoimprovedperformanceandpromotion/
tenuredecisions(4thed).SanFrancisco:Jossey-Bass.Stone,D.,&Heen,S.(2014).Thanksforthefeedback:Thescienceandartofreceivingfeedbackwell(evenwhenitisoffbase,
unfair,poorlydelivered,andfrankly,you'renotinthemood).NewYork:Viking.