“…the systematic collection, review, and use of information about educational programs...
TRANSCRIPT
DATA COLLECTION – TO CURRICULAR CHANGE
USING YOUR ASSESSMENT DATA TO DRIVE CURRICULUM
CHANGES
ASSESSMENT “…the systematic collection,
review, and use of information about educational programs undertaken for the purpose of improving student learning and development.”
Palomba & Banta, 1999, pg. 4
THE 50,000 FOOT LEVEL Start with the culture because faculty
must be comfortable with the assessment culture before they feel comfortable with assessment!
Determine the role that assessment will play in college processes – and what it WON’T be used for! Improving Student LearningProgram EvaluationBudgetingNOT Faculty Evaluation
START WITH THE CULTURE
Value Campus Culture & History
(Assessment is not a one size fits all)
Respect and Empower People – especially
Faculty!
Value Assessment by providing
appropriate resources and infrastructure
Value innovation & risk taking to improve
teaching (even if it fails)
VALUE INNOVATION
“I have not failed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison
Assessment
I think we are about to turn the corner on this whole “Assessment” thing…
EFFECTIVE ASSESSMENT Is linked to decision making about the
curriculum – Paloma & Banta
Measures real-life gaps in desired skills & performance – Swing, et. al.
Leads to reflection and action by faculty – Paloma & Banta
WE’RE COLLECTING DATA BUT…
Institutions and faculty are good at collecting data – just not as good at using data to drive curricular changes
This part of the assessment cycle is often called:“Closing the loop”What loop – and who left it open?
HELPING FACULTY GET TO CLOSING THE LOOP Start with asking the right questions
Questions to help faculty focus their assessment activitiesWhat should students learn?How well are they learning it?What evidence do you have?What are you doing with the evidence?
ASSESSMENT TRAIL
Question
Plan
Collect & Score
Analyze/Reflect
Report/Act
Assessment Cycle
ASK THE “RIGHT” QUESTION The right question is:
Meaningful – it is a question that faculty want to know the answer to, and that knowing the answer will help them impact student learning
Measurable – work at asking a question that faculty can answer – usually that means narrowing down the question
Manageable – it is important to keep the question and the process of collecting data manageable – this isn’t the only or primary job that faculty have
ASSESSMENT FOR LEARNING Once faculty has determined the actual
question that is going to be answered this will drive the methodology of assessmentPre/Post TestEmbedded test questionsProject Based Assessment PortfoliosSurveysPerformance, etc., etc.
How often will they collect the data, in what classes – What makes sense? (Remember keep it manageable)
DATA COLLECTION – NOW WHAT First encourage faculty to take time to
organize the dataExcel is an easy accessible tool available to
99% of all facultyOffer Data management workshops
What does on a row, column, what are some analyses you can run in Excel
QUALITATIVE DATA ANALYSIS Qualitative Data is an appropriate tool
for many disciplinesMap out the requirements of the
assignmentSearch for themes in student responsesTrack how often key curricular themes
appear
STATISTICS - SMATISTICS If you are a IR person – or big on Stats –
close your ears for a minute… Rather than worry so much about
reliability, validity, statistical significance, think about these assessment projects as Action Research
Action Research focuses on getting information that will enable faculty to change conditions in a particular situation in which they personally involved Seeks to solve a problem, or Inform local practice – specifically –
classroom/course practice!
STEPS IN ACTION RESEARCH Identify the Research Question Gather the necessary information Analyze and interpret the information Develop an action plan
Sound familiar?? – Think Cycle of Assessment
LOOKING AT THE DATA Move beyond “Averages” Look at the Spread of the Data
What does the spread indicate? Is the data evenly distributed?Are there large gaps? Where do they exist?
Has the faculty member or department decided what is an acceptable level of performance?
WHEN TO CHANGE CURRICULUM – WHEN TO MAKE OTHER CHANGE?Curricular Change Follow the data trail – and then talk to
invested participantsData leads to students not understanding a
concept What is the benchmark of performance? Curriculum mapping – where does the concept
occur? How is the concept taught (pedagogy) Where is the concept reinforced? (Scaffolding) What changes can we (faculty/department)
make to the curriculum to help students understand and apply concept
How will we measure this curricular change to see if it is successful
WHEN THE DATA SUPPORTS OTHER CHANGES
Case Study – Visual CommunicationsGeneral Education Curriculum Assessment
Cycle Visual Communications
Mass Exodus of Visual Communications Classes
Faculty couldn’t come up with acceptable assignments to show how they were assessing visual communications
Focused discussion with department chairs/faculty groups on the issueFaculty felt unqualified to “teach” visual
communications Professional Development, in-service workshops,
teaching circles, etc.
MAKING THE CURRICULAR CHANGE If the focus throughout the process has
been on student learning (versus report writing), then faculty will be more open to making curricular changes
Next Assessment Cycle – what difference did the change make? Was there a difference in performance?Make sure appropriate time has elapsed for
changes to be in effectMake sure the measurement is parallel to
the previous assessment
ARE WE DONE YET?“CLOSING THE LOOP”
“When are we done with a learning outcome?”
Did you see improvement?
Did you meet your benchmark
performance?
Are you satisfied?
Do you see a greater need/question that
needs to be asked?
RANDOM THOUGHTS
KEEP THE FOCUS ON WHAT MATTERS MOST – STUDENT LEARNING
Reports and data analysis that don’t focus on student learning are a waste of paper
Faculty must be engaged in making sense of and interpreting assessment results – administration can’t do it for them
Share SuccessesWith permission – share assessment results
from other departments/disciplinesGet the faculty to tell their success stories –
it carries more weight with their peers
MAKE REALISTIC PLANS TO AVOID PITFALLS Effective Assessment Takes time to Plan,
Implement and SustainDon’t expect instant results – reliable data
takes time to gather
Make sure the assessment is asking a question that can be answeredThis means being narrow in scope – rather
than throwing as much stuff as possible at a wall and seeing what sticks!
DON’T FORGET TO ENGAGE ADMINISTRATION While it is important for faculty to “own”
assessment, support needs to come from administration:Recognizing faculty effortsAttending faculty functionsProviding appropriate resourcesConsidering policy implications
Questions??
Sheri H. Barrett, EdDDirector of Outcomes AssessmentJohnson County Community [email protected]
913-469-7607