how do we know it works? evaluating learning technology projects educause learning initiative...
TRANSCRIPT
![Page 1: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/1.jpg)
How do we How do we know it works?know it works? Evaluating Learning Technology Projects Evaluating Learning Technology Projects
EDUCAUSE Learning Initiative SeminarClare van den Blink
![Page 2: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/2.jpg)
Seminar Goals
Seminar participants will be able to….
Select project goals that can be
evaluated.Identify relevant
indicators to evaluate
project goals.
Identify data collection methods
Assess considerations for
developing evaluation activities
Create an evaluation plan for an academic technology
project that is tied to their project assumptions and
strategies.
![Page 3: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/3.jpg)
Please indicate the type of projects Please indicate the type of projects you’re interested in evaluatingyou’re interested in evaluating..
![Page 4: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/4.jpg)
What are the key challenges in fully evaluating your projects?
![Page 5: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/5.jpg)
A FrameworkProject GoalsFocus of EvaluationEvaluation DesignOverviewIndicators - measures of ”success”Data collection -Methods-Population-ProceduresTimelineData AnalysisReporting Findings
![Page 6: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/6.jpg)
Introduction
How this process was developed…
“To evaluate the effectiveness of the technology enhancement and it’s impact on student
learning”
…..With constraints of staff, time and budget limitations.
![Page 7: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/7.jpg)
Staff Effort
Complexity of Evaluation
Low
High
Small Project with technology
intervention
LMS Pilot Project-Service Decisions
Quasi-experimental Research on Tech
Intervention
Importance-Complexity
![Page 8: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/8.jpg)
Staff Effort
Complexity of Evaluation
Low
High
Small Project with technology
intervention
LMS Pilot Project-Service Decisions
Quasi-experimental Research on Tech
Intervention
Importance-Complexity
Interview
Survey
![Page 9: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/9.jpg)
Staff Effort
Complexity of Evaluation
Low
High
Small Project with technology
intervention
LMS Pilot Project-Service Decisions
Quasi-experimental Research on Tech
Intervention
Importance-Complexity
Interview
Survey
Interviews
Surveys
Tech Review
Usability
![Page 10: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/10.jpg)
Staff Effort
Complexity of Evaluation
Low
High
Small Project with technology
intervention
LMS Pilot Project-Service Decisions
Quasi-experimental Research on Tech
Intervention
Importance-Complexity
Interview
Survey
Interviews
Surveys
Tech Review
Usability
Interviews
Surveys
Observations
Control group
![Page 11: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/11.jpg)
To inform the evaluation
Based onPrior
research?
What led to the development of
the project?
Literature review-
What will inform the evaluation?
Assumptions about the
strategies & technologies
selected?
![Page 12: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/12.jpg)
Selecting Goals….Since not all projects may be evaluated within the timeframe of the project…
OR
may be difficult to measure…
![Page 13: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/13.jpg)
Selecting Goals….Since not all projects may be evaluated within the timeframe of the project…
OR
may be difficult to measure…
How can you SELECT goals that can be evaluated within the scope of the project?
![Page 14: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/14.jpg)
Selecting Goals….Since not all projects may be evaluated within the timeframe of the project…
OR
may be difficult to measure…
How can you SELECT goals that can be evaluated within the scope of the project?
How would you PRIORITIZE the goals that are the most critical to evaluate?
![Page 15: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/15.jpg)
Sample Goals:Review examples ofgoals and idenitfy goals that could be evaluated within the project constraints.
EXAMPLE #1Instructional Goals:1.) Encourage active participation and critical thinking skills by using video clips of main stream movies to initiate class discussions.2.) Encourage student involvement and active learning by creating a mechanism for students to record interviews in the field. (part of a class assignment)3.) Create a repository of student-collected audio interviews for ongoing use in the curriculum. Audio clips will be used to illustrate the diversity of public education experiences. Other Project Goals:.4.) Develop work flow and documentation for student recording of audio interviews and video clip processing. 5.) Choose, create, and provide an archiving mechanism for cataloguing clips.
![Page 16: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/16.jpg)
Sample Goals:Review examples ofgoals and idenitfy goals that can be evaluated within the project constraints.
EXAMPLE #2Instructional Goals:A. ) Students will be able to practice application of fluid therapy, under various conditions, employing a unique computer-based simulation B.) Students will be able to interpret symptoms presented in a sick dog, select an appropriate treatment, administer fluids, monitor patient’s reaction and modify treatment plan accordingly.C.) Case simulation will enable students to experience clinical variability in a manner similar to hands-on practice.Other Project Goals:D.) Simplify the creation of a set of teaching models, or prototypes, that are the basis of the cases.E.) Create a method for generating unique computer-based cases that build from the prototypes. F.) Provide a method for saving case data for comparison.
![Page 17: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/17.jpg)
![Page 18: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/18.jpg)
Developing Developing the the
evaluation evaluation planplan
![Page 19: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/19.jpg)
Evaluation Process
Identify Project Goals
Survey
Observation
Interviews
Focus Groups
Data analysis
Case Study
![Page 20: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/20.jpg)
Evaluation Process
Identify Project Goals
Focus of the Evaluation?What goals will be evaluated?
Survey
Observation
Interviews
Focus Groups
Data analysis
Case Study
![Page 21: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/21.jpg)
Evaluation Process
Identify Project Goals
Focus of the Evaluation?What goals will be evaluated?
Indicators: type of data to collect?
Survey
Observation
Interviews
Focus Groups
Data analysis
Case Study
![Page 22: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/22.jpg)
Evaluation Process
Identify Project Goals
Focus of the Evaluation?What goals will be evaluated?
Indicators: type of data to collect?
Methods: how will data be
collected?
Survey
Observation
Interviews
Focus Groups
Data analysis
Case Study
![Page 23: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/23.jpg)
Evaluation Process
Identify Project Goals
Focus of the Evaluation?What goals will be evaluated?
Indicators: type of data to collect?
Methods: how will data be
collected?
Survey
Observation
Interviews
Focus Groups
Data analysis
Case Study
Population
![Page 24: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/24.jpg)
Process: Select Goals …Focus
What goals will be the Focus of the evaluation?
Project Goals1.Use Personal Response System (polling) with questions to encourage critical thinking & student engagement.2.Use of PowerPoint presentations with interactive lecture.3.Implement use of a Tablet PC for annotating presentations for visual and interactive lectures
Evaluation Process
![Page 25: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/25.jpg)
Process: Select Goals …Focus
What goals will be the Focus of the evaluation?
What is feasible to evaluate in project’s timeframe?
Project Goals1.Use Personal Response System (polling) with questions to encourage critical thinking & student engagement.2.Use of PowerPoint presentations with interactive lecture.3.Implement use of a Tablet PC for annotating presentations for visual and interactive lectures
Focus of the Evaluation•Use Personal Response System (polling) with questions to encourage critical thinking & active participation.
(student engagement)
Evaluation Process
![Page 26: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/26.jpg)
Process: Select Goals …Focus
What goals will be the Focus of the evaluation?
What is feasible to evaluate in project’s timeframe?
Identify what INDICATORS can be used to collect data, both indirect and direct measures.
Project Goals1.Use Personal Response System (polling) with questions to encourage critical thinking & student engagement.2.Use of PowerPoint presentations with interactive lecture.3.Implement use of a Tablet PC for annotating presentations for visual and interactive lectures
Focus of the Evaluation•Use Personal Response System (polling) with questions to encourage critical thinking & active participation.
(student engagement)
Evaluation Process
![Page 27: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/27.jpg)
Evaluation Focus: Example 1Formative:
Since the project is assisting with the development of online modules, a formative evaluation of the modules will be conducted to look at the interface design, navigation, usability, organization and presentation of content, and the usefulness for student learning.The key focus will be on the “functionality” of the module.Interface / Navigation / Design (not important in phase I)Technology performance: test across browsers, OS, distance, etc.Organization/presentation of contentUse of images, illustrations, imagesLearning objectives**
Summative: (part of the overall program evaluation)Since the project is assisting with the development of web-based modules, the summative evaluation will examine the effect on student perception of learning from the implementation of instructional technology in the course.
Measures of success may include the student perception of greater ease in learning difficult concepts, and positive feedback about new modules.
Evaluation Process
![Page 28: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/28.jpg)
Select MethodsDevelop data collection METHODS for the indicators, such as surveys, interviews, observations, etc.
What method(s) would you select evaluate the focus area- “functionality” of the online module?1.Surveys2.Interviews3.Observation4.Log analysis5.All of the above6.Other – post in chat
Evaluation Process
![Page 29: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/29.jpg)
Select MethodsDevelop data collection METHODS for the indicators, such as surveys, interviews, observations, etc
What method(s) would you select to evaluate how well the online module met the “learning objectives”?1.Surveys2.Interviews3.Observation4.Log analysis5.All of the above6.Other – post in chat
Evaluation Process
![Page 30: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/30.jpg)
![Page 31: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/31.jpg)
Other considerations Identify the population from
where the data will be collected. Can you contact this group?
Identity other data sources, such as logs, document, etc.
Are there considerations for human subjects research & informed consent on your campus?For example, using grades as data, what permissions are necessary?
Evaluation Process
![Page 32: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/32.jpg)
Timelines: How much time do you have?
Need?
Dec-Jan Project Transition
& Closeout
Fall Project Development
Aug – DecFall Semester Project Implementation
Implement Project Evaluation
Project Evaluation Planning
Evaluation:Data analysis
& reports
Evaluation Process
![Page 33: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/33.jpg)
Evaluation Timeline Develop an initial Develop an initial timelinetimeline & & staffingstaffing effort effort. .
Develop evaluation plan. September 1
Create an observation protocol September 15
Observe EDU 271 during two class sessions Sept - October
Complete the student survey. (instrument) October 1, 2005
Create an interview protocol October 15
Administer student survey. Mid- November
Conduct student interviews End of November
Conduct data analysis December - January
Complete evaluation report February 1
Evaluation Process
![Page 34: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/34.jpg)
Evaluation Timeline Develop an initial Develop an initial timelinetimeline & & staffingstaffing effort effort. .
Develop evaluation plan. September 1
Create an observation protocol September 15
Observe EDU 271 during two class sessions Sept - October
Complete the student survey. (instrument) October 1, 2005
Create an interview protocol October 15
Administer student survey. Mid- November
Conduct student interviews End of November
Conduct data analysis December - January
Complete evaluation report February 1
Where reality meets ideal evaluation methods….
Evaluation Process
![Page 35: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/35.jpg)
![Page 36: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/36.jpg)
Implementing Implementing the Plan the Plan
& & Reporting Reporting Results Results
![Page 37: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/37.jpg)
Implementing Methods
Surveys:
Identify or develop questions.
Do survey questions map to indicators?
Survey distribution & associated permissions.
Interviews:
Develop interview questions & protocols.
Schedule and conduct interviews.
Resources about quantitative and qualitative methods can guide development and implementation of methods and data analysis.
Evaluation Process
![Page 38: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/38.jpg)
Analysis & ReportingDATA ANAYSIS
What type of analysis will be completed?
Quantitative: survey analysis
Qualitative: interview analysis based on interview protocols.
Example:
Overall, I am satisfied with the use of instructional technology in this course.
Mean = 1.21 Strongly Agree 82%2 Agree 18%3 Neutral 04 Disagree 05 Strongly Disagree 0
Evaluation Process
“I interviewed Prof. X about her experience with the email
simulation…..”
![Page 39: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/39.jpg)
Analysis & Reporting
Evaluation Process
![Page 40: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/40.jpg)
![Page 41: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/41.jpg)
ConstraintsConstraints
![Page 42: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/42.jpg)
StaffingHow can this planning all be completed within limited staff
hours, while maintaining the INTEGRITY of the evaluation process?
Evaluation Considerations & Tools
![Page 43: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/43.jpg)
Do you think it is feasible to re-train staff for evaluation?
![Page 44: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/44.jpg)
StaffingHow can this planning all be completed within limited staff hours, while maintaining the INTEGRITY of the evaluation
process?
How can staff be trained in this process without having a deep evaluation background?
What staff skills might be adapted?
Other campus resources?
Evaluation Considerations & Tools
![Page 45: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/45.jpg)
What type of existing skills could be adapted for evaluation?
![Page 46: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/46.jpg)
Supporting Tools Have an overall evaluation plan template that an be
adapted to other projects. Informed consent templates
Use common interview/observation protocols.
Develop question banks for survey questions. Use of Video in Course Presentations and lectures
Use of Online instructional tutorials
Use of Presentations in Lecture
Evaluation Considerations & Tools
![Page 47: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/47.jpg)
question banks
Evaluation Considerations & Tools
![Page 48: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/48.jpg)
question banks
Evaluation Considerations & Tools
![Page 49: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/49.jpg)
SummarySummary
![Page 50: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/50.jpg)
Consider… How can this METHODOLOGY be applied to your projects and
institution?
How VIABLE is this as an evaluation methodology for your projects?
When does a more ROBUST process need to be put in place?
![Page 51: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/51.jpg)
A FrameworkProject GoalsFocus of EvaluationEvaluation DesignOverviewIndicators - measures of ”success”Data collection -Methods-Population-ProceduresTimelineData AnalysisReporting Findings
![Page 52: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/52.jpg)
A Framework1. Project Goals2. Focus of Evaluation3. Evaluation Design
Overview Indicators - measures of ”success” Data collection
- Methods- Population- Procedures
4. Timeline: what, when, how5. Data Analysis6. Reporting Findings
![Page 53: How do we know it works? Evaluating Learning Technology Projects EDUCAUSE Learning Initiative Seminar Clare van den Blink cv36@cornell.edu](https://reader035.vdocument.in/reader035/viewer/2022070308/551c074b550346a34f8b4f27/html5/thumbnails/53.jpg)
[email protected]@cornell.edu