evaluation of learning resources
DESCRIPTION
TRANSCRIPT
![Page 1: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/1.jpg)
Evaluation of Learning Resources
Damian Gordon
![Page 2: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/2.jpg)
There is a plethora of learning materials of all sorts now available on-line.
The range of ways in with these materials are produced and consumed means that there cannot be one single way to evaluate these materials.
However there is a major issue with quality control with regard to this material.
The Quantity versus Quality issue
Continued:
![Page 3: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/3.jpg)
A great deal of material is developed without any evidence of it ever having being tested on learners.
A logical way to evaluate the quality of learning materials is to try it out on learners and get their feedback (both formative and summative).
It perhaps make most sense to submit this learner feedback in conjunction with the learner materials to verify that the materials have been tested by learners.
The Quantity versus Quality issue
![Page 4: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/4.jpg)
There are two main evaluation tools for learning materials◦ MERLOT◦ LORI
Two slightly less used tools are◦ MicroSoft Evaluators Guide◦ National Council for Educational Technology for
evaluating CD-DROM
Evaluation Tools
![Page 5: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/5.jpg)
MERLOT (www.merlot.org) is a repository containing educational resources classified into seven broad subject categories: Arts; Business; Education; Humanities; Mathematics and Statistics; Science and Technology; Social Sciences.
MERLOT
![Page 6: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/6.jpg)
There are three general categories of evaluation standards used within MERLOT: 1. Quality of Content 2. Potential Effectiveness as a
Teaching-Learning Tool 3. Ease of Use
MERLOT
![Page 7: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/7.jpg)
Quality of Content There are two general elements to quality of
content:1. Does the software present valid (correct)
concepts, models, and skills? 2. Does the software present educationally
significant concepts, models, and skills for the discipline?
MERLOT
Continued:
![Page 8: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/8.jpg)
Quality of Content To evaluate the educational significance of
the content, reviews use the following guidelines:
MERLOT
Continued:
![Page 9: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/9.jpg)
Quality of Content ◦ Content is core curriculum within the discipline. ◦ Core curriculum topics are typically covered to
some degree in the introductory classes within the discipline and/or "Everyone teaches it" and/or it is identified as a core area by the discipline's professional organizations
◦ Content is difficult to teach and learn. ◦ Content is a pre-requisite for understanding more
advanced material in the discipline
MERLOT
![Page 10: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/10.jpg)
Potential Effectiveness as a Teaching-Learning Tool
Evaluating POTENTIAL effectiveness is asking the Peer Reviewer to judge, based on his expertise as a teacher, whether the learning material is likely to improve teaching and learning given the ways the faculty and students could use the tool.
MERLOT
Continued:
![Page 11: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/11.jpg)
Potential Effectiveness as a Teaching-Learning Tool
What stage(s) in the learning process/cycle could the materials be used? ◦ Explanation or description of the topic/stating the
problem ◦ Demonstration of the curriculum/exploration of the
problem ◦ Practice using the curriculum/analysis of the outcomes
from solving the problem ◦ Applying the curriculum to "new" problems/application
of the outcomes to other problems
MERLOT
Continued:
![Page 12: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/12.jpg)
Potential Effectiveness as a Teaching-Learning Tool
What is(are) the learning objective(s)? What should students be able to do after successfully learning with the materials?
What are the characteristics of the target learner(s)
MERLOT
Continued:
![Page 13: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/13.jpg)
Potential Effectiveness as a Teaching-Learning Tool
There are other general elements to effectiveness as a teaching-learning tool that MERLOT asks reviewers to consider:◦ Does the interactive/media-rich presentation of material improve
faculty and students' abilities to teach and learn the materials? ◦ Can the use of the software be readily integrated into current
curriculum and pedagogy within the discipline? ◦ Can the software be used in a variety of ways to achieve
teaching and learning goals? ◦ Are the teaching-learning goals easy to identify? ◦ Can good learning assignments for using the software
application be written easily?
MERLOT
![Page 14: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/14.jpg)
Ease of Use The basic question underlying the ease of use
standard is: how easy it is for teachers and students to use the software for the first time?
Elements that affect ease of use include:
MERLOT
Continued:
![Page 15: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/15.jpg)
Ease of Use◦ Are the labels, buttons, menus, text, and general layout of the
computer interface consistent and visually distinct? ◦ Does the user get trapped in the material? ◦ Can the user get lost easily in the material? ◦ Does the module provide feedback about the system status and
the user's responses? ◦ Does the module provide appropriate flexibility in its use? ◦ Does the learning material require a lot of documentation,
technical support, and/or instruction for most students to successfully use the software?
◦ Does the material present information in ways that are familiar for students?
◦ Does the material present information in ways that would be attractive to students?
MERLOT
![Page 16: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/16.jpg)
eLera is a distributed group that researches and evaluates e-learning. Their specific interests include learning objects, e-portfolios, and learning design specifications, and related topics.
eLera provides tools and information for learning object evaluation and research, maintains a database of learning object reviews, and supports communication and collaboration among researchers, evaluators and users of online learning resources.
LORI from eLera
![Page 17: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/17.jpg)
The goals of eLera are to:◦ improve the quality of online learning resources
through better design and evaluation◦ develop effective pedagogical models that
incorporate learning objects◦ help students, teachers, professors, instructional
designers and others to select pedagogical models and digital resources that meet their requirements
LORI from eLera
![Page 18: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/18.jpg)
The Learning Object Review Instrument (LORI) is used to evaluate the quality of e-learning resources. LORI is an online form consisting of rubrics, rating scales and comment fields. The current version of LORI available from eLera is version 1.5.
http://www.elera.net/eLera/Home/Articles/LORI%201.5.pdf
LORI from eLera
![Page 19: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/19.jpg)
Nine Dimensions of LORI1. Content Quality2. Learning Goal Alignment3. Feedback and Adaptation4. Motivation5. Presentation Design6. Interaction Usability7. Accessibility8. Reusability9. Standards Compliance
LORI from eLera
![Page 20: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/20.jpg)
Nine Dimensions of LORI
1. Content Quality: Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail
LORI from eLera
![Page 21: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/21.jpg)
Nine Dimensions of LORI
2. Learning Goal Alignment: Alignment among learning goals, activities, assessments, and learner characteristics
LORI from eLera
![Page 22: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/22.jpg)
Nine Dimensions of LORI
3. Feedback and Adaptation: Adaptive content or feedback driven by differential learner input or learner modelling
LORI from eLera
![Page 23: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/23.jpg)
Nine Dimensions of LORI
4. Motivation: Ability to motivate and interest an identified population of learners
LORI from eLera
![Page 24: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/24.jpg)
Nine Dimensions of LORI
5. Presentation Design: Design of visual and auditory information for enhanced learning and efficient mental processing
LORI from eLera
![Page 25: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/25.jpg)
Nine Dimensions of LORI
6. Interaction Usability: Ease of navigation, predictability of the user interface, and quality of the interface help features
LORI from eLera
![Page 26: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/26.jpg)
Nine Dimensions of LORI
7. Accessibility: Design of controls and presentation formats to accommodate disabled and mobile learners
LORI from eLera
![Page 27: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/27.jpg)
Nine Dimensions of LORI
8. Reusability: Ability to use in varying learning contexts and with learners from differing backgrounds
LORI from eLera
![Page 28: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/28.jpg)
Nine Dimensions of LORI
9. Standards Compliance: Adherence to international standards and specifications
LORI from eLera
![Page 29: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/29.jpg)
9 items rated on a 5-point scale
![Page 30: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/30.jpg)
LORI Review process
diagram here is showing individual reviews merging to a panel review and then published on web.
![Page 31: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/31.jpg)
MicroSift Evaluators Guide INSTRUCTIONAL QUALITY The purpose of the package is well
defined The package achieves its defined
purpose Presentation of content is clear and
logical The level of difficulty is appropriate to
the target audience Graphics/colour/sound are used for
appropriate instructional reasons Use of package is motivational The package effectively stimulates
student creativity Feedback on student responses is
effectively employed The learner controls the rate and
sequence of presentation and review Instruction is integrated with previous
student experience Learning is generalisable to an
appropriate range of situations
CONTENT The content is accurate. The content has educational value The content is free of race, ethnic,
sex, and other stereotypes
TECHNICAL QUALITY The user support materials are
comprehensive The user support materials are
effective Informative displays are effective Intended users can easily and
independently operate the program
Teachers can easily employ the package
The program appropriately uses relevant computer capabilities
The program is reliable in normal use
![Page 32: Evaluation of learning resources](https://reader033.vdocument.in/reader033/viewer/2022061120/546cbc7cb4af9f7a2c8b517b/html5/thumbnails/32.jpg)
National Council for Educational Technology for evaluating CD-DROM
1. Which computer system will the disc run on? 2. Will your computer system do justice to the illustrations? 3. Is the operation by keyboard or mouse, or both? 4. Can we have the disc for a trial period? 5. Is the language and spelling on the disc Queen’s English or American
English? 6. How much bias is there in the content of the disc? 7. Is printing out easy and intuitive? 8. Can the selected material readily be down-loaded to disc? 9. Can subsections of the disc be searched? 10. Is the software to control the CD-ROM on the disc itself or is it supplied on a
separate floppy disc? 11. Does the software manage memory resources well? 12. What search procedures are available? 13. What is the language level on the disc? 14. Is the user interface tolerant of typing and spelling errors? 15. Can you select exactly what you want to print out or save to disc? 16. Are there any supporting features? 17. Can the illustrations be printed out? 18. Can images be readily transferred? 19. Is there a sound capability to accompany the pictures?