using technology to close the loop on student learning assessment - atp europe 2012
DESCRIPTION
Presentation given by Dr. Mary Beth Hanner, provost and chief academic officer of Excelsior College, and Dr. Patrick Jones, vice provost, at the 2012 Association of Test Publishers Conference in Berlin, Germany. Hanner, and Jones’ presentation, “Using Technology to Close the Loop on Student Learning Assessment” focused on the variety of emerging tools on the market, best practices, and future trends in assessments.TRANSCRIPT
![Page 1: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/1.jpg)
Using Technology to Close the Loop on Student Learning AssessmentMary Beth Hanner, Ph.D, RN, ANEF Patrick Jones, Ph.D.Provost and Chief Academic Officer Vice Provost
Excelsior College, Albany, NY - USA
![Page 2: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/2.jpg)
Purpose
o Describe context and framework for student learning assessment at a post-traditional degree completion institution.
o Provide examples of key assessments conducted at institutional, program and course levels.
o Show how technology is used to improve the rigor and effectiveness of student learning assessments.
![Page 3: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/3.jpg)
Background
o Excelsior College founded in 1971 (previously Regents College).
o In 1998, chartered as a private, independent college.
o Large adult student body (approximately 33,000 students).
o Distance education, flexible learning format.
Philosophy:
“What you know is more important that where or how you learned it”
![Page 4: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/4.jpg)
Backgroundo Valuing prior learning is a central premise for adult learners
(Green, 1998).o Acceptance of a variety of credit sources:
Examples:• Regionally accredited institutions• ACE/PONSI approved courses• Exams – CLEP, DANTES,EC• Portfolio assessment for prior learning• Online courses from Excelsior and preferred providers
o Residency requirement (presently minimal, historically none).
![Page 5: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/5.jpg)
Background
o Accreditation
• Middle States (all programs)• NLNAC (specialized accreditation - Nursing)• ABET (specialized accreditation - Technology)• IACBE (specialized accreditation - Business)
![Page 6: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/6.jpg)
o Career oriented working adult; no 1st time, full-time students.
o Transfer credits: 25-75% of degree program.
Challenge:Who is a typical student?
Average age: 39 year
College 1
College 4
College 3
College 2
College 5
Excelsior College At a distance
![Page 7: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/7.jpg)
Accountability and Continuous Improvement
OAIR Steering Committee
OAIR Office
School ofBusiness &
Tech
School of Liberal Arts
School of Nursing
School of Health
Sciences
Institutional Assessment Plan for Student Learning (IAPSL)
![Page 8: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/8.jpg)
Institutional Assessment Plan for Student Learning (IAPSL) & Institutional
Effectiveness Plan (IEP)
o Provide frameworks for assessment of learning outcomes at course, program and institutional level and institutional effectiveness, respectively.
o Address the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011).
o IAPSL and IEP links:www.excelsior.edu/IAPSL
www.excelsior.edu/IEP
![Page 9: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/9.jpg)
Establish clear learning outcomes
Use the Results Provide learning opportunities
Assess Student Learning
Assessment Process at Schools
![Page 10: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/10.jpg)
Institutional Assessments
o ETS Proficiency Profile™: internet-based test to measure general education outcomes
o Internet-based alumni & supervisor surveys to assess student satisfaction & career progression
o National Survey of Student Engagement: internet-based measure of baccalaureate student participation
![Page 11: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/11.jpg)
Institutional Assessments
o Priorities Surveys for Online Learners: web-based measure of student satisfaction
o Service Quality Survey: internet-based assessment of student perceptions of Excelsior College service quality
![Page 12: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/12.jpg)
Program-level Assessments
o Licensure and certification exams for professional degree programs (e.g., NCLEX® for nurses, Uniform CPA Examination for accountants)
o Capstone courses and assessments for all graduates
o Alumni and supervisor surveys as indirect measures of program outcome mastery
![Page 13: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/13.jpg)
Course-level Assessments
o Discussion posts, assignments, quizzes, exams, simulations and essays administered in BlackBoard 9.1 Learning Management System
o Credit-by-exams administered at Pearson VUE testing centers
![Page 14: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/14.jpg)
Course-level Assessments
o Credit for prior learning (i.e., portfolio assessment) to recognize college-level learning achieved through independent study, workplace training and other means
![Page 15: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/15.jpg)
Compiling & Reporting Assessment Results
o Challenge: frequency and scope of assessment activities yield large amounts of data
o Need technology tool to organize and report assessment results for decision makers
o TracDat™ selected as software tool for data storage and reporting
![Page 16: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/16.jpg)
Example of TracDatTM Report
Outcomes Means of Assessment and Criteria
Results Action and Follow-up
Analyze legal, ethical and policy issues within health care delivery systems.
HSC 431 Project Analysis of Oregon Health Plan
Criterion: 75% of the students will receive a grade of B or better on this project.
57% (28/49) of students received a grade of B or higher – Needs improvement
Area of concern - problem area was student’s inability to propose new approaches to effectively control health care costs.Assessment modified to compare and contrast Oregon plan with another selected state.
School of Health Sciences
![Page 17: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/17.jpg)
Technology Tools to Increase Student Persistence
o Challenge: faculty need mechanisms to track student progress in courses and intervene with students who are not persisting
o Starfish Early Alert™ system integrated in Excelsior courses
![Page 18: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/18.jpg)
Technology Tools to Increase Student Persistence
o System identifies students who are not persisting based on login activity, failing to submit assignments and/or low grades
o Faculty receive notifications of students who exhibit these behaviors and intervene to improve persistence
![Page 19: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/19.jpg)
Technology Tools to Verify Student Identity
o BlackBoard 9.1 requires student user name and password to access courses
o Excelsior has piloted use of Acxiom challenge question system to verify student identity
![Page 20: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/20.jpg)
Technology Tools to Verify Student Identity
o Acxiom system uses student’s name, birth date and address to select and administer multiple-choice questions (e.g., Which of the following was your address in 2008?)
o Access to course content and assessments limited to students who answer challenge questions correctly
![Page 21: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/21.jpg)
Technology Tools to Detect Plagiarism
o Excelsior uses Turnitin.com to perform an originality check of student essays and papers to detect plagiarism
o Turnitin.com compares student work to databases of papers and publications to determine if plagiarism is evident
![Page 22: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/22.jpg)
Future Directions
o Increased use of biometrics for student identity verification
o Integration of adaptive models for instruction and assessment
o Reliance on open educational resources to improve access
![Page 23: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/23.jpg)
Questions?
Thank you for attending!
![Page 24: Using Technology to Close the Loop on Student Learning Assessment - ATP Europe 2012](https://reader038.vdocument.in/reader038/viewer/2022110115/54b807b74a795971058b4681/html5/thumbnails/24.jpg)
Thank You!
Please Complete Your Evaluation.