1ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Conference
#ASSESS2015
2ExamSoft Assessment Conference | #ASSESS2015
Coming Full Circle: Tips & Tricks from Test Item Creation to Assessment Data Retrieval
3ExamSoft Assessment Conference | #ASSESS2015
Introduction
Western University of Health Sciences – Pomona, CA
4ExamSoft Assessment Conference | #ASSESS2015
Learning Objectives
• Discuss methods of engaging faculty members in appropriately tagging questions to Institutional Learning Outcomes (ILOs), Program Learning Outcomes (PLOs), & relevant topics.
• Understand the process in pulling informative longitudinal data using the tagged categories to derive meaningful data relevant to student performance/progress.
• Identify strengths and weaknesses of the curriculum.• Identify strengths and weaknesses of student
performance.
5ExamSoft Assessment Conference | #ASSESS2015
6ExamSoft Assessment Conference | #ASSESS2015
Our Previous Process
• Word document/RTF import• Problems with category tagging• Technical issues with import• Ineffective reporting
7ExamSoft Assessment Conference | #ASSESS2015
OUR CURRENT PROCESS
8ExamSoft Assessment Conference | #ASSESS2015
Landmark #1
9ExamSoft Assessment Conference | #ASSESS2015
Test Item Collection
• Question Export using Item Analysis feature– Point Bi-Serial– Question ID/Rev– Upper/Lower– Disc Index
• Creation of Question Bank
10ExamSoft Assessment Conference | #ASSESS2015
Test Item Collection
• Benefits– User Friendly– Quality Assurance– Collaboration
Platform– Organize Data
11ExamSoft Assessment Conference | #ASSESS2015
Test Item Collection
• Question Submission Form– Purpose: To collect questions from multiple faculty and
house in one central location– Structure:
• Question type• Instructor• Question text and indicate correct answer• Attachment (any file type accepted,
allows for hi-res images)• Lecture title• Location of content in lecture• Randomize answers & group questions
(Optional)
12ExamSoft Assessment Conference | #ASSESS2015
Test Item Collection
• Category TaggingTRICK! Clean up your categories by regularly organizing them in ExamSoft at the end of each term.
13ExamSoft Assessment Conference | #ASSESS2015
Test Item Collection
• Form interacts with Question Bank
• Increases efficiency for faculty & staff
• Simplified method of tagging categories that engages faculty members
• Ease of use promotes faculty buy-in
• Deadline CalendarTIP! Hold
development
workshops to
train faculty on
how to tag
consistently.
14ExamSoft Assessment Conference | #ASSESS2015
Landmark #2
15ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Creation
• Data from the Question Submission Form is extracted as an ExamSoft template Excel file
• Staff converts spreadsheet to .txt file for import into ExamSoft
TRICK! Eliminate special character errors by using the“Find & Replace” feature in Microsoft Excel.
16ExamSoft Assessment Conference | #ASSESS2015
ExamSoft Assessment Creation
• Staff create assessment in ExamSoft portal• Staff export an exam draft to SharePoint
– Faculty members collaborate on reviewing/making necessary edits and corrections prior to final approval
17ExamSoft Assessment Conference | #ASSESS2015
Landmark #3
18ExamSoft Assessment Conference | #ASSESS2015
Generating Reports
• Pull reports & post to SharePoint for review:– Exam Taker Results– Item Analysis– Summary Report– Category Reports
TIP! Maintain a location that
houses question bank, exam
draft, and exam reports for faculty to access.
19ExamSoft Assessment Conference | #ASSESS2015
Generating Reports
• Faculty recommend key changes– Review item analysis/question metrics– Improve test items in question bank
• Score reports are released via ExamSoft• Scores uploaded to
students’ web-based academic profile.
20ExamSoft Assessment Conference | #ASSESS2015
Landmark #4
21ExamSoft Assessment Conference | #ASSESS2015
Reporting Longitudinal Data
• Mid-Term & End-of-Term• Pulled from ExamSoft
22ExamSoft Assessment Conference | #ASSESS2015
Reporting Longitudinal
• Uses of Data (Faculty, Staff, & Administration):– Academic Dean: University Strategic and
Effectiveness Report– Course Directors/Managers: Evaluate student
performance in courses– Curriculum Design Committee: Consider changes to
curriculum– Advisors: Pinpoint areas of weakness for student advisees
23ExamSoft Assessment Conference | #ASSESS2015
Conclusion
Benefits
• Streamline• Efficiency• Accountability• Technical• Reporting• Organization
Improvements to Workflow
• Staff time– Previous Process: 4 hours– Current Process: 30 minutes
• More manageable Question Bank
• Quicker turnaround time for assessment creation
24ExamSoft Assessment Conference | #ASSESS2015
Conclusion
• Coming Full Circle• Updating questions • Curriculum changes• Students track
progress from term-to-term and compare to class average. Beneficial for targeted learning.
• Lessons Learned• Questions?
25ExamSoft Assessment Conference | #ASSESS2015
Contact Information
Jayzona Alberto, MS
Academic Manager
(1st & 3rd Year Curriculum)
Jorge Godinez, BS
Academic Manager
(2nd & 4th Year Curriculum)