assessing online instruction at carteret community college
DESCRIPTION
“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.” Oosterhof, et al. (2008) Assessing Learners Online. Assessing Online Instruction at - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/1.jpg)
“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”
Oosterhof, et al. (2008) Assessing Learners Online
![Page 2: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/2.jpg)
Assessing Online Instruction at Carteret Community College
Donald Staub, Title III Director Carteret Community College
?
?
?
?
?
?
![Page 3: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/3.jpg)
The Lineup
• DL at CCC
• The Need to Assess
• How We’re Doing It
• Conclusions & Solutions
• Discussion
![Page 4: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/4.jpg)
DL @ CCC• DL is 10 years old
• 34 programs @ CCC
• ~1700 enrollment SP09
• Fall 2008• 348 total sections taught
• 270 seated sections
• 61 Internet
• 15 Hybrid
![Page 5: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/5.jpg)
The Need to Assess
• SACS
• Common Sense• it’s the right thing to do
• we need some answers
![Page 6: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/6.jpg)
“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”
(CS 3.3.1)
COC Distance Education Policy Statementhttp://www.sacscoc.org/pdf/081705/distance education.pdf
![Page 7: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/7.jpg)
DL @ CCC• 270 seated sections
• 73% success / 22% failed or withdrew (15% W)
• 61 internet sections• 65% success / 33% D, F, WD (16% W)
• 15 hybrid sections• 81% success / 18% D, F, WD (9% W)
• FA07 internet: 59 sections; 34% D, F, W
• FA06 internet: 48 sections; 34% D, F, W
![Page 8: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/8.jpg)
How We’re Doing It
• QAP
• Grade Distributions
• ILLOs & PLLOs
• Same instructor
![Page 9: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/9.jpg)
The QAP(Quality Assessment Plan)
http://ccctitle3.wordpress.com/
• Developed in-house (using models)
• Peer-reviewed by content specialist
• Stipends
• All current online courses have been reviewed
• All new online courses must be reviewed
![Page 10: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/10.jpg)
http://ccctitle3.wordpress.com/
The QAP Score Card
![Page 11: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/11.jpg)
The QAP Score Card: Learning
![Page 12: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/12.jpg)
Grade Distributions(Fall 2008)
TOTAL ENROL A B C D F WD WD*
Seat 3643 0.31 0.26 0.14 0.72 0.04 0.07 0.06 0.09 0.26IN 1159 0.32 0.22 0.12 0.65 0.05 0.12 0.08 0.08 0.33HY 188 0.48 0.22 0.11 0.81 0.04 0.05 0.02 0.06 0.18
![Page 13: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/13.jpg)
ILLOs
• Communication
• Computer Literacy
• Critical Thinking
• Humanities & Fine Arts
• Information Literacy
• Personal Growth & Responsibility
![Page 14: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/14.jpg)
Computer Literacy - CIS 110Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers.
Assessment: Final Exam in CIS 110. This exam consists of a hands-on skills assignment for Word, Excel, Access and PowerPoint and a 25 question Multiple choice assignment with questions regarding computer hardware, system software, internet browsers and other related computer terminology.
![Page 15: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/15.jpg)
Computer Literacy 5 Sections: CIS 110 3-WB; 2-IN
• Attrition: IN01: 35% IN02: 26% WB03: 17% WB04: 28% WB05N: 18%
• Cross-section comparison (i.e. one course, different instructors)
![Page 16: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/16.jpg)
Student Name Word pts. (20%)
Excel pts. (15%)
PPT pts. (10%)
Access pts. (15%)
Integration pts. (15%)
Theory pts. (25%)
Final Score 75%
A 19 14 10 12 15 15 85 1B 17 14 10 10 13 21 85 1C 17 12 10 15 15 25 94 1D 10 7 0 0 0 12 29E 15 15 10 14 15 16 85 1F 19 15 10 15 15 23 97 1G 16 15 10 12 15 20 88 1H 19 15 10 15 15 24 98 1I 20 12 10 10 12 22 86 1J 20 12 10 13 15 25 95 1K 10 15 10 15 8 20 78 1L 19 15 10 15 15 24 98 1M 15 15 10 15 15 16 86 1N 14 11 10 12 15 24 86 1
Average 16.42857 13.35714 9.285714 12.35714 12.92308 20.23077 8582.14% 89.05% 92.86% 82.38% 86.15% 80.92%
% of students >=75 92.86%
![Page 17: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/17.jpg)
Word pts. (20%)
Excel pts. (15%)
PowerPoint pts. (10%)
Access pts. (15%)
Integration pts. (15%)
Theory pts. (25%) Final Score
IN01 80 73.8 95.7 64.7 83 81 79IN02 91 95 100 79 94 81.9 89WB03 91 75.5 96 78 88.5 80.8 84.6WB04 92.5 89 100 85 92.7 77 80.9WB05 82 89 92.8 82 86 80.9 85
![Page 18: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/18.jpg)
Computer Literacy - CIS 110Same instructor teaching one section of each
Spring 2008 online <75% seated <75%# of Ss @ 10% 24 16Final # of Ss 16 12Attrition 25% 25%Word (20) 15.75 9 17.8 5Excell (15) 12.3 5 12.2 5PPT (10) 6 16 6.3 11Access (15) 13.4 6 12.75 6Integration (15) 13.25 4 13.8 4Theory (25) 19.29 3 20.5 3Final (100) 73.3 6 81.2 6
![Page 19: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/19.jpg)
Personal Growth & Responsibility ACA 115
Outcome: Students demonstrate the ability to understand and manage self effectively.
Points Understanding Self ~~ Grading Criteria PointsEarned
30Points
(6 pts. each)
1. Identify long term and short term educational goals: a. Two (2) Long Term: 1 year—5 years b. Three (3) Short Term: 1 week—16 weeksCompleted Goal Development Form must be submitted.
20Points
(5 pts. eachfor results + 5pts. total forexplanation)
2. Evaluate the results of the following inventories in relation to your educational goals: a. Career Inventory b. Learning Styles Inventory c. Personality InventoryResponses should be written in paragraph form using completesentences and appropriate grammar.
20Points
(4 pts. pervalue with
explanation)
3. Identify five values that shape your belief system. Explain why these values are important to you.
Responses should be written in paragraph form using completesentences and appropriate grammar.
Points Managing Self ~~ Grading Criteria PointsEarned
15Points
(3 pts. each)
Advisement Status Sheet1. Placement Test or SAT Scores2. Distance Learning Orientation3. Keyboarding Test4. Computer Lit eracy Test5. Advisor/Program Coordinator Information
Completed Advisement Status Sheet must be submitted.
15Points
(5 pts. each)
Two Semester Class Schedule a. Complete the curriculum check sheet for your Program of Study (Major) by checking courses currently enrolled in and indicating the grade for courses completed. College Preparation courses should be listed on a separate piece of paper and attached to the check sheet. b. Create a trial course schedule for Spring 2009. c. Create a trial course schedule for Summer 2009 or for Fall 2009.
![Page 20: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/20.jpg)
Personal Growth & Responsibility ACA 115
![Page 21: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/21.jpg)
Personal Growth & Responsibility ACA 115
Advisement Project
• In the fall of 2008, 79% of all ACA students (10 sections) scored 80 or above.
• For online sections (2 sections), 70% (7 out of 10) scored 80 or above.
![Page 22: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/22.jpg)
Personal Growth & Responsibility PSY 150
1
Missing,inadequate ormarginallycompleted
2
Acceptable oradequately metrequirements
3
Exceptional orexceededrequirements
Pointsearned
Studentdeveloped anaction plan forpersonal growthor responsibility.
No action plan.Hint of actionplan withoutspecifics.Plan does notrelate to selfhelp book.Lacks timeline.
Action plan hasspecifics andrelates to selfhelp book.Action plan hastimeline.
Action planmeets acceptablestandards as wellas includesadditionalresearch beyondself help book.
Studentidentifiedobstacles whichcould preventgrowth orchange andidentified how toovercome saidobstacles.
Obstacles notidentified.Only oneobstacleidentified.Obstaclesidentified butlacks how toovercome them.
Student identifiesat least oneobstacle and planfor overcomingobstacle, butlacks depth.
Identifiedmultipleobstacles anddetermined howto overcomethem.
Studentanalyzed howhis or her lifewould beimproved by thischange.
Failed todemonstrateawareness ofhow life will beimproved byproposedchange.
Student mentionschange, butanalysis andexplanation arecursory.
Demonstratedawareness ofhow life will beimproved byproposed change.
Row 1 2+ 3/3Sec 04 7/10 (70%) 2/10 (20%)IN06 4/6 (66%) 1/6 (16%)IN07 4/6 (66%)
Row 2 2+ 3/3Sec 04 9/10 (90%) 5/10 (50%)IN06 1/6 (16%)IN07 5/6 (83%) 3/6 (50%)
Row 3 2+ 3/3Sec 04 7/10 (70%) 5/10 (50%)IN06 3/6 (66%)IN07 6/6 (66%) 2/6 (33%)
![Page 23: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/23.jpg)
PLLOsSame instructor teaching one section of each (take a look at a common assignment from each section)
• ACA 115
• ACC 120
• BUS 110
• ECO 251
• EDU 119
![Page 24: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/24.jpg)
ACA 115 -
Learning Styles Project
<75%Instructor 1 WB02 WB05 IN10
18% 41% 36%
Instuctor 2 WB01 IN096% 24%
Students identify and explain the strategy used to prepare for an exam.
![Page 25: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/25.jpg)
ACC 120
![Page 26: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/26.jpg)
BUS 110
ECO 250
![Page 27: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/27.jpg)
EDU 119
4 test test items related to: working/communicating with parents, parental rights, and regulations dealing with parental contact/information.
Item # # incorrect– online
# incorrect- seated
4 1 021 0 022 2 223 1 1
n=20 n=12
![Page 28: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/28.jpg)
Conclusions & Solutions
“The largest issue confronting online assessments is that of the dynamics that occur when the instructor and student do not share the same immediate space.”
Milam, et al. (2004). In, Developing and Implementing Assessment of Student Learning Outcomes
![Page 29: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/29.jpg)
• Instructors who don’t treat online & seated equitably (e.g. additional support to seated sections)
• Using the observation instrument correctly. Some of this was a direct result of them using a document that was scanned and they had to print/copy to use. I believe some of the students had trouble with the process. (EDU 119)
• Math & the fax machine
![Page 30: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/30.jpg)
“The seated students did however have the opportunity to submit their observation checks sheets for revision if needed.”
“Students will no longer have to print the inventory results. Instead, they will write a narrative explaining their results.”
“Students will no longer need to submit a curriculum check sheet (on-line students had difficulty accessing them from the website).”
![Page 31: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/31.jpg)
Discussion
“In many instances, the assessment of an online course is not in alignment with the type of instruction that may have occurred in the course.”
Palloff & Pratt (2009) Assessing the Online Learner
![Page 32: Assessing Online Instruction at Carteret Community College](https://reader034.vdocument.in/reader034/viewer/2022051623/56815ae7550346895dc8a9fc/html5/thumbnails/32.jpg)
Donald Staub Title III Director
Carteret Community [email protected]
252- 222-6010
http://ccctitle3.wordpress.com/