![Page 1: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/1.jpg)
1
“When learners are at a distance, the careful design of assessments is particularly important, because society somewhat unfairly imposes higher expectations on the assessment of online learners.”Oosterhof, et al. (2008) Assessing Learners Online
![Page 2: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/2.jpg)
2
Donald Staub, Title III Director Carteret Community College
?
?
?
?
?
?
Assessing Distance Learning: Are they learning what we think we’re teaching?
![Page 4: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/4.jpg)
4
The Lineup• DL @ CCC
• Why Worry about DL Assessment?
• How We’re Doing It
• The DL Program
• DL in Programs & Courses
• Possibilities & Challenges
![Page 5: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/5.jpg)
5
DL @ CCC• 33 programs @ CCC
• FA10 enrollment ~1800
• DL is 10+ years old
• FA10 336 total sections 243 seated sections (72%) 64 Internet (20%) 29 Hybrid (8%) 58% FT / 42% PT
5
![Page 6: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/6.jpg)
6
Why worry about assessing DL?
Student Success Persistence(btw: success vs. retention vs. persistence)
“Good assessment guides good PD/ID”
@ the College & Program-Level:
![Page 7: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/7.jpg)
7
@ the Course-Level:
Generalizability(c.f. Oosterhof, et al, 2008)
“…the degree to which learner performances that are observed generalize to performances that are not observed.”
[i.e. are they learning what we think we’re teaching?]
![Page 8: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/8.jpg)
8
So, how are we doing this?
Caveat: We’re not saying this is right or wrong…it’s just the way that we’re currently approaching
it.
![Page 9: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/9.jpg)
9
“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”
(CS 3.3.1)
COC Distance Education Policy Statementhttp://www.sacscoc.org/pdf/Distance%20and%20correspondence%20policy%20final.pdf
![Page 10: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/10.jpg)
10
The (former) Guidelines
![Page 11: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/11.jpg)
11
![Page 12: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/12.jpg)
12
2009-2012 Strategic Plan
We will create a culture where all institutional actions are focused on improving student learning and success and leading to student goal completion.
Q1: Is the distance education program reflected in the institution’s strategic (or master) plan?
PM 16—Success and Withdrawal Rates by Instructional modalities
![Page 13: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/13.jpg)
13
Q2: Is there evidence that outcomes for the program have been identified?
DL-AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner.
DL-AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.
![Page 14: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/14.jpg)
14
AO #1: Faculty demonstrate the ability to teach in the online environment in a proficient (student centered) manner.
Assessment: Quality Assessment Plan, Success/ Persistence Rates, Student Surveys/Evaluations.
AO #2: Students have more educational options to pursue degrees, diplomas, and certificates due to expanded online course offerings at CCC.
Assessment: The number of internet/hybrid sections offered.
Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program?
![Page 15: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/15.jpg)
15
QAP
![Page 16: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/16.jpg)
16
The QAP(Quality Assessment Plan)
• Developed in-house (using models)
• Peer-reviewed by content specialist
• Stipends (T3)
• All current online courses have been reviewed
• All new online courses must be reviewed/ certified
http://web.carteret.edu/keoughp/TitleIII/FinalQAP.xls
![Page 17: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/17.jpg)
17
The QAP Score Card
![Page 18: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/18.jpg)
18
The QAP Score Card: Learning
![Page 19: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/19.jpg)
19
QAP
Success/Persistence
![Page 20: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/20.jpg)
20
Success - Spring 2010
• 75 sections
• seated: 75% success (persist: 87.1%)
• internet: 66% success (persist: 81.7%)
• hybrid: 73% success (persist: 87.8%)
• FA09 internet: 62 sections; 68% success
• FA08 internet: 61 sections; 67% success
• FA07 internet: 59 sections; 66% success
• FA06 internet: 48 sections; 66% success20
![Page 21: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/21.jpg)
21
QAP
Success/Persistence
Program/Course Evaluations
![Page 22: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/22.jpg)
22
Surveys & Evaluations: DL Program
![Page 23: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/23.jpg)
23
Surveys & Evaluations: DL Program
23
![Page 24: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/24.jpg)
24
Surveys & Evaluations: Course-Level
![Page 25: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/25.jpg)
25
Q3: Is there evidence that the effectiveness of the distance education program is regularly assessed and steps taken for improvement of the program?
Modality Comparison Grade Distributions Program Reviews ILLOs (Gen Ed outcomes)
![Page 26: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/26.jpg)
26
Modality Comparison
![Page 27: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/27.jpg)
27
Modality Comparison
Institutional-level
![Page 28: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/28.jpg)
28
Modality Comparison
Program-level
![Page 29: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/29.jpg)
29
Modality Comparison
Course-level
![Page 30: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/30.jpg)
30
Modality Comparison
Grade Distributions
![Page 31: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/31.jpg)
31
Grade Distributions institution-level
%s
![Page 32: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/32.jpg)
32
Grade Distributions course-level
![Page 33: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/33.jpg)
33
Modality Comparison
Grade Distributions
Program Reviews
![Page 34: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/34.jpg)
34
Evaluate Instructional Modalities
![Page 35: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/35.jpg)
35
Modality Comparison
Grade Distributions
Program Reviews
ILLOs
![Page 36: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/36.jpg)
36
ILLOs (Gen Ed outcomes)
1. Communication
2. Computer Literacy
3. Critical Thinking
4. Humanities & Fine Arts
5. Information Literacy
6. Personal Growth & Responsibility
![Page 37: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/37.jpg)
37
Computer Literacy - CIS 110
Outcome: Students demonstrate proficiency in the use of computer technology including application software, system software, and internet browsers.
Assessment: Final Exam in CIS 110.
![Page 38: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/38.jpg)
38
Computer Literacy - CIS 110 Section analysis
![Page 39: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/39.jpg)
39
Computer Literacy - CIS 110Same instructor teaching one section of each
![Page 40: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/40.jpg)
40
![Page 41: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/41.jpg)
41
Personal Growth & Responsibility ACA 115
Outcome: Students demonstrate the ability to understand and manage self effectively.
Results Spring 2009: 131 total projects submitted; 81% scored 80 or above.
Seated 115: 85% scored 80 or above Online 115: 81% scored 80 or above
Assessment: Advisement assignment in ACA 115 & 118 evaluated by rubric.
![Page 42: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/42.jpg)
42
Q4: Has the institution developed student learning competencies for the courses/programs offered by distance education? If these are the same competencies for courses/programs offered by “traditional” methodologies, is assessment identified for distance learning students separate from students taking courses by “traditional” methodologies?
![Page 43: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/43.jpg)
43
“Comparability of distance education programs to campus-based programs and courses is ensured by the evaluation of educational effectiveness, including assessments of student learning outcomes, student retention, and student satisfaction.”
So, are they learning what we think we’re teaching?
![Page 44: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/44.jpg)
44
Yes! … No?…maybe??
• PLLOs
• Course-Level Assessment
And the answer is…
![Page 45: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/45.jpg)
45
PLLOs
![Page 46: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/46.jpg)
46
PLLOs (SLOs)
Same instructor teaching one section of each (common assignment from each section)
Business Administration
• ACC 120
• BUS 110
• ECO 251/252
![Page 47: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/47.jpg)
47
PLLO: preparation of financial statements as part of mid-
term Assessed in: ACC 120
47
![Page 48: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/48.jpg)
48
BUS 110
ECO 250
![Page 49: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/49.jpg)
49
PLLOs
Course-Level Assessment
![Page 50: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/50.jpg)
50
Course-level assessment - CLLOs
• ACC 120
• ART 111
• EDU 119
• ENG 111
• HIS 111
• PSY 150 & 241
• RCP 114
• SOC 21050
![Page 51: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/51.jpg)
51
Course-level assessment ~ CLLOs
ACC 120 … Students will be able to:
1. Illustrate the steps in the accounting cycle.
2. Compute cost of goods sold and ending inventory using various pricing methods.
3. Prepare a multiple step income statement, retained earnings statement and classified balance sheet.
![Page 52: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/52.jpg)
52
“Students placed more value on the CMS as an effective learning tool when they perceived the CMS as supporting interactivity among student, instructor, peers, content.” (Chou, Peng, Chang, 2010)[as quoted by Cook, G. & Boldt, S.]
![Page 53: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/53.jpg)
53
ART 111
Grade Evaluation ( Percentages and Points )
AssignmentCategory
Number ofAssignments
% of ea.towardFinalGrade
Total pointsgenerated by ea.
individualassignment
Total % ofCategory inFinal Grade
Total Pointsof Category
in Final grade
WeeklyQuizzes
14 2.5% ea. 80 35% 1,120
DiscussionBoard
16 2.5% ea. 80 40% 1,280
Mid TermEssay
1 10% ea. 320 10% 320
Final Exam 1 15% ea. 480 15% 480
Totals 32 100% 3200
![Page 54: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/54.jpg)
54
A - (100 - 90): - You posted 3 or more times and you completed the Assignment with a well thought through response. (The more you post MEANINGFUL responses over 3 the higher your grade. Though welcome and expected, please understand that a One or Two sentence "feel good" post such as (... Great response Michaela!!) or (... I love how you put the meaning and then went down and talked about them. it made it easy to understand!!) will count for very little towards your grade.
54
![Page 55: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/55.jpg)
55
HIS 111
Essays (online) vs. Research Paper (seated) … same rubric
![Page 56: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/56.jpg)
56
RCP 114
![Page 57: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/57.jpg)
57
Other possibilities
• Downstream assessment (course or clinical)
• E-ports
57
![Page 58: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/58.jpg)
58
Challenges
![Page 59: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/59.jpg)
59
“I got rid of [the discussion board]…it’s a pain in the ass.”
“I have to pull teeth to get them to post even when they get a score”
“Value of discussion board is suspect at best.”
![Page 60: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/60.jpg)
60
“My first goal is to get the instructors to agree on common CLLOs. Then we’ll work on common assessments.”
Can you say Parasympathemimetic?
“I don’t know” [what texts and assessments the other instructors are using in this multi-section course]
Outcome-Assessment alignment
![Page 61: Donald Staub, Title III Director Carteret Community College](https://reader030.vdocument.in/reader030/viewer/2022012922/56813e72550346895da88529/html5/thumbnails/61.jpg)
61
Donald Staub Title III Director
Carteret Community [email protected]
252- 222-6010
http://ccctitle3.wordpress.com
www.carteret.edu