tech-driven education reform: a model for simultaneously improving student retention and performance...
TRANSCRIPT
1
An ExamSoft Client Webinar
Tech-Driven Education Reform: A Model
for STEM Disciplines
2
Tech-Driven Education Reform: A Model for Simultaneously Improving
Student Retention and Performance in STEM Disciplines
Rob Petros, Ph.D. | Assistant Professor, UNT
Department of Chemistry
3
4
5
6
Roadmap of Presentation
• The state of STEM/recent results • Course data and information • NextGen concepts • Chemistry Redesign
– Engagement – Learner Centered and Strategies
• Questions and Feedback
7
Dr. Rob Petros: Chemistry The Problem
• Only 40% of entering college students that declare STEM majors complete degrees in STEM disciplines
• Economic projections indicate 1 million more STEM professionals will be needed in the US than will be produced over the next decade at current rates
Why are we losing these students? 1. Lack of inspiration in low level science classes 2. Frustration with the ‘weed out’ courses 3. STEM graduate programs prepare researchers
not educators
8
Problem Solved!? • Redesign has increased enrollment from 145 to 192
(32%), while reducing actual class size from 145 to 96 ($120K+ in annual tuition).
• Retention rate has increased dramatically ($250K?)
* first year incorporating engaged learning activities (peer group learning)
• Percent effort for teaching course has decreased dramatically
# of students receiving D, F, or W
# of students registered
retention rate (%)
2013/2014* 39 377 90 2012/2013 83 285 71 2011/2012 83 291 71 2010/2011 87 248 65 2009/2010 77 218 65
9
Student Performance During Redesign
0
10
20
30
40
50
60
May-‐11 May-‐12 May-‐13 May-‐14† May-‐14‡ May-‐14*
% of class
Results from ACS Standardized Final Exam
Above Natl. Average BoLom quarNle
† All students taking second semester with Petros
‡ Students that were in Petros' section for both OChem I and II * Students that were in Petros' section for only OChem II
podcasts introduced
lecture eliminated
NextGen launched
10
• A 10 question quiz covering basic concepts from OChem I was administered on the first class day of the second semester.
• Students enrolled in Petros’ first semester course performed significantly better.
• Remediation activities are being developed for transfer students.
Student Preparedness for OChem II
N = 172 Former Petros students – 97 Students transferring in – 75
0 10 20 30 40 50 60 70 80
Pretest
Aver
age
Scor
e OChem I w/Petros
OChem I w/someone else
11
12
Student Learning Outcomes
• What the student is expected to know and be able to do – Useful in the classroom
• To students • To Faculty
– Connects to “larger” goals • Program • Institution • Accreditation
13
NextGen Course Redesign
Course ObjecNves
Assessment* InstrucNonal Strategies
*Carriveau, R. S., Connecting the dots : developing student learning outcomes & outcome based assessments. Fancy Fox Publications:
Denton, TX, 2010.
14
1. Goal: The student will understand literature (by) 1.1. General Learning Outcome (GLO): The student will demonstrate
accurate, critical, analytic reading of literature (by) specific learning outcome statement (sLO): The student will (be able to) 1.1.1 Identify important and supporting details 1.1.2 Recognize assumptions and inferences 1.1.3 Identify sequence of events 1.1.4 Determine the main idea/theme of a passage or
piece of literature
Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
Example of Three Level SLO Structure
15
Code each item to a specific learning outcome
1 Goal: The student will understand literature (particular period or genre) 1.1 General Learning Outcome (GLO): The student will demonstrate
accurate, critical, analytic reading of literature. Specific learning outcome statements (sLO): 1.1.1 Determine the main idea/theme of a passage or piece of literature. Test Item or Rubric Dimension 1.1.1 What was this passage mostly about? (Could also be CR) A. Bias and prejudice can affect intellectual growth.* B. Economic growth is impacted by bias and prejudice. C. Current thoughts on bias and prejudice lack insight.
Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
16
The Link Between Outcomes and Items Calculating Outcome Attainment Values
Goal 1
Avg = 83
Source: Carriveau, R.S. (2011), Connecting the Dots: Developing Student Learning Outcomes and Outcome-Based Assessments
1. 87 2. 90 3. 65 4. 58 5. 63 6. 52 7. 66 8. 77 9. 84 10 93 11. 96 12. 88 13. 82 14. 88 15. 90 16. 80 17. 92 18. 81 19. 81 20. 82
Specific Outcome 1.1.1
Avg = 81
Specific Outcome 1.1.2
Ave = 60
Specific Outcome 1.1.3
Avg = 88
Specific Outcome 1.2.1
Avg = 85
Specific Outcome 1.2.2
Avg = 85
General Outcome 1.1
Avg = 81
General Outcome 1.2
Avg = 85
17
ExamSoft “Categories”
Use the “Categories” feature to create three levels of outcome statements
18
Writing OChem Course Objectives
• What are the essential principles every student should take away from the course?
• Who are your students and what are you preparing them for?
• Can your course teach important lessons beyond your own discipline-specific content?
• How will you assess student attainment?
19
Course Objectives
• Goal 1 – Students will understand molecular structure and its implication for basic chemical reactivity
• Goal 2 – Students will understand the concept of isomerism in organic chemistry
• Goal 3 – Students will understand various mechanistic pathways commonly encountered in organic chemistry
• Goal 4 – Students will understand how spectroscopy can be used to determine molecular structure
• Goal 5 – Students will understand the relationship of organic chemistry to their everyday lives
• Goal 6 – Students will be aware of the impact of the globalization of scientific research on US competitiveness in science and technology and be motivated to pursue and obtain degrees in STEM majors
• Goal 7 – Students will demonstrate elements of collaboration, leadership, innovation, problem solving, creativity, teamwork, and critical thinking
20
Objectives and Professional Standards
21
22
23
Online Content
252 podcasts totaling ~23h of recorded material produced covering all topics from both semesters posted on UNT’s iTunesU site
24
STEM Incentives Program
• Information on graduate school in the sciences
• Careers in chemistry • Where my classmates are now • Need for diversity in STEM • Science and Engineering Indicators 2012 • Activity demonstrating the globalization of
scientific research – List authors’ home institution for first 20
articles in the first issue of JACS in 1982, 1992, 2002, and 2012
25
STEM Incentives Program
26
Engaged Learning Activities
*Office of Science and Technology Policy – Engage to Excel (2012)
27
Class Time – Small Group Work
• 192 students divided into 32 groups of 6 • Each group meets 2hr/wk • Can observe skills related to course goals 5-7
28
Group Assignments
29
30
Formative Assessment On Demand
31
More Robust Analysis
Learning Objective # Assessments # Items Group Average 1. Students will understand molecular structure and its implication for basic chemical reactivity
11 66 73.36%
1.1 Students will know and apply the naming system for organic compounds
11 23 71.06%
1.1.1 Students will recognize and correctly name molecules containing functional groups
9 23 71.24%
1.1.2 Students will correctly name stereoisomers 2 1 70.56% 1.2 Students will understand the relationship between structure, hybridization, resonance, and aromaticity
2 21 81.80%
1.2.1 Students will identify the correct hybridization state for C, N, O and other relevant atoms
2 18 81.19%
1.2.2 Students will identify factors that lead to stabilization in resonance structures
2 3 84.86%
1.3 Students will understand the role of acidity⁄basicity in reactions
4 23 70.67%
1.3.1 Students will predict products of an acid⁄base reaction
2 2 73.89%
1.3.2 Students will identify acid⁄base conjugate pairs 2 6 89.73% 1.3.3 Students will use pKa values to predict relative acidity
4 15 66.86%
32
Tracking Student Learning in Real Time First Half of Semester
Second Half of Semester
Number of Items
Group Average
Number of Items
Group Average
1.1.1 Students will recognize and correctly name molecules containing functional groups 10 91% 5 97%
1.3.3 Students will identify acid⁄base trends for common functional groups 10 82% 2 69%
2.1.1 Students will recognize the mechanisms for electrophilic and nucleophilic aromatic substitution reactions
2 38% 1 84%
2.1.2 Students will predict the site of substitution based on substituents present on the aromatic ring 2 44% 1 85%
2.1.5 Students will predict products of reactions given specific starting materials 2 80% 12 87%
3.1.1 Students will predict reaction products based on starting materials 35 76% 29 83%
3.1.2 Students will differentiate between reversible and irreversible addition reactions 8 87% 4 93%
5.2.1 Students will analyze spectral data to predict molecular structure 12 84% 9 88%
33
Rob Petros – Improved from 56% to 71%
Fourth Exam Performance (split into two section)
Remediation Assignment Performance (sections combined)
34
Rob Petros – Early intervention
Student Feedback • Student feedback
released on learning outcome performance and areas of strength/weakness
• Displayed score
upon exam exit
35
Engagement in the Class • The purpose of the UNT Student Engagement Inventory
(SEI) is to provide information about student engagement that instructors can use to make meaningful continuous improvements to their course.
• Part 1, Personal Internal Motivation, is a measure of the
students’ personal internal predisposition to be engaged in learning.
• Part 2, Classroom Interaction is a measure of the students’ perception of engagement based on interactions in the teacher-created classroom learning environment.
• Part 3, Social Interaction, is a measure of the social interaction among classmates in interactive classroom experiences.
36
Review
• Identify relevant student and faculty data related to STEM course redesign
• Describe effective, learner focused instructional practices • Describe the connection between outcome based education and
improvement of instruction • Describe outcome based assessment strategies • Identify basic milestones for success • Discuss how various assessment and instructional best
practices can be applied to a variety of settings and institutions • Develop strategies for implementing instructional and
assessment best practices at course, department and institutional level.
• Develop a basic outline of ideas for incorporating outcome based, learner centered instruction and assessment into their own courses, programs, or institutions.
