leveraging analytics and data to promote faculty
TRANSCRIPT
Leveraging Analytics and Data to Promote Faculty Engagement in Student Success Partnerships
Nikki Glenos, Director of Advising TechnologyDr. Stephany Dunstan, Assistant Vice Provost for Assessment
Dr. Carrie Zelna, Associate Vice Chancellor, DASA
March 2, 2020
NC State: Institutional Context
• ~36,000 undergraduate and graduate students
• Decentralized advising model
• Over 700 academic advisors
• Variability in structure/models creates challenges to encouraging strong adoption of new analytics
Why Mini-Grants?• Wealth of untapped valuable institutional data
• Support and encourage faculty-led, focused institutional research that encourages data-informed decision making and interventions
• Encourage systematic analysis of data related to student success
• Continue to develop a positive relationship with DASA and faculty
• Opportunities to develop new partnerships
• Increase usage of GPS Institution Reports
What is Student Success GPS?
• Suite of tools combining technology, analytics, advising services, and real-time data
• Impacts on student success, retention, and graduation
• Analytics suite: Institution Reports– Up to 10 years of historical student data– Gain insights into patterns of student
performance & factors impacting graduation
Proposals
• Collaboration with Office of Assessment and Office of Advising Technology
• Call for proposals: Directors of Undergraduate Programs, Associate Deans, Curriculum Committees
• 2019- Awarded 5 mini-grants for $2,000 each
• Changes for current mini-grant cycle
Proposals
Leverage Institution Reports tool in conjunction with additional campus data sources to analyze topics related to student success within a program/department, including:
● Early intervention indicators● Trends and patterns impacting retention and graduation● Barriers to progression in curriculum● Challenges in course sequencing● Grade distributions and predictive cutoffs for student success● Major changing trends and patterns● Top predictor courses for additional student support
Proposal Evaluation
Assessment & Advising Technology team reviewed proposals using the following criteria for selection:
● Project Need
● Aims, objectives, and measurable outcomes
● Description of the assessment methods that would be used in the project
● Use of findings for program improvement
● Appropriateness of proposed expenditures
Initial Consultations
Assessment & Advising Technology team met with grant recipients in late spring 2019 to:
● Discuss data confidentiality protocol
● Create plan for sourcing the data (OA/GPS)
● Provide overview/preliminary training using the platform
● Further discuss plan of analysis and use of data
● Timeline for completion
Follow-Up Meetings
Assessment & Advising Technology team met with grant recipients in early summer 2019 to:
● Discuss progress/challenges
● Review use of platform
● Discuss student record data and suggestions for use
● Pull additional data as needed
Example Case
Department of PsychologyDr. Daniel Gruehn, Director of Undergraduate Programs
Dr. Dana Kotter-Gruehn, Director of Undergraduate Advising
Overarching Research Question:What patterns of performance exist around courses that either pose
barriers to retention and graduation or are early indicators of top-performing students?
Example Case
Course of interest: PSY 230 Research Methods in Psychology
• Required course for Psychology majors and minors
• Critical foundational course for the successful progression in program.– Students are encouraged to take this class early on in their academic career
• Course may be both a barrier and an early indicator for high performance
Example CaseSub-Research Questions: Grades in PSY 230
a) Are low grades in PSY 230 - maybe in conjunction with low grades in required biological science courses (BIO 105, BIO 106, or BIO 181) and mathematics courses (e.g., MA 114, MA 107, MA 121 or higher) - a predictor for poor retention in the major, low graduation rates, and low GPA?
b) Are low grades in PSY 230 - maybe in conjunction with low grades in required biological science courses (BIO 105, BIO 106, or BIO 181) and mathematics courses (e.g., MA 114, MA 107, MA 121 or higher) - a predictor of low grades in required higher-level courses in Psychology, such as PSY 420 Cognitive Psychology and PSY 430 Biological Psychology?
Data available to answer these questions: Aggregate data within GPS for these particular courses; Can view predictive cutoffs, data of grades earned for these particular classes, DFW rates, and average lifetime earned credits when attempted. When looking at individual course effects and timing of courses, supplemental data comes from Office of Assessment.
Example CaseSub-Research Questions: Timing of ST 311 and PSY 230
Identifying potential buffer effects: the timing of taking ST 311 Introduction to Statistics (which is a required class) in relation to PSY 230.
• Most students take PSY 230 before ST 311; students who take ST 311 before PSY 230 are able to apply concepts in PSY 230
1. Does it matter whether ST 311 is taken before, concurrent, or after PSY 230 for retention, graduation, and GPA?2. Does it matter whether ST 311 is taken before, concurrent, or after PSY 230 for grades in required higher-level
courses in Psychology, such as PSY 420 Cognitive Psychology and PSY 430 Biological Psychology?
Data available to answer these questions: Data from GPS in advanced search if looking at a few semesters of data. For longitudinal data, Office of Assessment provides student record data.
Example Case- Findings
• Predictive course rank
• Cumulative GPA
• Time to graduation
• PSY 420/430
• Timing of ST 311
Example Case- Action Plan1. Assessment practices:
a. Examine SLOs in PSY 230b. SLOs across Psychology curriculum (curriculum mapping)c. Deeper analysis of PSY 230
2. Advising practices:a. Timing of PSY 230 and ST 311b. PSY 230 should remain as a preferred requirement in the CODA process.c. Proactive engagement for students who struggle/do well in PSY 230
i. Advising interventionsii. Peer tutorsiii. Advanced course options; undergraduate research; honor societies
Moving Forward
● Adjust final report template based on observations from 2019 cycle
● Host a mini-grant showcase to highlight the project findings and action plans
● Continue relationship to sustain use of data and outcomes
● Encourage use of findings at SLO level when appropriate
● Integrate co-working sessions into mini-grant process
Questions/Comments?
Contact Us
Dr. Carrie ZelnaAssociate Vice Chancellor
Dr. Stephany DunstanAssistant Vice Provost for Assessment
Nikki GlenosDirector of Advising Technology