QUANTITATIVE LITERACY PROGRAM

RAJ B OPPANAMARY DIXSONKIM MASSAROGAIL PIZZOLA

KIMBERLY WARD

2015 QLP Workshop

Agenda

9:30 – 9:50am Introduction & Purpose of QLP Rajendra Boppana

9:50 – 10:10 am QLCDS – Data Submission Process Kim Massaro

10:10 – 10:20 am Q-course Logistics Kimberly Ward

10:20 – 10:40 am Incorporating QL into MAT 1043 Jonathan Brucks

10:40 – 11:00 am Incorporating QL into WRC 1013/1023 Gail Pizzola

11:00 – 11:30 am Analysis of Q-course Results Rajendra Boppana

11:30 – 12:45 am Lunch Break

12:45 – 1:05pm Incorporating QL into CRJ 3013 Rob Tillyer

1:05 – 1:25 pm Incorporating QL into KIN 3323 Sakiko Oyama

1:30 – 2:30 pm Discussion with the Provost Dr. Frederick

2:30 – 3:00 pm Wrap Up and Questions QLP Team

Location: JPL Faculty Center, Assembly Room

QLP Team

Dr. Mary DixsonQLP Implementation/ Training Coordinator

Kimberly WardQLP Program Coordinator

Dr. Gail PizzolaQLP Implementation/ Training Coordinator

Dr. Rajendra BoppanaProject Director of QLP

Kim MassaroQLP Program Coordinator

Prajan PradhanQLP Data Specialist

Robin Schulze QLP Coordinator & Analyst

Quantitative Literacy Program (QLP) Program Goals

Develop quantitative skills in undergraduate students

Implement effective teaching pedagogies and assessments to support the development of an exemplary quantitative scholarship program at the undergraduate level

Provide the organizational framework and resources for an institutional transformation to graduate a quantitatively informed citizenry

Q-Course

Current Course QLP Q-Course

• Learn Quantitative Skills

• Think critically• Interpret and use

data that naturally exist in the subject area

• Make informed decisions

• Makes the course more engaging

Data + Q. Methods + Redesign

Student Participation in QLP

Completion of one or more Q-courses is a graduation requirement

Started with core courses, expanded to major-required upper division courses

QLAT Advising Q-courses QLATGraduatio

nStudent

s

Faculty Participation in QLP

167 faculty members, 150+ TAs/graders participated in QLP since Fall ‘11

QLP Timeline

Year 1 First cohort of Q-faculty and students; 10 Q-courses Program website is created Every incoming freshman takes the entrance QLAT Course data is collected

Year 2 Developed online QLAT entrance exam Workshop (QLW) is created to address core complete

transfer students Individual faculty and overall Q-course reports are

developed Surveys of students begin

QLP Timeline (contd.)

Year 3 QLP maximized its enhancement of core courses QLP invites upper division courses for redesign Surveys to faculty, department chairs, and advisors begin Exit QLAT is administered to compare to the baseline Develops online version of QLW Workshop for transfer

students QLP awards first Faculty Excellence Award

Year 4 Data Collection process is streamlined 8 upper division courses are enhanced with QL Surveys to employers and alumni begin QLP awards second Faculty Excellence Award

QLP Growth

Year One (2011-12)

Year Four (2014-15)

20 faculty 100 faculty

10 Q-Courses 27 Q-Courses

113 Sections 556 Sections

6,845 enrollments 26,599 enrollments

2015-16 Q-course List

ANT 2033ANT 2043ARC 4183ARC 4283BIO 1233BIO 1404COM 3073CRJ 3013ECO 2003ECO 2013ECO 2023ENG 2413ES 2013

HIS 2123 HIS 2133 KIN 3323MAT 1043MDS 4983PHI 1043POL 1013POL 1113

SOC 1013SOC 3323SPE 3603STA 1053WRC 1013WRC 1023

27 Q-courses 19 core, 8 upper division

Student Coverage: First-time, Full-time Students

Completion of a Q-course by year by freshmen cohortsEach colored segment in a bar represents one year

Student Coverage: Transfer Students

Completion of a Q-course by transfer student cohortsEach colored segment in a bar represents one year

Student Coverage: Graduating Students

QLC: Completed at least one Q-courseQLW: Completed a 3-hour workshop instead of a Q-courseQLE: Exempted based on major; Q-course not completed

Student Enrollments in Q-Courses

Fall 2015 enrollments are based on Aug 13 2015 data

Q-Course Performance Analysis (Fall 2014)

12 out of 19 (63%) of core level Q-Courses showed significant increase from pre to post-test Out of those 12, 11 courses reported an average score

greater than 70 on post-test.

5 out of 8 (63%) of upper-division Q-Courses showed significant increase from pre to post-test Out of those 5, 4 courses reported an average score

greater than 70 on post-test.

Q-Course Performance Analysis (Spring 2015)

13 out of 18 (72%) of core level Q-Courses showed significant increase from pre to post-test Out of those 13, 10 courses reported an average score

greater than 70 on post-test.

6 out of 8 (75%) of upper-division Q-Courses showed significant increase from pre to post-test Out of those 6, all 6 courses reported an average

score greater than 70 on post-test.

Presenter: Kim Massaro

QLP Data Submission Process

Pre/Post Test

Give the Pre-test before any Q-material is taught

Grade the Express QuestionBubble in the score for the Express question

on the Parscore for each studentTake Parscores to Testing ServicesMake sure to include a note: give permission

to upload data to the QLP drive

How to Bubble Express

Large Form: SUBJ Score

5

5

Small Form: Exam #

QLCDS Website

qlcds.it.utsa.edu (open using Mozilla Firefox) Log on with your abc123 and password

Download your courses’ template Course coordinators will create the templates at the

beginning of the semester

Upload the item level data

Course Coordinators

• Create pre-test template– Upload Pre-test document with SLO’s and taxonomy– Upload rubric for the Express Question with Answer

Key– Upload a dummy file to generate SLO’s

• Create Homework template– Upload Q-assignment with SLO’s, taxonomy, and

answers

• Create post-test template

Best Practices

For Course Coordinators: Create a course specific “primer” for faculty teaching

Ex: Materials handbook, Blackboard Learn shell, packet, etc…

Meet with new faculty prior to beginning of semester

Create QLCDS templates during first month of semester

When uploading materials, include SLOs, taxonomies, and correct answers clearly marked on document/rubric.

Best Practices (contd)

For New Faculty teaching Q for first time: Attend recommended trainings with the QLP team

Meet with course coordinator and other “Q” team members

Contact Testing Services for ParScore training (optional)

Visit QLP program website (http://qlp.utsa.edu) for more information on the program, workshop materials, video presentations, tutorials, and technical reports.

Best Practices (contd)

For All Q-Faculty Contact the Course Coordinator at beginning of each

semester Bookmark the following websites:

http://qlp.utsa.edu QLP program website http://qlcds.it.utsa.edu Data Collection Website (very

important!) http://qlp.utsa.edu/faculty (Resources include workshop

materials, technical reports, and data collection tutorials) https://medialibrary.utsa.edu/Brwose/Category/57 (video

presentations from QLP staff and other Q-faculty) Make sure TA/Grader attends QLP Training Workshop

(8/28) or schedules one-on-one training Verify that TA/Grader has all documents, rubrics, and

understands the needs of the course.

Presenter: Kimberly Ward

Q-Course Logistics

Q-Course Logistics

Beginning of the semester1. Course Coordinator creates course template for pre-test,

assignment, and post-test on QLCDS website

2. Faculty register pre/post-test with Parscore at Testing Services

3. Faculty meet with TA/Grader and provide documents and rubrics for grading Q materials.

4. Faculty give Blackboard gradebook access to TA/Grader (optional)

5. All Q-Faculty submit pre-test data by Wednesday Sept. 9th Must state “Give permission for Testing Services to send results to

QLP”

Q-Course Logistics (contd.)

Middle/End of the semester1. Faculty or TA/Grader downloads “Homework” Excel

template from QLCDS website

2. Faculty or TA/Grader enters student roster information and itemized student scores in Excel template columns.

3. Faculty or TA/Grader uploads the completed Excel template and indicates scoring criterion for each question.

4. “Q” Homework data is submitted to QLCDS by Friday Dec. 4th

5. Post-test ParScore forms are dropped off at Testing Services by Friday Dec. 4th

1. Unless on Final Exam, then due Tuesday Dec. 15th (Grades Due Deadline)

Q-Course Logistics (contd)

Should there be problems/missing data after submission, the QLP will email the faculty member to resolve the issue.

1st Email – Individual Faculty

2nd Email –Individual Faculty and Course Coordinator

3rd Email—Individual Faculty, Course Coordinator, and Department Chair

Murphy’s Law and QLP

When things go wrong/errors happen:TA/Grader Assigned FacultyFaculty Course CoordinatorCourse Coordinator QLP Team

Email the QLP team at qlp@utsa.edu NOT individual team member emails

QLP Data Submission Process & Due Dates

Pre-test Data (via Parscore) Due September 9th (one week after census)

Homework Data (for Upper-division courses only) Due December 4th (study days)

Post-test Data (via Parscore) Due December 4th (study days)

Introduction to MathematicsPresenter: Jonathan Brucks

Incorporating Quantitative Literacy into MAT 1043

Freshman Composition I and IIPresenter: Gail Pizzola

Incorporating Quantitative Literacy into WRC 1013/1023

Presenter: Rajendra Boppana

Analysis of Q-course Results

Will resume at 12:45pm

Break for Lunch

Agenda

12:45 – 1:05pm Incorporating QL intoCRJ 3013

Rob Tillyer

1:05 – 1:25 pm Incorporating QL intoKIN 3323

Sakiko Oyama

1:30 – 2:30 pm Provost’s Discussion Dr. Frederick

2:30 – 3:00 pm Wrap Up and Questions QLP Team

Research Design and AnalysisPresenter: Rob Tillyer

Incorporating Quantitative Literacy into CRJ 3013

BiomechanicsPresenter: Sakiko Oyama

Incorporating Quantitative Literacy into KIN 3323

With Dr. John Frederick UTSA Provost and Vice President for Academic

Affairs

Round Table Discussion

Thank you for your participation.