Blended, Just-in-Time Math: Faculty GuideTable of Contents

About the Project.........................................................................................................................................................3

Participating in the Evaluation Study.......................................................................................................................4

Inviting Students to Give Feedback......................................................................................................................4

Inviting Students to Participate in Data-Sharing for Research Purposes..............................................................4

Getting Started.............................................................................................................................................................5

About the Online Materials......................................................................................................................................5

Previewing the Online Modules...............................................................................................................................6

Setting Up a MyOpenMath Account.....................................................................................................................6

Enrolling in the Just-in-Time Courses as a Student...............................................................................................7

Setting up the Modules for a Course or Program.........................................................................................................7

Creating Your Course...............................................................................................................................................7

Copying the Modules to Your Course.......................................................................................................................9

Using the Diagnostic Pre-Test..................................................................................................................................9

Hiding Modules You Don’t Need............................................................................................................................10

Setting Due Dates...................................................................................................................................................10

Extensions and Late Passes....................................................................................................................................11

Teaching with the Materials......................................................................................................................................13

Adding Teachers and TAs to Your Course...............................................................................................................13

Enrolling Students..................................................................................................................................................14

DIY, Mastery and Practice Assessments.................................................................................................................15

Do It Yourself (DIY) Assessments........................................................................................................................15

Mastery (MA) Assessments................................................................................................................................15

Practice (PA) Assessments..................................................................................................................................16

Viewing Student Progress......................................................................................................................................16

Activity Reports..................................................................................................................................................16

Gradebook: Analyzing Performance on a Particular Assessment.......................................................................17

Gradebook: Monitoring Individual Students......................................................................................................17

Exporting the Gradebook to Excel......................................................................................................................17

Troubleshooting FAQ.................................................................................................................................................17

Math type looks weird, graphs aren't appearing...................................................................................................18

Pages load strangely when I click the "back" button on my browser.....................................................................18

I added someone to my course as a Teacher, but they can’t edit it.......................................................................18

Appendix: Module Topics..........................................................................................................................................181 | P a g e

Subject Area Topics and Associated Math Review Modules..................................................................................18

Full List of Math Review Topics..............................................................................................................................20

Additional Math Review Topics in Calculus and Physics.....................................................................................20

2 | P a g e

About the Project

In October 2014, an ad-hoc consortium of liberal arts colleges, including Allegheny, Bryn Mawr, Franklin & Marshall, Grinnell, Lafayette, Mills, Smith, St. Olaf and Vassar, were awarded a “First in the World” grant from the Fund for Improving Post-Secondary Education (FIPSE), U. S. Department of Education to develop and test a blended, “just-in-time” approach to providing support in fundamental mathematics to students enrolled in introductory STEM courses. Our goal was to support efforts these colleges were already taking to increase STEM completion rates for low-income, first-generation and under-represented minority students by addressing a key stumbling block: marginal mathematics preparation. Difficulty with the required mathematics is one of the top reasons why students of all kinds drop out of STEM programs. Low-income, first-generation and under-represented minority students are more likely to enter college with weaker math preparation and less confidence in their skills than college students generally, so addressing this stumbling block is particularly crucial for increasing diversity and equity in STEM.

The blended, “just-in-time” math fundamentals program is designed to provide affordable, scalable to support to students while they are taking introductory science courses. It is “blended” in the sense that it employs interactive online tutorials to give students individualized review and feedback, but relies on faculty, teaching assistants, and/or academic support staff to direct students to right tutorials, monitor how students are doing and step in with additional help if needed. The tutorials are designed to highlight the fundamental math skills and concepts that students need to have mastered in order to complete typical problems they encounter in their introductory calculus, chemistry and physics courses, and then steer them to review exercises for those skills and concepts if and when they need it. The target population is students who may meet their college’s standard for basic “college-readiness” in math, but who have some gaps in their preparation (e.g., limited high school exposure to trigonometry) or lack the fluency needed to thrive in an introductory science courses. The program will also help students who have completed developmental math programs transition into quantitatively demanding college-level coursework.

The grant that funded development of the blended, just-in-time math program has also funded a randomized, controlled cluster-level study to evaluate and improve its efficacy, conducted by Megan Richardson and Kelly Feighan of Research for Better Schools at PHMC (http://www.phmcresearch.org/). Participating colleges were paired; one was randomly chosen to pilot the program in Fall 2016 while the other served as a comparison case, with roles reversed for Fall 2017. Data collection associated with these trials ends in June 2018, and we hope to publish preliminary findings by September.

Although in-class testing for this initial study has ended, we have revised our research instruments and methodology to support continued evaluation as we scale the program. We invite all faculty who are pilot testing the materials in their classes to participate in the study. Detailed instructions are available in the next section. The data collected from these pilot tests will be used to evaluate the effectiveness of the materials and for on-going iterative development and improvement.

3 | P a g e

Participating in the Evaluation Study

We would like your help in collecting student data and feedback for a research study to evaluate the efficacy of these materials and improve them as needed. Bryn Mawr College's Institutional Review Board (IRB) has reviewed and approved the protocol for this study as exempt from further IRB review. Your college's IRB may want to review this protocol since it involves its faculty or students as participants in human subjects research, or it may accept approval from Bryn Mawr's IRB as sufficient. You are responsible for finding out what institutional policies are and securing approval from your college’s IRB board if needed. Please contact Jennifer Spohrer (jspohrer@brynmawr.edu) for any information you need to submit a proposal and send her a copy of the approval letter for her records.

All faculty who are piloting the materials in their courses are welcome to participate in the evaluation study. If you are a faculty member at a college that is a member of the Pennsylvania Consortium for the Liberal Arts, you are eligible to receive a $1000 honorarium for participating in the pilot study, funded by an AVD Teaching and Learning Award from that organization. Please contact Jennifer Spohrer, Director of Educational Technology Services at Bryn Mawr College (jspohrer@brynmawr.edu) for more information.

Inviting Students to Give Feedback

The last section of each module contains survey links for getting student feedback on that module. (Instructors and TAs who complete the module are welcome to use it to provide feedback as well.) The student feedback we have gathered so far has been extremely helpful as we have revised the modules, and we would love it if you would point the links out to students and encourage them to reply.

By default, the survey link visible to students gathers anonymous feedback. If you would like to offer students extra credit for feedback in order to encourage them to give it, we have created an alternative version of the survey with a place for students to enter their name and the name of their instructor in order to receive extra credit (they can also opt to fill it out anonymously). A link to this version is also included in each module, but it is hidden by default. If you wish to give students extra credit for feedback you will need to unhide these links, hide the anonymous ones:

1. Click the Quick Rearrange button at the top of the course on the right. 2. Expand a module by c in front of its title. 3. Find What did you think? Give us Feedback on [Module Title] and expand it. 4. Click Modify next to Feedback on [Module Title] for Extra Credit and set Show to Show Always.5. Click Modify next to Anonymous Feedback on [Module Title] and set Show to Hide from Students.6. Repeat for all modules.

When you are ready to award the extra credit, please email Jennifer Spohrer (jspohrer@brynmawr.edu) for a list of names and the number of modules they provided feedback on. We are also happy to share the anonymized, aggregated feedback from your students with you after your semester has ended and final grades have been posted — just let Jennifer know that you would like this information.

Inviting Students to Participate in Data-Sharing

We would also like to collect two types of data from classroom trials to evaluate the efficacy of the materials and identify areas for improvement:

1. Learning data that MyOpenMath generates as students work in the modules; namely, their final scores on assessments and how long it took to complete them.

2. Students’ final grades in the courses in which the modules were used.

4 | P a g e

In both cases, we do not have direct access to the data and need your help to obtain it. If you wish to participate in the research study, at the beginning of the semester, we ask that you:

1. Unhide the consent form in the For Faculty section of the MyOpenMath course. This form explains what data we are collecting and how we will use it.

2. Tell students about the evaluation study, and ask them read and fill out the consent form to indicate whether they wish to participate. To ensure students feel they can participate voluntarily, please let them know that won’t learn whether they consented to share data after the course has ended and final grades are posted.

After the course has ended and grades have been posted, we will send you a list of the names of students who consented to data sharing. To compile the data you will share with us:

3. Export learning data from MyOpenMath. a. Open your course, and select Gradebook.b. Click Export.c. On the Export Gradebook page make sure the following settings are selected:

i. Assessment times: Include time in questionsii. Include total number of logins: Yes iii. All other settings can be on their defaults.

d. Click Download Gradebook as CSV (this tends to work better than Excel).4. Delete any learning data for students who are not on the list of students who consented to share data.5. Add a column with the students’ final course grades.6. Delete student names and other personally identifying information from the file. 7. Send the anonymized file to Jennifer Spohrer (jspohrer@brynmawr.edu)

Getting Started

We have developed the online interactive materials for the Just-in-Time Math program on a free, open-source math homework and quizzing platform called MyOpenMath. They are organized into three “courses,” each customized for one of the three courses we are targeting in this pilot study: introductory calculus, general chemistry, and introductory physics.1 A course is essentially a suite of interconnected learning modules.

About the Online Materials

Each course contains two types of modules: subject-area modules (i.e., Calculus Topics, Chemistry Topics, Physics Topics), which are labeled in purple, and Math Review modules, which are labeled in blue.

The Calculus, Chemistry and Physics Topics modules are designed to introduce students to key problems they will encounter for each topic and highlight the fundamental math concepts and skills students need to have mastered in order to be able to successfully solve that problem. Each subject-area topic module has three parts:

1. Gateway Problem. A video walkthrough of a common problem for that topic. 2. Recommended Math Review. Links to Math Review modules where students can review the underlying

math skills and concepts needed to solve the problem.

1 We decided to limit this initial project to these three courses because they were relatively similar across participating colleges and prior research has shown direct correlations between student success in them and prior math preparation. Should the blended, just-in-time approach prove successful, we can envision developing variants for other quantitatively demanding introductory courses such as engineering or economics.

5 | P a g e

3. DIY (Do-It-Yourself) Problem. An opportunity for students to apply what they’ve learned to solve a problem similar to the one featured in the Gateway Problem video. Students can input their answers and get feedback on whether it is correct, and ask for video and textual hints as needed.

Each subject-area topic will be associated with more than one Math Review topic modules, and most Math Review modules are recommended multiple times. Most Math Review modules have a scored Mastery Assessment that students can use to demonstrate their proficiency, and a Practice Assessment that allows them to practice the skills and access video and textual review materials as needed. Some Math Review modules also include links out to additional auto-graded practice problems in Khan Academy, and in the Physics and Calculus courses, there are five additional Math Review modules that include only links to Kahn Academy practice problems on basic calculus concepts. A full list of subject-area topics and associated Math Review modules is in the Appendix to this guide on page 18.

The modules are designed to differentiate instruction to meet the needs of a range of students, providing extra help and review to those who need it while allowing those who have already mastered concepts to demonstrate that and move on. Students with solid math preparation may be able to complete DIY problems with little or no math review, whereas others may need to review in some or all areas. If you plan to give students credit or extra credit for work in the modules, see Grading and Monitoring Student Work for tips and feedback on how to give credit in a differentiated learning environment.

We have also created a diagnostic pre-test and math attitudes survey that instructors and students can use to assess student skill levels and identify areas of concern at the beginning of a semester. See Using the Diagnostic Pre-Test.

Previewing the Online Modules

We encourage faculty, TAs and academic support staff who were not involved in the development process to preview the online modules as a student before integrating them into a course or tutorial program in order to get familiar with the interface, the organization of the content, and the overall student experience.

Setting Up a MyOpenMath Account

To access the modules, you will need to create an account on MyOpenMath. If you will be setting up the online modules for your course or for the program at your college or if you will need to view student work and scores, you should request an instructor account. Otherwise you can simply create a student account.

1. Go to https://www.myopenmath.com (NOTE: the ending is .com, not .org)2. Under Getting Started, choose register as a new student or request an instructor account as shown

below.

3. Enter your college e-mail address and set up a username and password.

It’s important to use your college email address when requesting an instructor account, as it will be used to verify your affiliation. All instructors with the same domain (e.g., @brynmawr.edu, @grinnell.edu) will automatically be

6 | P a g e

included in the same group and be able to share courses and materials. You should receive an e-mail confirmation when the account is approved.

Note: The SPAM-checkers used at many colleges may block emails sent from MyOpenMath.com. To prevent this, see if you can "whitelist" the sender do-not-reply@myopenmath.com for your email account or ask your IT department if it can be whitelisted.

Enrolling in the Just-in-Time Courses as a Student

If you chose “register as a student” you can enroll in a course from the same dialog window:

1. Leave the default option (My teacher gave me a course ID) selected in the drop down menu, 2. Enter the appropriate Course ID in the blank provided:

Calculus: 28097 Chemistry: 28098 Physics: 28099

3. Enter the Enrollment key: FIPSE20174. Click Sign Up.

If you chose “request an instructor account” you will need to go to your Home page, and click the Enroll in a Course button in the Courses You Are Taking box. Then follow steps 1-4 above to enroll.

Setting up the Modules for a Course or Program

Setting up MyOpenMath modules for your course or the Blended Just-in-Time Math program at your institution is a two-step process. First, you will need to create your own course in MyOpenMath. Next, you will copy the contents from one of the model courses into the course you have created. Finally, you may want to adjust due dates to align the materials with your course curriculum.

Creating Your Course

You will need an instructor account to create courses on MyOpenMath. If don’t have one already, see the instructions under Setting Up a MyOpenMath Account on page 6.

1. Log in to MyOpenMath and navigate to your Home page.

7 | P a g e

2. In the Courses You’re Teaching Box, click Admin Page.

3. On the MyOpenMath Administration page, click Add New Course.

4. On the Course Settings page type in a Course name and an Enrollment key.

5. For now, you can leave the other settings on their defaults and click Submit.

8 | P a g e

Note: To revisit the Course Settings later, click the Course Settings link in the sidebar menu of your course page. More information on Course Settings is available here: https://www.myopenmath.com/help.php?section=administration#coursecreationsettings.

Copying the Modules to Your Course

Once you’ve created a course, you will need to copy over the modules from one of the Just-in-Time Math courses.

1. Under Course Items, click Copy (or click the Course Items: Copy link in the welcome message).

2. You will see a collapsed, hierarchical list of courses you can copy from. Expand Other’s Courses (you may have to wait a minute for loading), then expand Bryn Mawr College, then Hannah Weinstein, and choose one of the three courses: Just-in-Time Math: Calculus, Just-in-Time Math: Chemistry, or Just-in-Time Math: Physics.

3. Enter FIPSE in the Enrollment key box.

4. Leave the What to copy set to Copy whole course, and click Copy Items.

IMPORTANT: Make sure you copy the entire course. The subject-area and Math Review modules are interlinked, and these links will be broken if you do not copy the entire course.

Using the Diagnostic Pre-Test

On the advice of our faculty pilot-testers, we have created a built-in diagnostic pre-test that instructors and individual students can use to assess their mastery of math fundamentals at the beginning of a course. This test has two parts:

1. Diagnostic Skills Pre-Test, covering all topics included in the math review modules, and 2. Math Attitudes Inventory Short form, which asks students about their attitudes toward math.

By default, both parts are visible and available to students. To hide either or both parts:

9 | P a g e

1. Make sure you are in Instructor view.2. Click the gear icon next to the title of the Assessment you wish to hide and choose Settings.3. Just below the Instructions box, set the Show: setting to Hide.4. Click Save Changes

By default, the Diagnostic Skills Pre-Test contains questions for all of topics covered in the math review modules. If there are math review topics that you know your students won’t need and/or that you plan to hide from students, you can hide the associated questions from the Diagnostic Skills Pre-Test as well:

1. Make sure you are in Instructor view.2. Click the gear icon next to the title of the Assessment you wish to hide and choose Questions.3. Note that the questions are grouped by topic. 4. Select the check boxes for all questions you wish to delete, and click the Remove button. 5. Scroll down below the questions and click Done.

Note: if you accidentally delete questions you wanted to keep, don’t panic. You can always re-copy the assessment from the master course and start again.

Hiding Modules You Don’t Need

We have tried to pick subject-area topics that are covered in most introductory calculus, chemistry and physics courses. We also decided to leave all of the Math Review topic modules visible in all courses, even if they aren’t specifically recommended by any subject-area modules in that course, in case students wish to use them to review on their own. (For example, the first Math Review module, Calculator Input and Order of Operations, is not explicitly referenced in Physics or Calculus, but might nonetheless be useful for some students.)

You can hide modules that aren’t relevant to your course if you would prefer that students didn’t see them:

5. Make sure you are in Instructor view.6. Navigate to a module you wish to hide, hover over the title, and click Modify.7. Set the Show: setting to Hide.8. Set the When not available: setting to Hide from students.9. Scroll down to the bottom of the page and click Save Changes.

WARNING: We recommend hiding, rather than deleting modules you don’t need, at least until you become familiar with the materials. The subject-area and Math Review modules are interlinked. If you delete a module you will permanently break the links to it. You can always restore a deleted module by recopying it from the master course, but this restored module will not be linked to anything.

For a complete list of modules in each subject area and their associated Math Review modules, see the Appendix to this document.

Setting Due Dates

For simplicity’s sake, all of the modules and assessments in the Blended Just-in-Time Math courses have been set to be visible and accessible to students “Always.” However, most instructors will want to time their MyOpenMath assignments in a way that makes strong connections with their coursework. Our preliminary research has shown that such connections strongly help students engage with and get more out of the modules. We recommend providing due dates for the DIY assessments. Since the same Math Review module may be recommended as skills practice for several subject-area modules, we recommend not setting due date for Practice and Mastery

10 | P a g e

assessments or setting longer-horizon due dates (e.g., by the end of the semester, by the mid-term) to give students more flexibility.

Instructors have used two methods for setting due dates for work in the modules:

Conventional, not technical: List due dates for assigned modules on your syllabus and/or your LMS course page (e.g. Moodle, Canvas Blackboard, etc.), but leave the default settings in the software untouched.

o Pros: Easier to set up. No technical cut off to a student’s access, so no need for extensions. Students can work ahead and explore non-required modules.

o Cons: Harder to enforce/monitor. Students may get confused and complete assessments that aren’t required. Harder to stop cramming, and cognitive science research shows that learners retain things better if they space learning out over time.

Technical, as well as conventional: List due dates on the syllabus/course page and set due deadlines for key assessments in MyOpenMath.

o Pros: Easier to enforce/monitor. Students are forced to space their learning over time. o Cons: You will have to deal with extensions, but MyOpenMath has a few features that can help

automate this.

To set due dates for assessments in MyOpenMath:

1. Make sure you are in Instructor view.2. Under Mass Changes in the sidebar menu, click Dates.3. By default, the Mass Change Dates page shows everything in the course; choose Assessments from the

Filter by: drop-down menu to see only the assessments. 4. For each assessment, click in the Start Date column to add the date and time at which the assessment

becomes available and students can start entering answers.

Tip: If you want all assessments to be visible from the beginning of the course, set the first one to Start on the first day of classes, and then select the Copy Down Start date & time option from the drop-down menu in the Send Date Chg / Copy Down List column to copy that setting to all of the other assessments.

5. For each assessment, click in the End Date column and add the date and time at which the assessment is closed and students can no longer enter answers.

Tip: To avoid confusion, use 11:59pm rather than midnight (which changes the date) as a deadline. Or, since midnight isn’t “late” for most students, try an early morning deadline on the following day.

6. We recommend changing the settings in the Review Mode column to Always, so that assessments remain available for review, rather than being closed after the due date. In review mode, students can view all questions and answers, but no recorded response or scores are changed.

7. Scroll down to the bottom of the page and click Save Changes.

Note: We do not recommend setting due dates for Practice (PR) assessments, as these are designed to be used as much or as little as a student needs.

Extensions and Late Passes

MyOpenMath offers two mechanisms for granting extensions. The first is the LatePass, which is most useful for giving all students a set number of automated “late passes” to “spend” throughout the semester. Students redeem their late passes by clicking "Use LatePass" option that will appear next to any eligible assessment as shown below.

11 | P a g e

The second is a standard Exceptions mechanism that allows you to grant extensions to particular individuals. In both cases, extensions can be penalized or penalty-free and can be set in advance.

To use the LatePass feature, first set up the number and duration of the LatePasses granted to all students:

1. From your Course page, click Roster under Tools in the sidebar. 2. Click Manage LatePasses. 3. Enter the number of hours by which each LatePass will extend the deadline (the default is 24) and the

number of LatePasses each student receives by default.

Next, set which assessments will automatically accept LatePasses:

4. Under Mass Changes in the sidebar menu, click Assessments.5. By default, all assessments will be selected; uncheck the boxes next to any assessment that you don’t

want to include. 6. Scroll down beneath the list of assessments to the settings, and check the box next to Allow use of

LatePasses? then choose the maximum number of LatePasses a student may apply (out of his or her total allotment for the course) per assessment.

7. By default, a student has to click Use LatePass before a deadline passes in order to use it. This can be confusing, and we recommend checking Allow Late Passes after due date, within one LatePass period to allow students to allow the late pass after the deadline has passed, but within the extension windowed created by the pass. For example, if a student waited 10 hours after a deadline passed to apply a 24-hour LatePass, he or she would only have the remaining 14 hours to complete the assignment.

8. Check the box in front of Penalty for questions done while in exception/LatePass: if you wish to assign a penalty for assessments completed with a LatePass and enter a percentage in the box provided. (You can override this later for particular individuals if needed.)

9. Scroll down to the bottom of the page and click Apply Changes.

To give an individual an extension on one or more assessments:

1. From your Course page, click Roster under Tools in the sidebar. 2. Scroll down to the student’s name and click Exception. 3. Any existing exceptions for the student will be listed in the top box. To add an exception, scroll down to

the New Assessment Exception box, set a new end time and date, and check the boxes next to the assessments to which you want to apply the new deadline.

4. If you use LatePasses, you can optionally decrease this student’s quota of LatePasses (to offset the individual exception) by checking the Deduct __ LatePasses … option and entering a number in the box provided.

5. Scroll down to the bottom of the page and click Record Changes.

For more information on LatePasses and Exceptions, see this tutorial video: http://www.imathas.com/training/Mod6-exceptions/Mod6-exceptions.html

12 | P a g e

Teaching with the Materials

Adding Teachers and TAs to Your Course

You might want to give other instructors or teaching assistants access to your course. There are two types of roles you can give other MyOpenMath account holders: Teacher and Tutor. A Teacher must have an instructor account in MyOpenMath and will have the same viewing and editing permissions as the course owner. A Tutor can have either a student or an instructor account and will be able to view student responses and scores on the online assessments, and change scores on assessments if you give them permissions to do so. Tutors cannot edit the assessments or course materials themselves.

WARNING: If a Teacher or Tutor is already enrolled as a Student in the course, you will need to un-enroll them before they will have Teacher or Tutor privileges. Click Roster under Tools in the sidebar menu, check the names of the appropriate people, and click Unenroll.

To add a Teacher:

1. Go to the Home page (https://www.myopenmath.com/index.php)2. On the side bar, at the bottom of the "Courses I'm Teaching", click the button Admin Page

3. Under Courses, find the desired course and click Add/Remove under Teachers.

4. You will see a list of users who have instructor accounts on MyOpenMath using the same institutional domain (e.g. @brynmawr.edu, @grinnell.edu, etc.) that you used — click Add as Teacher next to the appropriate name.

Note: If you do not see the person you wish to add, it means they either do not have an instructor account or their account is tied to an email address with a different domain (e.g., @gmail.com). Have them request an instructor account with their college email address using the instructions above.

To add a Tutor:

1. Ask your tutors for the usernames they created when they set up their MyOpenMath accounts.

13 | P a g e

2. Click Roster under Tools in the sidebar menu of your course. 3. Click Manage Tutors.4. In the text box, type in the usernames to add, separated by commas and click Update.

Enrolling Students

Although it is possible for an instructor to enroll students in a course once they have created an account on the My Open Math site, most faculty opt to have students create accounts and enroll themselves using the Course ID and an Enrollment key.

To get your Course ID and Enrollment key:

Go to your Home page and click the Admin Page button in the Courses I am teaching box.

Click Course Settings. Write down the Course ID and Enrollment key and include them in the instructions below when giving them to your students.

Below are step-by-step instructions for logging in an enrolling in the course as a student. Feel free to copy these to your syllabus or LMS page.

Enrolling if you don’t have a MyOpenMath account:

1. Go to https://www.myopenmath.com (NOTE: the ending is .com, not .org)2. Under Getting Started, choose register as a new student.3. Enter an e-mail address and set up a username and password.

Tip: Use a username your instructor will recognize as you!

4. Leave the default option (My teacher gave me a course ID) selected in the drop down menu.5. Enter the following in the appropriate blanks:

Course ID:Enrollment key:

1. Click Sign Up.

Enrolling if you DO have a MyOpenMath account:

1. Go to https://www.myopenmath.com and log in.2. In the Courses You’re Taking box, click Enroll in a New Class.3. Leave the default option (My teacher gave me a course ID) selected in the drop down menu.4. Enter the following in the appropriate blanks:

Course ID:Enrollment key:

2. Click Sign Up.

14 | P a g e

DIY, Mastery and Practice Assessments

Our Just-In-Time courses have three types of assessments, each with its own default settings. In this section we will explain the intended use of each assessment and key settings. We strongly recommend NOT altering assessment settings until you become more familiar with the materials and the MyOpenMath interface, as they can have complex interdependencies and unintended consequences.

Do It Yourself (DIY) Assessments

These assessments are designed as formative assessments that enable students and their instructors or tutors to gauge their ability to apply the math concepts and skills they have reviewed to a problem within their subject area. Most students should attempt the DIY after watching the corresponding Gateway video and completing any Recommended Math Review modules needed to do in order to brush up on the underlying math skills. Advanced students may need to review only cursorily if at all.

Default settings for DIY assessments:

All students see the same questions, and questions do not regenerate. The Display method is set to show all questions on a single page, but allow students to answer question

one at a time and in any order and get feedback immediately after answering. Students can retake the assessment as many times as they want. Note that each attempt overwrites the

score and question responses for that assessment in the Gradebook. Each question offers a series of escalating series of hints, and the final hint reveals the answer. The

answer is also revealed after five incorrect attempts, even if hints are not requested. The Message instructor about this question option is enabled.

As these settings suggest, the DIYs are not intended to be summary assessments of a student’s abilities in the subject-area discipline — we assume that existing quizzes and exams in the target course serve that function.

Mastery (MA) Assessments

These Math Review assessments were designed to enable students to develop and demonstrate mastery in a particular Math Review topic area. They can take MA assessments as many times as they need to in order to reach mastery – i.e., get all five questions correct.

Default settings for Mastery assessments:

Questions regenerate (i.e., the variables and in some cases the formats change) each time a student attempts the assessment. Each student will see different problems, and those problems will change each time they retake the assessment.

Students can retake the assessment as many times as they want. Note that each attempt overwrites the score and question responses for that assessment in the Gradebook.

The Display method is set to show only one question at a time, but students receive feedback on their answers for all questions at the end of the quiz. Our student testers preferred this format for mastery-style assessments, reporting that it improved flow, reduced frustration, and was less overwhelming.

The Message the instructor about this question option is disabled. (However, every question type has a twin in the Practice assessment, in which this option is enabled.)

15 | P a g e

Practice (PA) Assessments

These assessments are designed to provide guided review and practice on skills and concepts covered in the Mastery assessments for students who need it. Each Practice contains the same types of problems as its Mastery twin, but has additional textual review and video tutorials available as hints to provide additional scaffold. They are designed to provide a supported, stress-free environment for students to build skills. The student and Teachers and Tutors in the course will be able to view scores and responses from his or her most recent attempt in the Gradebook, but those scores are not “counted” in Gradebook totals or averages.

Warning: This practice environment introduces a Create new version option the top right of the assessment. Clicking this link resets the entire assessment, deleting any responses up to that point. Students sometimes confuse this with the Try another similar question at the top left, which they can click after they answer a question to regenerate it with new variables.

Default settings for practice assessments:

Questions regenerate each time a student attempts the assessment. Students can retake the assessment as many times as they want. Note that each attempt overwrites the

score and question responses for that assessment in the Gradebook. The Display method is set to show all questions on a single page, but allow students to answer question

one at a time and in any order and get feedback immediately after answering. A Show answer button will appear after the student enters two incorrect answers for a question. The Message instructor about this question option is enabled

Viewing Student Progress

Student responses and scores for all assessments are recorded and displayed in the Gradebook. (Students see only their own responses and scores; Teachers and Tutors see responses or scores for all students.) Given the number of assessments in these materials, the Gradebook view can quickly become overwhelming if you are trying to monitor progress for multiple students on multiple assessments. This section suggests alternative methods for common monitoring and grading tasks.

Activity Reports

Activity Reports are efficient tool for getting an overview of student activity within a given timeframe. If you need a quick summary of student work in assessments for the purposes of giving credit or checking how a class is doing overall, use the Activity Report – Lab Style Courses:

1. On the course page, click Reports under Tools in the left sidebar. 2. Under Course Reports, click Activity Report – Lab Style Courses.3. By default, the report shows activity over the last 7 days. If you want to alter this interval, click Edit

Report Settings, and select the Time interval to display, and click Update.4. (Optional) Set a threshold criterion for students to receive “credit” for an assessment (by default, scoring

75% of possible points).

Note: If you’ve set credit/no credit criteria at the assessment level, those will be reported here instead. Report-level credit/no-credit distinctions are used only to give instructors and tutors a quick overview of student work. They are not recorded or “counted” in the Gradebook, nor are they visible to students.

16 | P a g e

The report is visible beneath the Edit Report Settings options. At the top is a brief summary of course activity during the period chosen. Beneath it are tables that break this activity down by student and by activity. Click on the rows in either table to see more details.

Gradebook: Analyzing Performance on a Particular Assessment

Use the Gradebook if you want to view responses and details about student work on a given assessment.

1. Under Tools on the left-hand toolbar, click Gradebook2. Expand the appropriate assessment category: DIY, Mastery, and Practice. 3. Locate the name of the assessment in the first row of the Gradebook grid and click [Isolate] to view only

that assessment. Students’ total scores for the assessment will be listed in the column beneath the title. 4. Click [Isolate] to view more details, including time students spent in the assessment and any feedback

tutors or teachers provided. a. To drill down to question-level responses and scores for an individual, click his or her grade.b. To see question-level data for all students, click View Item Analysis.

Gradebook: Monitoring Individual Students

You can also use the Gradebook to monitor how a particular student is doing across multiple assessments, including Practice Assessments. Knowing the extent to which a student has engaged with the practice assessments can be useful for conversations about performance on the Mastery and DIY assessments that “count.”

1. Under Tools on the left-hand toolbar, click Gradebook.2. Locate and click on the student’s name in the first column of the Gradebook.3. Here you can see the breakdown of data collected for each assessment in the course. 4. To drill down to question-level responses and scores on a particular assessment click on the grade for that

assessment.

Exporting the Gradebook to Excel

1. Under Tools on the left-hand toolbar, click Gradebook.2. In the drop-down Export to... menu, select file.3. If you are planning to use the exported data to calculate grades or credit, we recommend changing the

Separate header line for points possible? setting to Yes. This place the title and possible points value for each assessment in separate cells, which makes it easier to use the latter in calculations.

4. Click Download Gradebook for Excel once you are satisfied with the export settings.

Note for Windows 10 Users: Windows 10 often blocks downloaded Excel files (including, in our experience, files exporting from the Gradebook from opening. To remove this block, locate the file in your downloads folder, right-click on it, and click Properties. At the bottom there will be a note: “Security: This file came from another computer and might be blocked to help protect this computer.” Check the Unblock box next to this note, then click Apply. You should be able to open the file as usual.

Troubleshooting FAQ

Below are common issues that instructors and students have encountered while using MyOpenMath, and how to solve them. Another resource is the MyOpenMath Support Course, which instructors are automatically enrolled in. You can search the forums in this course for information on issues that aren’t covered in this FAQ or post questions

17 | P a g e

about any that haven’t already been discussed – the community is typically very responsive. To access the forums, go to your Home page and click Support Course in the "Courses you're taking" box. Click Forums in the sidebar menu to search across all forums.

Please also let us know of any problems or solutions you run across in your experience with MyOpenMath, so that we can add them to this document.

Math type looks weird, graphs aren't appearing.

This is usually caused by your Accessibility settings being set to something other than the default. Accessibility settings are set at log in, and it seems that sometimes they are reset to something other than the default when a person logs in after being logged out for inactivity (i.e., leaving MyOpenMath open in your browser for a long period, but not working in it). The solution is to log out, make sure the Accessibility drop down menu beneath your username and password is set to Use defaults, and then log back in.

Pages load strangely when I click the "back" button on my browser.

The page cache you are attempting to access may have expired. To avoid this, always use the "breadcrumbs," which represent where you are in the course, to navigate instead of your browser buttons (see image below). The right-most step represents the block, sub-block or assessment you are currently in, click the links to the left to move backward through the navigation to your Home page, which lists all of your courses.

I added someone to my course as a Teacher, but they can’t edit it.

They are probably enrolled in your class as a student as well. To remove them as a student, go to Tools > Roster, check their name and click Unenroll.

Appendix: Module Topics

We have grouped the FIPSE modules into three “courses,” one for each target course in the project/study. Each course begins with subject-area topic modules, which focus on problems that require fundamental math skills. These are then linked to the math-review modules for those skills. The math review modules are the same across all three courses, with the exception of modules 20-24, which were not needed in chemistry.

Subject Area Topics and Associated Math Review Modules

Calculus Area Topic Recommended Math Review TopicsComputing Limits (factoring) 3. Factoring Polynomials

4. Simplifying Rational Expressions (Fractions)Computing Limits (combining fractions) 3. Factoring Polynomials

4. Simplifying Rational Expressions (Fractions)

18 | P a g e

8. Operations with ExponentsComputing Limits (rationalization) 3. Factoring Polynomials

4. Simplifying Rational Expressions (Fractions)Compute Derivatives Using the Definition (square roots)

3. Factoring Polynomials12. Representing Functions

Compute Derivatives Using the Definition (rational functions)

12. Representing Functions4. Simplifying Rational Expressions (Fractions)

Implicit Differentiation 3. Factoring Polynomials4. Simplifying Rational Expressions (Fractions)6. Solving for a Variable in Equations with Many Variables

Related Rates 3. Factoring Polynomials4. Simplifying Rational Expressions (Fractions)

Curve Sketching (polynomials) 12. Representing Functions3. Factoring Polynomials15. Quadratic Functions

Curve Sketching (rational functions) 3. Factoring Polynomials4. Simplifying Rational Expressions (Fractions)

Curve Sketching (trigonometric functions) 12. Representing Functions17. The Unit Circle and Trigon

Optimization 3. Factoring Polynomials4. Simplifying Rational Expressions (Fractions)16. Trigonometry with Right Triangles

Chemistry Topics Recommended Math Review ModulesDimensional Analysis 1. Order of Operations and Calculator Input

2. Unit Conversion and Dimensional Analysis11. Scientific Notation

Solutions 1. Order of Operations and Calculator Input5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables2. Unit Conversion and Dimensional Analysis9. Defining Logarithms10. Logarithm Rules11. Scientific Notation

Stoichiometry 6. Solving for a Variable in Equations with Many Variables2. Unit Conversion and Dimensional Analysis

Graphing and Plotting Data 13. Linear Graphing6. Solving for a Variable in Equations with Many Variables11. Scientific Notation

Physics Topics Recommended Math Review ModulesMotion in a Straight Line 6. Solving for a Variable in Equations with Many Variables

15. Quadratic FunctionsMotion in Two Dimensions 19. Vectors

17. The Unit Circle and Trigonometric Functions6. Solving for a Variable in Equations with Many Variables

Circular Motion 17. The Unit Circle and Trigonometric Functions5. Solving Linear and Rational Equations in One Variable

19 | P a g e

19. VectorsForce: Static Objects 7. Solving Systems of Equations

5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables4. Simplifying Rational Expressions (Fractions)

Force: Moving Objects 7. Solving Systems of Equations5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables4. Simplifying Rational Expressions (Fractions)

Conservation of Energy 5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables

Conservation of Momentum 16. Trigonometry with Right Triangles19. Vectors

Gravity: Orbital Motion 5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables8. Operations with Exponents

Rotational Motion and Angular Momentum

5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables4. Simplifying Rational Expressions (Fractions)

Oscillatory Motion 5. Solving Linear and Rational Equations in One Variable6. Solving for a Variable in Equations with Many Variables4. Simplifying Rational Expressions (Fractions)

Wave Motion 5. Solving Linear and Rational Equations in One Variable12. Representing Functions17. The Unit Circle and Trigonometric Functions

Multiple Topics 16. Trigonometry with Right Triangles17. The Unit Circle and Trigonometric Functions19. Vectors

Full List of Math Review Topics

1. Order of Operations and Calculator Input2. Unit Conversion and Dimensional Analysis3. Factoring Polynomials4. Simplifying Rational Expressions (Fractions)5. Solving Linear and Rational Equations in One

Variable6. Solving for a Variable in Equations with Many

Variables7. Solving Systems of Equations8. Operations with Exponents9. Defining Logarithms

10. Logarithm Rules11. Scientific Notation12. Representing Functions13. Linear Graphing14. Linear Functions (Word Problems)15. Quadratic Functions16. Trigonometry with Right Triangles17. The Unit Circle and Trigonometric Functions18. Area, Perimeter, and Arc Length19. Vectors

Additional Math Review Topics in Calculus and Physics

20. Limits21. Differentiation I 22. Differentiation II (Applications of Derivatives)23. Introduction to Integrals24. Techniques of Integration

20 | P a g e

21 | P a g e