MathPASS: A Remedial Mathematics System with MathPASS: A Remedial Mathematics System with Automated Answer CheckingAutomated Answer Checking

Wei Su1 Paul S.Wang2 Lian Li1 Department of Computer Science, Lanzhou University, PRC Department of Computer Science, Kent State University, USA wsu@cs.kent.edu

July, 2010 on CICM’10

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

OutlineOutline

MathPASS Overview

Question Prototype and MAML

Answer Inputting and MathEdit

Answer Checking and MACP

User Statistics

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

1. Overview1. Overview

MathPASS is a Web-based intelligent mathematics assessment system

A drill-and-practice system which has been applied in remedial mathematics courses at KSU.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Taking Assignment Page in MathPASSTaking Assignment Page in MathPASS

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

System Modules of MathPASSSystem Modules of MathPASS

User Management

User Management

core

StudentManagement

StudentManagement

Account Setting

Account Setting

AssignmentManagement

AssignmentManagement

Class Management

Class Management

QuestionManagement

QuestionManagement

Practice Practice

GradeManagement

GradeManagement

QuestionsGeneration

QuestionsGeneration

AnswerChecking

AnswerCheckingAnswer

Inputting

AnswerInputting

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

The System Architecture of MathPASSThe System Architecture of MathPASS

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

2. Question Generation2. Question Generation

The questions in MathPASS could be generated automatically by question prototypes on each time when students take the assignment.

A question prototype is a mathematical question template for a specific question pattern containing random parameters satisfying well-defined mathematical conditions.

More than 400 question prototypes have been created in MathPASS by the combined efforts of mathematics teachers and mathematics education experts.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Question Prototypes and InstancesQuestion Prototypes and Instances

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

• The table also gives the statistic result including taken times, correct times, and the correct ratio of the question. • All of the statistic data can be accessed by a MathPASS teacher user who may use these data in preparing his/her course.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

MAML-Mathematical Assessment Markup LanguageMAML-Mathematical Assessment Markup Language

Paper (homework, quiz, examination) elements<paper> <pdescription> <ptitle> <psubject> <pstarttime> <pendtime> <psite> <pdifficulty> <pscore> <pspecification> <pschool> <pgrade> <pclass> <pname> <pstudentid> <psection>

Question elements<questionset> <question> <qdescription> <qbody> <qchoice> <qtext> <qblank> <qmathml> <qsvg> <qimage> <qtocorrect> <qparameter> <qkey>

Question Prototype elements < prototype > < prototypeBody> <prototypeCond> <require> <requirenot><and><or> <test><apply>

Description elements <qtopic> <qcorrectness> <qdifficulty> <qrscore> <qanswer> <qscore>

Presentation elements<qrow> <qbr>

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

An example of MAML for Question PrototypeAn example of MAML for Question Prototype<prototype>

<prototypeBody> <apply><or><plus/><minus/></or></apply> <apply name=“subexp1”> <divide/> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“9”>A</cn> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“15”>B</cn> </apply> <apply name=“subexp2”> <divide/> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“9”>C</cn> <cn type=“int” valtype=“rand” minVal=“1” maxVal=“15”>B</cn> </apply></prototypeBody>

<prototypeCond> <require> <test value=“subexp1” compare=“eq”>SF</test> <test value=“subexp2” compare=“eq”>SF</test> </require> </prototypeCond>

</prototype>

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

3. Answer Inputting and MathEdit3. Answer Inputting and MathEdit

An intuitive and convenient mathematical expression input tool is very important for a mathematical CAA system to enter answer.

MathEdit, a Web-based visual interactive mathematical expression editor, aims to provide an integrated solution of on-line entering and editing of mathematical expressions.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

MathEdit Overview MathEdit Overview

MathEdit provides convenient and intuitive graphical user interface.

MathML, OpenMath, LaTex, Infix, Picture (Gif, Png), PDF could be generated by MathEdit.

It also provides well-defined API.

It can work with IE and the Mozilla-family browsers

Written in JavaScript and MathML.

http://mathedit.lzu.edu.cn or http://wme.cs.kent.edu/mathedit

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

MathEdit Authoring EnvironmentMathEdit Authoring Environment

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

The architecture of MathEditThe architecture of MathEdit

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Some Examples of Math expressionSome Examples of Math expression

Demo

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

MathEdit Infix VersionMathEdit Infix Version

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

MathEdit Usage ExamplesMathEdit Usage Examples

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4. Answer Checking and MACP4. Answer Checking and MACP

Overview of MACP

Data Format of MACP

Answer Grade Rule

Full Simplification Form

Special Syntax Form

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4. 1 Overview of MACP4. 1 Overview of MACP

Mathematical Answer Checking Protocol (MACP) is an access protocol for communication between MACP service and its client.

MACP service is a Web service for checking answers and grading questions to mathematics of middle school and college.

The MACP service aims to grade the user answer through

– verifying expressions equivalence

– checking the expression form of the answer

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Overview of MACPOverview of MACP

The standard answer could be provided by the Computer-aided Assessment (CAA) system which is using MACP or automatically generated and computed by the MACP service.

The implementation of MACP is based on Representational State Transfer (REST).

The request data and service response will be encapsulated into JSON code.

In internal, MACP service use computer algebra systems (Maxima) to verify the expression equivalence and compute the correct answer.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

A usage example of MACPA usage example of MACP

MathEdit

AnswerGrade

Program

CAS Question

Generator

CAAPHP Proxy

MACP Service Web Server

MACP Client

BrowserAJAX

POST

JSON String

MACP Request

MACP Response

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4.2 DATA FORMAT OF MACP4.2 DATA FORMAT OF MACP[{"name":"<check item name>",

"cmd":"<check command>",

"response":"<response list>",

"exprs":[{

"expr":"<expression name>",

"encoding":"<encoding format>",

"role":"<expression role>",

"value":"<expression itself>",

"form":"<required expression form>",

"op":"<question operation>",

"var":"<variables>"

},...]

},...]

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

The Response DataThe Response Data[{

"name":"<check item name>",

"status":"<status>",

"error":"<error code>",

"correctness":"<correctness>",

"response":[{

"rname":"<response item name>",

"value":"<response item value>"

},...]

},...]

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

ExampleExample2x2x22 + 8x + 7 = 0 + 8x + 7 = 0

[{"name":"c1",

"cmd":"(u1=s1&u2=s2)|(u1=s2&u2=s1)",

"request":["ranme":"u1.mmlp","rname":"u2.mmlp"],

"exprs":[

{ "expr":"s1","encoding":"mmlc","role":"stdAnswer",

"value":"<math><plus/><apply><minus/><cn>2</cn></apply><apply><divide/>

<apply><root/><cn>2</cn></apply><cn >2</cn></apply></math>"},

{ "expr":"s2","encoding":"mmlc","role":"stdAnswer",

"value":"<math><minus/><apply><minus/><cn>2</cn></apply><apply>

<divide/><apply><root/><cn>2</cn></apply><cn >2</cn></apply></math>"},

{ "expr":"u1","encoding":"infix","role":"usrAnswer","value":"(-4-sqrt(2))/2"},

{ "expr":"u2","encoding":"infix","role":"usrAnswer","value":"sqrt(8)/2-2"}

]

}]

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Response Data of Response Data of the Examplethe Example

[{

"name":"c1",

"status":"normal",

"correctness":"96%",

"details":"<u2> is not simplest",

"response":[

{ "rname":"u1.mmlp",

"value":"<math><mfrac><mrow><mo>-</mo><mn>4</mn><mo>- </mo>

<msqrt><mn>2</mn></msqrt></mrow><mn>2</mn></mfrac></math>"},

{ "rname":"u2.mmlp",

"value":"<math><mfrac><msqrt><mn>8</mn></msqrt>

<mn>2</mn></mfrac><mo>-</mo><mn>2</mn></math>"

}]

}]

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4.3 Answer Grade Rule4.3 Answer Grade Rule

When can we say an answer is ”right”? – The answer which student entered can be computed with

the standard answer by CAS. Obviously the answer is wrong if the computation result is not equivalence in algebra. However we cannot easily say the answer is right even though they are equivalence.

8 5

5 1

6 y x

1/2

5 8

25 1

6 x y5 5 3

1 5

6y x y8 5

1 1 1*5

6 y x

5

8

5

6

x

y

2 3 3 5

1 5

6x y x y

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Answer Grade RuleAnswer Grade Rule

Hundreds of emails from students to report that the answer they entered is correct but the system grade it wrong.

On the other hand, some mathematical teachers also report that some student answers should be treated as ”wrong” or ”partial right” while the system grade them right.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

ClassificationClassification

Most correct answer or best answer be classified into two broad categories:

– simplest form

– special syntax form

This taxonomy does not separate strictly all the answers into two totally different categories. One part of the answer with special syntax form may also be required as a simplest form.

However this taxonomy has been found to be useful for evaluating the answer of students and solving the issues of partial credit.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4.4 Full Simplification Form4.4 Full Simplification Form

The question of this category included:

– simplification,

– calculation,

– writing the equation of a function,

– writing the domain of expression,

– …

Exmaples:

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Rule-based Grade MethodRule-based Grade Method

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Canonical FormCanonical Form

Rewrite Rule 1 (Remove Redundant Parentheses)

Rewrite Rule 2 (Commutative Rule of Addition andMultiplication)

Rewrite Rule 3 (Multiplication of Fractions)

Rewrite Rule 4 (Representation of Radical)

The expression will not be simplified during the canonical process and the length and complexity of the expression were not changed during the rewriting.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Simplest forms of a non-symbolic constant Simplest forms of a non-symbolic constant expressionexpression

Simplest Natural Number (SN)

Simplest Integer (SI)

Simplest Fraction (SF)

Simplest Decimal Number (SD)

Simplest Exponent/Radical (SE)

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Checking Rule AChecking Rule A

A non-symbolic constant expression is in simplest form when it satisfies one of the following conditions:

– It is one of the SN, SI, SD, SF, SE;

– If a non-symbolic constant expression is the addition or minus of one SE number and one number of SN, SI, SD, SF, the expression is a simplest form expression.

– If a non-symbolic constant expression is the multiplication of one SE number and one number of SN, SI, SD, or SF, where the base of the SE number is different with the other number, the expression is a simplest form expression.

– If a non-symbolic constant expression is the dividing of one SE number and one number of SN, SI, or SD, where the base of the SE number is different with the other number, the expression is a simplest form expression.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Simplest PrimarySimplest Primary

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

TermTerm

In elementary mathematics, a term is either a single SP, or the product of several SPs separated from another term by a + or - sign in an overall expression. For example, in 3*9+4*x+5*y*z*w, the 3*9, 4*x, and 5*y*z*w are all terms.

A term can be written in form

C*S*V

where C is a non-symbolic expression, S is a symbolic expression and V is a variable expression.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Checking Rule BChecking Rule B

A term of an expression is in simplest form when it satisfies the following conditions:

– each variable or symbolic constant only occurs once;

– the non-symbolic part is in simplest form.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Checking Rule CChecking Rule C

A polynomial expression is in simplest form when it satisfies the following conditions:

– each term of the polynomial expression is in simplest form;

– the variable parts are different for each two terms of the polynomial.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

4.5 Special Syntax Form4.5 Special Syntax Form

Students need write the answer expression as an appointed syntax form such as multiplication of polynomials, a sum or difference of logarithms.

This kind of answer does not need to be reduced to lowest terms. Contrarily, the question may be lowest terms and obviously the answer should not be same with the question.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

ExamplesExamples

The questions of this category include

– factoring,

– solving equation,

– rewriting the expression as request,

– find the function as request,

– …

Example: Factor Completely, leaving only positive exponents in your answer

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Special Syntax FormSpecial Syntax Form

Factoring Polynomial (FP)

Normalized Scientific Notation (NSN)

Sum and Difference of Logarithm (SDL)

Positive Exponent (PE)

…

The user may use logical combination of these special form as a request form. For example a client may send SF|SI to statement the answer could be a simplest fraction or a simplest integer.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Pattern Language for Mathematical ExpressionPattern Language for Mathematical Expression

Mathematical Pattern Language provides a concise and flexible means to represent classes of expressions.

A simple example of a mathematical pattern is the expression 2+x^$. This mathematical pattern represents the class of expressions with the form 2+x^[any exponent].

– Regular Expressions--> String

– Mathematical Pattern Language--> Mathematical expression

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

An Example of Mathematical Pattern LanguageAn Example of Mathematical Pattern Language

sin$1v+cos$1v <mq:query> <plus/> <apply> <sin/> <mq:st valuetype=“var” class=“1”> </mq:st> </apply> <apply> <cos/> <mq:st valuetype=“var” class=“1”> </mq:st> </apply> </mq:query>

sinx+cosx

sina+cosa

sin2+cos2

sinx+cosy

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

5. Statistics5. Statistics for MathPASSfor MathPASS

Name Numbers Name Numbers

Teachers 103 Students 6204

Courses 547 Assignments 3731

Taken Assignments 121770

Taken Questions 1282095

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

6. Future Work6. Future Work

Improve the MACP and answer grade rule.

Add the geometry question in MathPASS.

Wei Su

http://mathpass.math.kent.edu

Lanzhou University

Thank you！