Impact of Mindspark’s Adaptive Logic on StudentLearning

Ramkumar RajendranIITB-Monash Research Academy

Indian Institute of Technology, Bombay, India

ramkumar.r@iitb.ac.in

Aarthi MuralidharanEducation specialist, EI India Pvt Ltd

Bangalore, India

aarthi.muralidharan@ei-india.com

Abstract—Mindspark is a math Intelligent Tutoring System(ITS), which provides personalized learning content, to studentsof grade 3 to 8, based on their response and performance inthe topic. Mindspark is used in more than 100 schools in Indiaas part of the school curriculum. In this paper, we study theimpact of the adaptive logic of Mindspark on students’ learning.We focus our research on the remedial module presented to thestudents by Mindspark when they are unable to understand thelearning units and have misconceptions in the topic. The studywas conducted to evaluate the impact on 199 students of grade5 from a school, where the students use Mindspark to learnfractions. In this paper we choose four students’ learning pathfor detailed analysis. The preliminary analysis shows that theremedial module of Mindspark helps the students to resolve theirmisconception and complete their learning goal.

Keywords: Intelligent Tutoring system, Learning path,Mindspark, Adaptive Logic, Remediation.

I. INTRODUCTION

An Intelligent Tutoring System (ITS) is a computer basedtutoring system that provides personalized learning contentto students based on factors like their performance, priorknowledge, etc [1], [2]. In the ITS, the sequencing of learningcontent is personalized to avoid cognitive mismatch suchas cognitive overload for low performers and boredom forhigh performers. An ITS consists of the learning content, thestudent model and the adaptation engine [1]. Student modelsare constructed from the log files available in the ITS. Thestudents’ interaction with ITS, such as responses to questions,number of attempts at a task, and the time taken for variousactivities (such as responding or reading) are captured inthe ITS log file. Adapting the learning content based on theinformation from the student model, and personalizing thelearning for the student, enables the ITS to work with thestudents of different abilities.

ITSs are typically a web-based learning system [3], whichare used to teach wide range of topics from math to medicine[4], [5], [6]. Math tutors such as Active Math [7], Cognitivetutor [8], are widely used in schools as part of the curriculum.In this paper we focus on the web based ITSs, which are usedto teach math in schools.

Most of the existing math ITSs provide the learningcontent in small learning units and student’s performance inthe learning units are evaluated. When the student fails tounderstand the learning unit, then s/he is asked to repeat thelearning unit, [9]. In cases where the student has a strong

misconception in the unit, a simple repetition of the unit willnot be sufficient [10]. Hence, the student is asked to repeatthe previous learning unit which would form the basis ofunderstanding for the current unit. And repeating the previouslearning unit helped the students in their leaning [10]. Howeverrepeating the previous learning unit may not clear the student’smisconception in the learning unit. To address the aboveproblem, Mindspark’s adaptive logic identifies the student’smisconception and clears it using remedial module. In thispaper, we analyze the impact of clearing misconceptions, usingMindspark remedial modules, on students’ learning.

Research Question: Does addressing of misconceptionusing Mindspark’s remedial module impact student learning?

Mindspark, a web based math ITS 1, is used in more than100 schools in India as part of the school curriculum. Wehave studied the performance of Mindspark’s adaptive logicby analyzing the learning path of four students of grade fivefrom one school in India. We used the topic ’fractions’ inour research. We chose this topic as the students had beenintroduced to fractions in grade four and the topic acts asa building block for learning other topics. The preliminaryanalysis shows that, the remedial module helped the studentsto clear their misconceptions and achieve their leaning goal.In the next section, we discuss the related works.

II. RELATED WORK

The need for a personalized learning and immediate feed-back led many researchers to work on ITS. It resulted in widerange of ITSs in different fields, for example SHERLOCK [4]tutor teaches the Air-Force technicians to diagnose electricalproblems in Jets, Cardiac [6] tutor simulates the function ofheart to the medical students, Why2-Atlas [5] helps to learnphysics principles, and AutoTutor [11] assists students to learnabout Internet, operating systems etc. The recent review onimpact of ITS on student learning [12], reported that, in generalITSs improves student learning. In this paper we focus onresearches which analyzed the impact of math ITS. Most ofthe commercial ITSs, such as Active Math [7], Cognitive tutor[8], [13], Mathematics Tutor [14], [15], and PAT Algebra Tutor[16], [17] are tested in schools and reported improvement instudents’ learning.

All these ITSs allows the students to redo the questionsor learning unit based on their performance in the learning

1http://www.mindspark.in/

2013 IEEE Fifth International Conference on Technology for Education

978-0-7695-5141-8/13 $31.00 © 2013 IEEE

DOI 10.1109/T4E.2013.36

119

unit. For example Wayang Outpost [18], a web based mathITS used as the supplement for high school geometry, allowsthe students to redo the questions based on their performance.Wayang Outpost adapts the learning content based on no ofhints used, student’s performance in last five questions etc., andallows the student to master the skill. The Impact of WayangOutpost was tested on rural and urban schools [18] using 190students, and reports that the ITS is useful for the students toachieve higher post test scores compared to control group.

On-line Knowledge Diagnose System (OKDS) [10], an in-telligent e-learning system, identifies the student’s misconcep-tion and provides personalized learning guidance to improvelearner’s learning performance. OKDS customizes the learningpath based on concept map of the topic and student’s pre-testscore. To clear the misconceptions, OKDS takes the studentto the concept that is a pre-requisite to concept to which thestudent failed, as indicated in the concept map. The impact ofOKDS was tested on 45, first grade university students usingpre-post test, control group analysis and OKDS [10] reports thesignificant different in post test scores of experimental groupcompared to control group.

Discussion: In the existing ITS [10], the student’s mis-conception is handled by providing the pre-requisite content.However, by repeating the basic content the student’s skill mayimprove but it may not clear the misconceptions. To avoidmisconception, addition to repeating the learning unit, ITS canuse cognitive dissonance [19] theory which creates the aware-ness to conflict the student’s prior belief (misconception), andguide the student for correct explanation [20] to resolve theirmisconceptions. In Mindpsark, the students misconceptions arecleared using cognitive dissonance theory in remedial module.In our research we analyze the impact of Mindspark’s remedialmodule on students’ learning. In next section we describe aboutMindspark in detail.

III. SYSTEM: MINDSPARK

Mindspark is a commercial mathematics ITS developedby Educational Initiatives India (EI-India)2. Mindspark hasbeen incorporated into the school curriculum for different agegroups (grades 3 to 8) of students. Mindspark is currentlyimplemented in 100 schools and is being used by 80,000students on an average of four sessions per week, with eachsession ranging from 30 to 40 minutes.

Mindspark is a computer based self-learning system, inwhich students learn mathematics by answering questionsposed by the system. The student gets feedback and detailedexplanation upon answering the question. Mindspark consistsof a sequence of specially designed learning units (clusters),which contain questions on concepts that make up the topic.Each topic consists of questions of progressively increasinglevels of complexity. Mindspark covers a wide range of topicsin school level mathematics such as linear inequalities, matri-ces, quadratic equations, fractions, decimals, and polygons. Innext subsection we discuss the Mindspark’s adaptive logic indetail.

2http://www.ei-india.com/

A. Mindspark’s Adaptation Logic

Mindspark adaptation logic selects the questions to bepresented based on a student’s response to the current questionand his/her overall performance in the topic, thereby allowingthe student to move at his/her own pace. When a studentanswers a question incorrectly, the system provides detailedexplanation to the student on the concept involved. The studentis then presented another question of the same difficulty level(DL) from the same learning unit, to give the student anotheropportunity to demonstrate his/her understanding.

In Mindspark, questions are designed to identify the causesof error based on the options students select. The errors aregrouped into carelessness, lack of concept understanding andmisconception. The sample question to identify the causes oferror is shown in the Figure 1.

Fig. 1. A sample question from Mindspark to detect the causes of error. Ifthe student selects the option A, it shows that the student didn’t understand theconcept of fraction or due to carelessness, option B indicates that the studentunderstood the concept of fraction, option C indicates carelessness, or notunderstanding the concept of fraction and option D indicates the that studenthas misconception of ’all parts are equal in a shape’

When a student gets 25% or more questions incorrectin a learning unit, it is considered as learning unit failure.At the first failure of learning unit, Mindspark allow thestudent to repeat the learning unit. At the second failure ofthe learning unit, the causes of student’s failure is identifiedbased student’s performance in the learning unit. If the studentgot the questions incorrect due to lack of understanding offundamental concept, then the adaptive logic takes the studentto the immediate previous learning unit. If the student got thequestions incorrect due to misconception, then the remedialmodule is given to the student.

Remedial modules are designed to resolve the miscon-ception based on cognitive dissonance theory [19]. In cog-nitive dissonance method, the system conflicts the student’sprior understanding (misconception), then gives the correctexplanation to resolve the misconception. The sample conflict-resolve question is shown in the Figure 2. The series ofsimilar questions as in Figure 2, for different shapes, clearsthe student’s misconception.

IV. METHODOLOGY

To find the answer to our research question, does ad-dressing of misconception using Mindspark’s remedial moduleimpact student learning, we observe the students’ learning pathand analyze it to check whether the students’ who is havingmisconception are able to achieve their learning goal. In thissection we explain our methodology to select the topic, school,and analysis done.

120

Fig. 2. A sample question from Mindspark remedial module. The questionclears the student’s misconception of ’all parts are equal in a shape’ by conflict-resolve method

A. Topic: Fractions

For this research, we chose a topic that spanned across atleast 3 grade levels (for example, grades 4, 5 and 6). Thiswas done so that we could observe how the adaptive logic ofMindspark takes the student through lower/higher grade levelsdepending on the ability of the student. We chose the topicFractions - basic concepts, equivalence and comparison whichis taught across grades 4, 5 and 6. The topic Fractions - basicconcepts, equivalence and comparison consists of the following9 learning units for grade 5.

1) Naming fractions and understanding fraction notation2) Working with unit fractions3) Understanding fractions as part of a collection4) Understanding mixed numbers5) Working on mixed numbers with 1/2,1/4 and 3/4 as

fractional parts6) Understanding equivalence of fractions7) Finding equivalent fractions, and reducing fractions8) Introduction to comparison of fractions9) Comparing fractions by finding common denominator

The learning goal of students in class 5 is to clear all thenine learning units in the topic.

B. Sample

Our research aim is to analyze the impact of Mindspark’sremedial modules on student learning, hence we choose aschool where half of the class had completed the topic fractionsin Mindspark. The school had 199 students in grade 5 (10-12year old), out of which 75 students had attempted more thanthree learning units in the topic. We selected only the data ofthese 75 students for our analysis, as the rest of the studentshad not attempted the one-third of the topic and hence wouldnot have got an opportunity to experience the customizedlearning path presented by the adaptive logic for the entiretopic.

C. Analysis Procedure

The learning path of the four students out of 75 studentsare analyzed to measure the impact of remedial module onstudent’s learning.

V. RESULTS

We selected four students’ learning path in the topicfractions to check how the remedial module resolved thestudents’ misconception and how it helped the students toachieve their learning goal. We selected four students as partof our preliminary analysis. The learning paths of four studentsare shown in Figure 3.

In Figure 3, student 1 got the remedial module afterthe second attempt of learning unit 1. Similarly student 1got remedial module in latter learning units to resolve themisconceptions and student 1 is able to clear all the learningunits assigned to his/her grade after 23 attempts of learningunits.

Student 2, got remedial module after the second attemptof learning unit 1. However the student didn’t understand theconcept of fraction, hence he was not able to claer the learningunit 2. The ITS moved the student back to learning unit 1 andafter second attempts of learning unit 1, remedial module wasprovided. Still the student was not able to clear the learningunit 2, hence Mindspark moved the student to pre-requisiteof fractions to improve student basic concept. It helped thestudents to clear the higher learning units. However the studentwas not able to clear all the learning units assigned to his/hergrade and dropped out after 16 attempts of learning unit.

Student 3, cleared all the learning units in first attempt. Stu-dent 4, initially struggled to clear the learning units, howeverafter the remedial module at the learning unit 3, s/he clearedall the learning units assigned to his/her grade.

A. Discussion

When a student gets a wrong answer, depending on thecause of the error, Mindspark’s adaptive logic may lead toeither of: (i) repeating the current learning unit, (ii) revisit-ing the previous learning unit, or (iii) taking up a remedialmodule to resolve the student’s misconception. Our analysisof four students’ learning path indicates that Mindspark’sremedial modules, resolved the students’ misconception andhelped the students to achieve the learning goal. Our resultsreconfirm the results reported by OKDS [10] that repeatingthe previous learning unit positively impacts the student’slearning. Thus, we give the answer to our research question as -Yes, addressing of misconceptions using Mindspark’s remedialmodule positively impacts student learning. However, sincewe cannot isolate the effect of the remedial module fromother adaptations, we cannot quantify this impact. It is likelythat students might have achieved their learning goal due tocombined effect of the three adaptation mechanisms. Furtherdetailed study is needed to measure the impact of only theremedial module, by controlling the adaptive logic of the othertwo mechanisms.

121

Fig. 3. The learning path of four students of grade 5 in topic fractions. The RED marks indicate the places, where the students can get the remedial moduleto resolve the misconception, based on his responses to the Mindspark questions.

VI. CONCLUSION

In this paper we analyzed the impact of Mindspark’sremedial module, which resolves the students’ misconception,on learning. We analyzed the learning path of four students ofgrade five from a school, where Mindspark was used to learnfractions. The analysis shows that remedial module helped thestudent to achieve their learning goal. In our future work,we propose to evaluate the impact of remedial module bycomparing the impact of repeating the previous learning unitto student’s learning.

ACKNOWLEDGMENT

The authors would like to thank the reviewers for theirvaluable comments. We would also like to thank the staff andthe Managing Director of Education Initiatives for their supportwhile we conducted our research.

REFERENCES

[1] P. Brusilovsky and E. Milln. User models for adaptive hypermediaand adaptive educational systems. In The adaptive web: methods andstrategies of web personalization, pages 3–53. Springer-Verlag, 2007.

[2] Albert T Corbett, Kenneth R Koedinger, and John R Anderson. Intelli-gent tutoring systems. Handbook of humancomputer interaction, pages849–874, 1997.

[3] Peter Brusilovsky, Elmar Schwarz, and Gerhard Weber. Elm-art: Anintelligent tutoring system on world wide web. In Intelligent tutoringsystems, pages 261–269, 1996.

[4] Susanne P Lajoie and Alan Lesgold. Apprenticeship training in theworkplace: Computer-coached practice environment as a new form ofapprenticeship. Machine-Mediated Learning, 3(1):7–28, 1989.

[5] Kurt VanLehn, Pamela W Jordan, Carolyn P Rose, Dumisizwe Bhembe,Michael Bottner, Andy Gaydos, Maxim Makatchev, Umarani Pap-puswamy, Michael Ringenberg, Antonio Roque, et al. The architectureof why2-atlas: A coach for qualitative physics essay writing. InIntelligent tutoring systems, pages 158–167. Springer, 2002.

[6] Chris Eliot and Beverly Park Woolf. Reasoning about the user withina simulation-based real-time training system. In Fourth InternationalConference on User Modeling, Hyannis Cape Cod, Mass, pages 15–19,1994.

[7] Erica Melis and Jorg Siekmann. Activemath: An intelligent tutoringsystem for mathematics. In Artificial Intelligence and Soft Computing-ICAISC 2004, pages 91–101. Springer, 2004.

[8] Steven Ritter, John R Anderson, Kenneth R Koedinger, and AlbertCorbett. Cognitive tutor: Applied research in mathematics education.Psychonomic bulletin & review, 14(2):249–255, 2007.

[9] Carole R Beal, Ivon M Arroyo, Paul R Cohen, Beverly P Woolf, andCarole R Beal. Evaluation of animalwatch: An intelligent tutoringsystem for arithmetic and fractions. Journal of Interactive OnlineLearning, 9(1):64–77, 2010.

[10] Ling-Hsiu Chen, Yi-Chun Lai, and Yi-Hsu Weng. Intelligent e-learningsystem with personalized misconception diagnose and learning pathguidance. In International Conference on Electronic Business, 2009.

[11] Arthur C Graesser, Katja Wiemer-Hastings, Peter Wiemer-Hastings, andRoger Kreuz. Autotutor: A simulation of a human tutor. CognitiveSystems Research, 1(1):35–51, 1999.

[12] ELENA Verdu, LUISA M Regueras, MARIA JESUS Verdu,JUAN PABLO de Castro, MA Perez, Qing Li, SY Chen, and AnpingXu. Is adaptive learning effective? a review of the research. In WSEASInternational Conference. Proceedings. Mathematics and Computers inScience and Engineering, number 7. World Scientific and EngineeringAcademy and Society, 2008.

[13] John R Anderson, Albert T Corbett, Kenneth R Koedinger, and RayPelletier. Cognitive tutors: Lessons learned. The journal of the learningsciences, 4(2):167–207, 1995.

[14] Mia Stern, Joseph Beck, and Beverly Park Woolf. Adaptation ofproblem presentation and feedback in an intelligent mathematics tutor.In Intelligent tutoring systems, pages 605–613. Springer, 1996.

[15] CR Beal, JE Beck, and BP Woolf. Impact of intelligent computerinstruction on girls math self concept and beliefs in the value of math.In Poster presented at the annual meeting of the American EducationalResearch Association, San Diego, 1998.

[16] Steven Ritter, John Anderson, Michael Cytrynowicz, and OlgaMedvedeva. Authoring content in the pat algebra tutor. Journal ofInteractive Media in Education, 1998(2), 1998.

[17] Kenneth R Koedinger, John R Anderson, William H Hadley, Mary AMark, et al. Intelligent tutoring goes to school in the big city.International Journal of Artificial Intelligence in Education (IJAIED),8:30–43, 1997.

[18] Ivon Arroyo, Carole Beal, Tom Murray, Rena Walles, and Beverly PWoolf. Web-based intelligent multimedia tutoring for high stakesachievement tests. In Intelligent Tutoring Systems, pages 468–477,2004.

[19] Leon Festinger. A theory of cognitive dissonance, volume 2. Stanforduniversity press, 1962.

[20] Barbara J Guzzetti, Tonja E Snyder, Gene V Glass, and Warren SGamas. Promoting conceptual change in science: A comparativemeta-analysis of instructional interventions from reading education andscience education. Reading Research Quarterly, pages 117–159, 1993.

122