Learning a scientific discipline such as biology is a daunt-ing challenge. In a typical advanced high school orintroductory college biology course, a student is expect-

ed to learn about 5000 concepts and several hundred thou-sand new relationships among them.1 Science textbooks aredifficult to read and yet there are few alternative resources forstudy. Despite the great need for science graduates, too fewstudents are willing to study science and many drop outwithout completing their degrees. New approaches are need-ed to provide students with a more usable and useful resourceand to accelerate their learning.

The goal of the Inquire Biology textbook is to provide betterlearning experiences to students, especially those studentswho hesitate to ask questions.2 We wish to create an engag-ing learning experience for students so that more studentscan succeed — and specifically to engage students in moreactively processing the large number of concepts and rela-tionships. Inquire Biology aims to achieve this by interactivefeatures focused on the relationships among concepts,because the process of making sense of scientific concepts isstrongly related to the process of understanding relationshipsamong concepts (National Research Council 1999). Toencourage students’ engagement in active reading, our peda-gogical approach is to help a student to articulate questionsabout relationships among concepts and to support them infinding the answers.

Inquire Biology incorporates multiple technologies from thefield of artificial intelligence. It includes a formal knowledgerepresentation of the content of the textbook, reasoningmethods for answering questions, natural language process-ing to understand a user’s questions, and natural languagegeneration to produce answers. It is based on a systematicknowledge-acquisition process that educators can use to rep-resent the textbook’s knowledge in a way that the computercan reason with to answer and suggest questions. A unique

Articles

FALL 2013 55Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Inquire Biology:A Textbook that Answers Questions

Vinay K. Chaudhri, Britte Haugan Cheng, Adam Overholtzer,Jeremy Roschelle, Aaron Spaulding, Peter Clark,

Mark Greaves, Dave Gunning

n Inquire Biology is a prototype of anew kind of intelligent textbook — onethat answers students’ questions,engages their interest, and improvestheir understanding. Inquire Biologyprovides unique capabilities through aknowledge representation that capturesconceptual knowledge from the textbookand uses inference procedures to answerstudents’ questions. Students ask ques-tions by typing free-form natural lan-guage queries or by selecting passages oftext. The system then attempts toanswer the question and also generatessuggested questions related to the queryor selection. The questions supported bythe system were chosen to be education-ally useful, for example: what is thestructure of X? compare X and Y? howdoes X relate to Y? In user studies, stu-dents found this question-answeringcapability to be extremely useful whilereading and while doing problem solv-ing. In an initial controlled experiment,community college students using theInquire Biology prototype outperformedstudents using either a hard copy or con-ventional ebook version of the samebiology textbook. While additionalresearch is needed to fully developInquire Biology, the initial prototypeclearly demonstrates the promise ofapplying knowledge representation andquestion-answering technology to elec-tronic textbooks.

aspect of our approach is the use of human-comput-er interaction design methods to create a productthat combines these advanced AI technologies into auser experience that is compelling for students.Inquire Biology is one of the products of Project Halo,3

a long-term research effort to develop a capability toanswer questions on a wide variety of science topics.

Inquire Biology is based on Campbell Biology (Reeceet al. 2011), which is a widely used textbook inadvanced high school and introductory collegecourses. Campbell Biology is published by PearsonEducation, and we are using it under a research-uselicense. Using a popular textbook as the basis for ourresearch ensures that the content is pedagogicallysound and the results have immediate applicabilityto a large group of students already using the text-book. The basic concepts in the design of Inquire Biol-ogy are, however, broadly applicable to other text-books and scientific disciplines.

Exploration of the application of AI technology toan electronic textbook comes at an opportune timewhen we are witnessing a large-scale transition frompaper to electronic textbooks. Textbooks made avail-able in electronic medium offer tremendous oppor-tunities to leverage a variety of AI techniques muchbroader than what we have considered in our work sofar. Textbooks made available in electronic mediumoffer tremendous opportunities to leverage a varietyof AI techniques much broader than what we haveconsidered in our work so far, for example, recom-mending books (Pera and Ng 2012) and enhancingelectronic textbooks with content from onlineresources such as images and videos (Agrawal et. al.2011). Our focus on the use of knowledge represen-tation leverages the synergy between the need forprecise knowledge content in an electronic textbookand the strength of knowledge representation andreasoning to facilitate that.

We begin this article with a description of InquireBiology, introducing its key features and describinghow a student might use Inquire Biology for activereading and homework. We give an overview of theAI technology used in Inquire Biology and then pres-ent the results of a day-long pilot experiment show-ing that the students studying from Inquire Biologyreceived approximately 10 percent higher grades onhomework and posttest problems as compared to stu-dents studying from the paper or electronic versionof the textbook. We discuss the generalizability andscalability of the approach and conclude the paperwith a discussion on related work and directions forfuture work.

Features of Inquire BiologyInquire Biology connects three kinds of information tomeet student needs: the Inquire Biology textbook, con-cept summary pages, and computer-generatedanswers to questions.

The Campbell Biology textbook contains the origi-nal content of the textbook as published by PearsonEducation. The textbook content is hyperlinked tothe concept summary pages. These pages summarizesalient facts and relationships that a student needs toknow about a concept. Summary pages combine con-tent from the glossary entries from the back of thebook with automatically generated descriptions ofsalient properties and relationships for each of the5000 plus concepts in our knowledge base. Thesepages also include links to related concepts, relevantpassages from the book, and follow-up questions use-ful for further exploration.

A student can ask questions of Inquire Biology inthree ways. First, a question may be typed into a free-form dialogue box, from which Inquire Biology com-putes the closest questions that it can answer andgives the user a choice for selecting the best matchamong them. Second, in response to the studenthighlighting in the text, Inquire Biology computes themost relevant questions it can answer for the selec-tion. Finally, Inquire Biology suggests questions for fur-ther exploration on the concept summary pages andwith each page that contains an answer. We nextillustrate how students interact with these features ofInquire Biology.

TextbookFigure 1 shows the textbook interface of Inquire Biol-ogy. Like most electronic textbooks, Inquire Biologysupports highlighting the text, taking notes in themargin, and interacting with graphics. Inquire Biolo-gy leverages its knowledge base to expand on thesestandard features in the following three ways: (1) Thetoolbar provides a table of contents, index to conceptsummaries, and navigation history. Students can aska question at any time by tapping the Q icon. (2)Within the text, biology terms are automaticallylinked — students can tap to see a quick pop-up defi-nition or to navigate to the full concept summary. (3)Students can highlight with a quick and easy gesture,and each highlight serves as the anchor for a notecard and a list of related questions, encouraging stu-dents to dig deeper into the material.

Concept Summary PageFor every biology term in the book, Inquire Biologypresents a detailed concept summary page. Each ofthese pages begins with a human-authored definitionof a concept (marked as 1 in figure 2). Most of thesedefinitions are already available at the back of Camp-bell Biology, but some were added as part of theknowledge base (KB). The text definition is followedby key facts and relationships about that concept(marked as 2 in figure 2). These are automaticallygenerated from the KB. Since the KB is specific toCampbell Biology, the concept summary never has thesort of unnecessary details students might find onWikipedia or other general-purpose sites.

Articles

56 AI MAGAZINE

The sidebar of a concept summary page containsthe figures from the textbook that are most relevantto that topic (marked as 3 on figure 2). Finally, ques-tions relevant for further exploration of the topic arelisted (marked as 4 in figure 2). Knowledge in a con-cept summary page can span multiple chapters, butthe concept summaries put all the relevant facts andfigures in one place. Unlike most glossaries, theInquire Biology concept summary is not a dead endbut can act as a catalyst to further learning — pop-updefinitions and follow-up questions help studentsexplore relationships between concepts and dig deep-er into the material.

Asking QuestionsStudents can ask questions at any time, but they maynot always know what to ask. Inquire Biology is proac-tive, suggesting questions in a variety of contexts to

Articles

FALL 2013 57

help illustrate the types of questions Inquire Biologycan answer.

The question-asking interface, shown in figure 3,is invoked by tapping the Q icon. The student caneither enter a free-form question directly, or enter alist of biology terms. Inquire Biology will suggest ques-tions related to free-form questions as the studenttypes, helping students formulate their questions andoffering alternatives for cases where Inquire Biologycannot understand the free-form question directly.These suggested questions are in three main cate-gories: definitions and reference information, com-parisons between two concepts, and questions thatexplore relationships, such as how structure affectsfunction. These three types provide the best overlapbetween the system’s capabilities and the types ofquestions students find useful.

In response to each question, Inquire Biology returns

3

2

1

Figure 1. Textbook Interface.

The textbook content is from page 132 of Biology (9th edition) by Neil A. Campbell and Jane B. Reece; Copyright © 2011 by Pearson Edu-cation, Inc. Used with permission of Pearson Education, Inc.

a detailed answer with context to help students focuson what is important. Answers are designed to be con-cise and understandable, which means that differentanswers require different presentations.

Comparison questions compute the key differ-ences between two concepts, building on the learn-ing principle of conceptual contrast (Ausbel 1965).The result is displayed in a table, keeping the infor-mation well organized. Context is provided withshort definitions and superclass information at thetop, followed by important differences near the top.Figure 4 shows the answer to a comparison questionabout chloroplasts and mitochondria.

Answers to relationship questions are rendered asgraphs (figure 5), as these best highlight the connec-tions among concepts, building on the learning prin-ciple of helping students make sense of concepts byexploring relationships among concepts (Scar-damalia and Bereiter 2006). The graph is simplified

from what might exist in the underlying KB, withrelations designed to be clearly understandable to abiology student. Labels alone may not be enough —brief definitions of each term appear next to thegraph, plus the links to follow-up questions.

For more difficult problems, our goal is to supportstudent investigations. In accordance with the learn-ing principle of scaffolding (Bliss, Askew, and Macrae1996), Inquire Biology links to related concepts andsuggests follow-up questions, helping students askthe right questions and work their way toward a solu-tion. By providing such scaffolding, students canmake progress on more difficult investigations thanthey could handle unaided. In the current system,such scaffolding is always available to them. Further,by providing ready access to basic facts, Inquire Biolo-gy may reduce cognitive load (Sweller 1988) associat-ed with information retrieval and allow students tostay focused on higher-order learning objectives.

Articles

58 AI MAGAZINE

4

1

2

3

Figure 2. Concept Summary Page for Plasma Membrane.

This figure contains figure 6.6 from Biology (9th edition) by Neil A. Campbell and Jane B. Reece; Copyright © 2011 by Pearson Education,Inc. Used with permission of Pearson Education, Inc.

Concept of UseInquire Biology is designed to support two education-al use cases: active reading and homework support. Itis certainly possible to use Inquire Biology in a class-room and for exam preparation, but we did notinvestigate such use in our work. Here, we illustratehow a student might use its capabilities in the con-text of active reading and homework support.

Active ReadingActive reading is reading with a purpose, usually tounderstand a concept or to answer a question. Activereading has a strong track record in empirical learningresearch (Palinscar and Brown 1984). As a learningstrategy, it consists of four activities that take placebefore, during, and after reading a text: predict, ask,connect, and explain. The predict activity encouragesstudents to look ahead before reading a passage. Forexample, students may skim a section, or look atheadings, figures, and captions for organizationalcues. The student combines this preview with priorknowledge to form predictions of how the topic fitsinto the larger themes of the domain and to antici-pate the content that will follow. In the ask phase, stu-dents make questions to reflect on what they arelearning and are encouraged to create practice testquestions. In the connect phase, students relate thenew content to related knowledge, including person-al experience, current events, prior knowledge, andconcepts in other subjects. In the explain phase, stu-dents restate the read content in their own words,through notes, summarizations, or diagrams, such astables, flow charts, outlines, or concept maps. Fol-

lowing is a scenario to illustrate how a student mightengage in active reading while using Inquire Biology:

I’m reading section 10.2 on photosynthesis. I havealready highlighted some sentences in this section,and I encounter a term that I do not understand: thy-lakoid membrane. I tap on this term to see more infor-mation about it and I am taken to a concept summa-ry page. I am first shown the definition: thylakoidmembrane is the membrane surrounding the thy-lakoid. This definition is the same as the definition inthe paper version of the textbook and not particular-ly useful. It is followed by additional information thatis not directly available in the glossary of my papertextbook: that a thylakoid membrane is a type of dou-ble membrane. Further, I can see the parts that areunique to it: the electron transport chain and ATPsynthase. I can tap on electron transport chain andview detailed information about its parts such ascytochromes. From this summary page, I can also seethat the functions of a thylakoid membrane are lightreaction and photophosphorylation. But, what aboutthe functions of its parts? I can ask a question: Whatdoes an electron transport chain reaction do? I get ananswer: noncyclic electron flow. As I return to thetextbook, I can see that it is the next section that I amabout to read.

In this example, Inquire Biology has supportedactive reading in the following ways:

Connect: The pop-up definitions and concept sum-mary pages help students connect to additionalmaterial and allow a quick and seamless way torefresh and review previous knowledge. In thisexample, the student was able to review the structureand function of thylakoid membrane, and alsoobtained more detailed information about some of

Articles

FALL 2013 59

ask a question

de�ne

structure What is the structure of a chloroplast?

function

compare

relate

search

De�ne cellular respiration

What is the function of a plasma membrane in a eukaryotic cell?

What are the differences between chloroplasts and mitochondria?

If the chloroplasts were removed from a plant, what events would be affected?

Search book for photosynthesis

Figure 3. Asking Questions with Assistance.

its parts — for example, electron transport chain.Explain: Inquire Biology’s highlighting and note-tak-

ing capabilities allow students to mark passages theyfind particularly important and to extend and sum-marize the information in their own words. InquireBiology also explains the basics of a concept — forexample, in this case it points out that a thylakoidmembrane is a double membrane.

Ask: Inquire Biology suggests questions whenever a

student makes a highlight. Related questions appearon concept summary pages and answers, and stu-dents can tap to see Inquire Biology’s answer. In addi-tion Inquire Biology allows the student to ask free-formquestions. In this example, the student asked a ques-tion about the function of an electron transportchain, deepening the understanding of the overallconcept of a thylakoid membrane.

Predict: Students can attempt to answer Inquire Biol-

Articles

60 AI MAGAZINE

Figure 4. Comparison Question.

This figure contains figures 10.17, 6.16, 10.8, and figure 6.18 from Biology (9th edition) by Neil A. Campbell and Jane B.Reece. Copyright © 2011 by Pearson Education, Inc. Used with permission of Pearson Education, Inc.

ogy’s suggested questions on their own and test theirpredictions by tapping through to see the generatedanswers. In this example, the answer to a student’squestion about the function of an electron transportchain leads to a preview of what is coming up next inthe textbook, appropriately setting up the student’slearning goals.

Homework SupportInquire Biology assists students in understanding and

constructing answers for complex conceptual home-work problems. Inquire Biology generally cannotanswer complex homework questions directly, butprovides support in three ways: (1) reducing theemphasis on memorization by providing quickaccess to facts and relationships needed to solve ahomework problem, (2) comparing and relating con-cepts that may not necessarily be discussed in thesame section of a textbook, and (3) deconstructingproblems by suggesting simpler questions. We illus-

Articles

FALL 2013 61

Figure 5. Relationship Question.

This figure contains figures 5.3, 5.5, 5.6 and 5.7 from Biology (9th edition) by Neil A. Campbell and Jane B. Reece. Copyright© 2011 by Pearson Education, Inc. Used with permission of Pearson Education, Inc.

trate this capability by considering a homeworkproblem from section 6.5 of Campbell Biology (figure6). The scenario below suggests how a student mightuse Inquire Biology to understand and answer thisproblem.

To answer the first question, I ask Inquire Biology: Whatis the similarity between a chloroplast and a mito-chondrion? Inquire Biology handles this questiondirectly and gives me two nicely organized tables: onelisting the similarities and the other the differencesbetween a mitochondrion and a chloroplast. Theanswer to the similarity question tells me that bothhave a function of energy transformation. The simi-larity answer gives me little information about struc-ture. So, I review the answer to the differences section(See figure 4). Under the structure section of theanswer, I am told that a chloroplast has a chloroplastmembrane and a mitochondrion has a mitochondrialmembrane. They are named differently, but seem sim-ilar, and since the first problem asks me to considerstructure, I ask the question: What is the similaritybetween a chloroplast membrane and a mitochondri-al membrane? In the answer, I am told that they areboth double membranes. I bet that this is one of thekey structural features that should be an answer to theproblem.

To answer the second question, I can directly askInquire Biology factual questions such as: Is it true thatplant cells have a mitochondrion? Inquire Biology tellsme that the answer is: Yes. This straight fact retrievalreduces the need to memorize, but developing anexplanation is more challenging. I could ask the ques-tion: What is the relationship between a mitochon-drion and a plant cell? Inquire Biology presents a graphshowing the relationship between the two. This helpsme develop an explanation that a mitochondrion is apart of plant cells and functions during cellular respi-ration.

The third question is open ended, and Inquire Biologyhelps me in deconstructing this question into simplerquestions. When I highlight this question in InquireBiology, it suggests several questions some of whichare: What is an endomembrane system? What are ele-ments/members of an endomembrane system? Whatare the differences between a mitochondrion andendoplasmic reticulum? What event results in mito-chondrial membrane? What event results in endo-plasmic reticulum membrane? None of these ques-tions directly answers the third question, but they giveme starting points for my exploration that might helpme eventually construct an argument of the kind thatthis problem is asking for.

In the previous scenario, we can see that InquireBiology does not directly answer a homework prob-lem, but provides answers to portions of the ques-tion, which a student must then assemble to con-struct an overall answer. Through this process, itprovides an easier access to facts and relationships,but the student is still required to develop a concep-tual understanding of the material to answer a prob-lem. By using Inquire Biology’s question-answeringfacility, students may not necessarily be able to dotheir homework faster, but we do expect them toacquire a deeper understanding of material and bemore engaged in doing their homework. The fact-retrieval questions as we saw in the solution of thesecond question help the student learn basic facts,and the comparison question that we saw in the firstquestion relates pieces of information that may notbe presented next to each other in the textbook. Thisreduces cognitive load and encourages a focus onconceptual understanding.

Articles

62 AI MAGAZINE

CONCEPT CHECK 6.5

1. Describe two common characteristics of chloroplasts and mitochondria.Consider both function and membrane structure.

2. Do plant cells have mitochondria? Explain.

3. A classmate proposes that mitochondria and chloroplasts should be classi�ed in the endomembrane system. argue against the proposal.

Figure 6. An Example Homework Problem.

Concept check 6.5 is from Biology (9th edition) by Neil A. Campbell and Jane B. Reece. Copyright (c) 2011 by Pearson Educa-tion, Inc. Used with permission of Pearson Education, Inc.

AI Technology in Inquire BiologyWe now give an overview of the AI technology thatenables the functioning of Inquire Biology. Detaileddescriptions of various system components are avail-able in previously published papers (Chaudhri et al.2007, Clark et al. 2007, Gunning et al. 2010).

The overall system consists of modules for creatingknowledge representation from the book, modulesfor performing inference and reasoning methods onthe knowledge, and modules for asking questionsand presenting answers to the student. Figure 7 illus-trates the functional components of the system.

For the purpose of Inquire Biology, a subject-matterexpert (SME) is a biologist with a bachelor’s degree inbiology or a related discipline. The SME uses a knowl-edge authoring system and the authoring guidelinesto represent knowledge from the textbook contribut-ing to a growing knowledge base. Inquire Biology usesAURA as its knowledge authoring system. Details onAURA have been published in AI Magazine (Gunninget al. 2010). The authoring guidelines specify aprocess by which SMEs systematically read the text-book content, arrive at a consensus on its meaning,encode that meaning, test their encoding by posingquestions, and validate the knowledge for its educa-tional utility.

AURA uses the knowledge machine (KM) as itsknowledge representation and reasoning system(Clark and Porter 2012). KM is a frame-based repre-sentation language that supports a variety of repre-sentational features including a class taxonomy, arelation taxonomy, partitions (disjointness and cov-

ering axioms), rules (attached to a frame’s slots), andthe use of prototypes (canonical examples of a con-cept). The knowledge base itself uses an upper ontol-ogy called the component library or CLIB (Barker,Porter, and Clark 2001). The CLIB is a domain-inde-pendent concept library. The SMEs access the CLIBthrough AURA and use it to create domain-specificrepresentations for knowledge in Campbell Biology,resulting in a biology knowledge base. The currentknowledge base has been well-tested for the ques-tions used in Inquire Biology for chapters 2–12 ofCampbell Biology.

KM also provides the core inference services forAURA. KM performs reasoning by using inheritance,description-logic-style classification of individuals,backward chaining over rules, and heuristic unifica-tion (Clark and Porter 2012). A focus of innovationin our work has been to identify these core featuresand to specify them in a declarative manner(Chaudhri and Tran 2012). We export the biology KBin a variety of standard declarative languages, forexample, first-order logic with equality (Fitting1996), SILK (Grosof 2009), description logics (DLs)(Baader et al. 2007) and answer-set programming(Gelfond and Lifschitz 1990).

In addition, AURA incorporates several special-purpose reasoning methods for answering compari-son and relationship reasoning questions (Chaudhriet al. 2013). We briefly explain the computation usedin these methods. For comparison questions, we firstcompute the description of a typical instance of eachclass in terms of its types and locally asserted slot val-ues and constraints. Next, we compute the similari-

Articles

FALL 2013 63

AuthoringGuidelines

KnowledgeAuthoring

SystemDomainExpert

Textbook

Legend

User

servercomponents

client (app)components external

KRLanguage

and ConceptLibrary

KnowledgeBase Inference

SuggestedQuestion

Generation

QuestionAnsweringMethods

AnswerGeneration

SuggestedQuestionSelection

Inquire UserInterface

QuestionInterpretation

AnswerPresentation

Figure 7. System Architecture

ties and differences between the two descriptions.The result is organized into a table. We use a varietyof presentation heuristics to identify the most impor-tant similarities and differences and to present thedifferences that correspond to each other in the samerow of the table. As an example heuristic, a slot thatindeed has a value that is different is shown before aslot in which there is a value for one class but not avalue for another. The presentation heuristics alsorely on input from biologists on how the slots shouldbe organized. For example, the slots such as has partand has region are grouped together as they corre-spond to structural information. To compute therelationship between two individuals, we first calcu-late all paths in the knowledge base between twoindividuals. Since calculating all paths is expensive,we use heuristics to control the search. As an exam-ple, we first explore only taxonomic relationshipsand then relationships that can be found in a singleconcept graph. The computed paths are ranked inthe order of importance. As in the case of comparisonquestions, the importance is computed using heuris-tics provided by the biologists. As an example, thepaths involving only structural slots are preferredover paths that contain arbitrary slots.

Inquire Biology works through an HTTP connectionto an AURA server running on a Windows machine.The AURA server contains the biology knowledgebase and supports the question-answering and ques-tion-suggestion facilities, the details of which havebeen previously published (Gunning et al. 2010).AURA also pre-computes all the concept summarypages, which are included in the Inquire Biology appli-cation during the build process.

The question-answering facility in AURA relies ona controlled natural language understanding systemthat was described by Clark et al. (2007) and Gun-ning et al. (2010). With the direct use of controllednatural language, the students often had difficultyanticipating which questions the system can or can-not handle. To address this issue, AURA supports asuggested question facility. In response to questionsentered by the student, AURA computes the mostclosely matching questions that can be answered andpresents these suggestions to the student. This com-putation relies on a large database of questions thatare automatically generated by traversing the knowl-edge base. For example, for two concepts that are sib-lings of each other in the class taxonomy, the systemwill generate a comparison question that asks forsimilarities and differences between the two con-cepts. Questions are generated that conform to thetemplates that are supported in the system. As thestudent types a question and even partially entersquestions, the precomputed database of questions isused to find most closely matching questions andpropose them as suggestions. This approach substan-tially reduces the occurrence of questions that can-not be parsed by AURA and greatly enhances the

usability of the question-answering facility. Ourapproach to question generation is similar to the spir-it of other recent work on question generation4 withone important difference: our system has access to adetailed knowledge representation of the textbookthat can be used for question generation while mostother question-generation systems attempt to gener-ate questions from the surface representation of thetext.

After the inference and question-answering com-ponents produce the basic content of an answer, theanswer-generation module constructs the full,human readable answer. Each kind of answer has aspecific screen layout and uses a mixture of presenta-tion types that include bulleted lists, graphs, and nat-ural language statements to describe the base factsfrom the knowledge base. The answers leverage a nat-ural language generation component that automati-cally generates English sentences from the KB (Baniket. al. 2012).

The Evaluation of Inquire BiologyThe goal of evaluating Inquire Biology was to assess theextent to which the AI enhancements to CampbellBiology were useful to students for the active readingand homework support tasks, and to determinewhether Inquire Biology leads to better learning. Theformal evaluation of Inquire Biology was preceded bya series of user studies, which we used to refine InquireBiology’s capabilities and ensure that the system wasusable. In discussing the evaluation, we describe theexperimental design, participant profile, procedureused, and both qualitative and quantitative results.

Experimental DesignOur evaluation used a between-subjects research

design, with students randomly assigned to one ofthree conditions. The full Inquire Biology group (N =25) used the system version that had all the AI-enabled features that we discussed earlier in thispaper. The textbook group (N = 23) used a standardprint copy of Campbell Biology. The ablated InquireBiology group (N = 24) used a version of Inquire Biolo-gy that lacked the concept summaries and question-answering capabilities but included all the otherebook features such as highlighting and annotation.The number of subjects in each group was chosen toensure that the results are statistically significant.

The purpose of these three groups was to comparestudying with Inquire Biology to two forms of existingpractice: studying with a paper textbook and study-ing with a conventional ebook reader version of atextbook. We wished to control for unexpectedeffects that may arise from the iPad platform, or idio-syncrasies with the Inquire Biology interface. Specifi-cally, we wanted to avoid criticisms that anyimprovements observed by using Inquire Biology weresimply due to the excitement of using an iPad and

Articles

64 AI MAGAZINE

that use of an unenhanced electronic textbook low-ers student performance as compared to using apaper textbook.

In actual practice, the students might use lecturenotes, Wikipedia, and other teacher-providedresources during active reading and homework prob-lem solving. However, we did not incorporate theseinto our study because of the huge variation in avail-able supplementary resources.

ParticipantsWe recruited 72 participants from a local communi-ty college. The number of subjects was chosen toensure that the study had sufficient power to detectexpected differences. All the participating studentswere enrolled in an introductory biology course thatused the same edition of the Campbell Biology text-book that is contained within Inquire Biology. The stu-dents were prescreened to ensure that they had notpreviously covered the sections of the text used inthe evaluation. Based on the student backgroundvariables, such as grade point average, age, gender,educational background, native language, and famil-iarity with iOS devices, the three groups were com-parable at the onset of the study. All students werefinancially compensated for their time; they did notreceive any academic credit for their participation;and, there was no attrition of the participants duringthe exercise.

ProcedureAll groups performed an identical learning task: theywere asked to study a unit on cellular membranestructure and function. We chose this topic as it isfundamental to cell biology, and students frequentlyhave difficulty understanding the material. Students’specific learning objectives were to (1) understandspecific components of membranes and their func-tions, (2) understand what specific components ofmembranes contribute to their selective permeability,and (3) explore how structure influences function atmolecular and macromolecular levels.

The evaluation took place outside of the classroomto ensure that study conditions were the same foreach group. Each session took place on a single dayand lasted between four and six hours depending onthe group. The evaluation consisted of four primarycomponents: introduction and training, active read-ing task, homework support task, and a posttest. Inaddition, a researcher interviewed students in theInquire Biology groups afterwards to better understandtheir experience.

The introduction and training consisted of an ori-entation to the study and a 30-minute training exer-cise designed to familiarize participants with theprocess of active reading and how they could useInquire Biology for homework problem solving. Thetraining on active reading and homework problemsolving was conducted by a biology teacher.

The active reading training introduced the stu-dents to habits of active reading (predict, ask, con-nect, and explain), and illustrated these habits usingan example. The example used for this training wasgravitation, and was specifically chosen to be unre-lated to the topic of the actual evaluation. After a lec-ture on active reading, students were given a script-ed tutorial on active reading. This tutorial used asection from chapter 6 of Campbell Biology that con-cerned cell structure and provided step-by-stepinstructions on how a student could engage in activereading using the features of Inquire Biology, ablatedInquire Biology, or the print version of the text.

The training segment for problem solving focusedon understanding homework problems, differentquestion types, understanding answers, and con-structing follow-up questions. The training also cov-ered question formulation for questions that contain“how” and “why” in the question statement. Manysuch questions can be rephrased as questions that aresupported in Inquire Biology (“How” and “Why”questions are not directly supported). The trainingsegment also included a five-minute video illustrat-ing how a student can use Inquire Biology for solvinga nontrivial homework problem.

Most of the active reading training was adminis-tered to all three groups. However, only the fullInquire Biology group received training on usingInquire Biology features for active reading and home-work problem solving.

The problem sets used for homework and theposttest were designed by a biology instructor. Thehomework problem set had six problems (with sev-eral subparts), and the posttest had five problems.Both problem sets included problems that theinstructor would normally use during the teachingof the course and were not specifically aligned to thecapabilities of Inquire Biology. The problem sets weretested for any obviously confusing information byfirst trying them with students in the user studiespreceding the evaluation. This ensured that any con-fusing information about the problem statementwould not confound the results. Students were ableto use their book while doing the homework exer-cises but not during the final posttest.

Students were asked to read the first two sectionsof chapter 7 from Campbell Biology in 60 minutes, dohomework in 90 minutes, and then take a posttestin 20 minutes (table 1). The exercise was followed byan informal discussion session in which the studentscompleted usability and technology-readiness sur-veys and provided qualitative feedback. The stu-dents’ homework and posttest were each scored bytwo teachers, and score discrepancies were resolved.All student work was coded such that the teachershad no knowledge of which student or conditionthey were scoring.

Articles

FALL 2013 65

Quantitative Results from The PosttestFigure 8 shows the quantitative posttest results. Boththe homework and posttest tasks had a maximumscore of 100. The scores in figure 8 show the meanscore of students in each group.

Our primary interest is in the effect on the posttestscore of Inquire Biology as compared to the two con-trasting, more typical conditions. The mean posttestscore of 88 for the Inquire Biology group was higherthan both the corresponding score of 75 for theablated Inquire Biology group and the score of 81 forthe textbook group. The difference between theInquire Biology group and the ablated Inquire Biologygroup was statistically significant (p value = 0.002).The difference between the Inquire Biology group andthe textbook group was also significant (p value =0.05). These differences suggest that students whoused Inquire Biology learned more.

The mean homework score of 81 for the InquireBiology group was higher than both the correspon-

ding score of 74 for the ablated Inquire Biology groupand a score of 71 for the textbook group. The differ-ence between the Inquire Biology group and the ablat-ed Inquire Biology group was not statistically signifi-cant (p value = 0.12), but the difference between theInquire Biology group and the textbook group was sig-nificant (p value = 0.02). The observed trend is con-sistent with our hypothesis that Inquire Biologyenhances learning by helping students perform bet-ter on homework.

The comparison between the paper textbook andthe ablated version of Inquire Biology is also interest-ing. Although the mean homework score of 74 forablated Inquire Biology and a mean score of 71 for thepaper textbook differ, there was no statistical differ-ence between these conditions. Similarly, there wasno statistical difference between the homeworkscores of the two conditions (p value = 0.18). Thisresult is consistent with the prior research that simplychanging the medium of presentation has no impacton student learning (Means et al. 2009).

Articles

66 AI MAGAZINE

1 hour 2 3 4 5

intro Training Active reading task (60 minutes)

Lunch Homework support task

(90 minutes)

Posttest

(20 minutes)

Debrief

Table 1. Typical Structure for Each Condition.

HOMEWORK SCORES QUIZ SCORES

A

B

C

D

INQUIRE ABLATED INQUIRE PAPER BOOK

81

74

88

75

81

71

P-value from 2 tailed t-tests:HW Scores Quiz ScoresFull versus Ablated: 0.12 Full versus Ablated: 0.002Full versus Textbook: 0.02 Full versus Textbook: 0.05Ablated versus Text: 0.52 Ablated versus Text: 0.18

Figure 8. Quantiative Posttest Results

We gathered extensive data on how the differentfeatures of Inquire Biology were used but did not dis-cover any significant correlation between the usageof a particular feature and the improvement onposttest performance.

We did not explicitly measure to what extent stu-dents followed the specific steps of active reading, orto what extent they acquired deeper understandingof knowledge, or whether they were more engagedwith the material. The posttest scores are a measureof depth of understanding, and increased engage-ment was apparent in their qualitative feedback,which we discuss next. Additional analysis could bedone by analyzing their note taking behavior, butsince it was orthogonal to the core AI features thatwere the subject of the study, we left it open forfuture work.

Qualitative ResultsWe gathered three forms of qualitative data to assessthe usefulness of Inquire Biology. The data includedthe usage of Inquire Biology features as well as resultsfrom a usability survey and a technology readinesssurvey.

We tracked the questions asked by students andthe concept summary pages visited by them. All stu-dents in the full Inquire Biology group asked questionsand visited concept summary pages. Students vieweda total of 81 concept summary pages during the exer-cise; 38 of these were viewed by at least two students.Approximately 30 percent of the questions were ofthe form “What is X?”; another 30 percent were fac-

tual questions; and the remaining 40 percent wereevenly split between comparison questions and rela-tionship questions. This usage data is suggestive ofthe usefulness of these features to students (see figure9).

During the active reading task, students cumula-tively asked 120 questions, and during the home-work task they asked a total of 400 questions. Thisdifference is expected, as students naturally have agreater need for asking questions during the home-work task. During active reading, the question askingpredominantly came from students clicking on ques-tions suggested in response to their highlighting oftext. During homework problem solving, however,students were more prone to use Inquire Biology’squestion-asking dialogue. A total of 194 uniquequestions were asked by the students, out of whichonly 59 questions were asked by two or more stu-dents. The variety of questions would make it verydifficult to anticipate all questions in the textbookitself. This suggests that automatic question answer-ing can be a useful addition to a textbook.

In figure 10, we show the actual questions that thestudents asked and how many different studentsasked each question. It can be seen that there is agood spread of different question types even thoughthe questions of the form “What is X” predominat-ed the mix.

After use, the students rated the usability of InquireBiology using the SUS scale (Brooke 1996). The SUSscore for ablated Inquire Biology was 84.79 with astandard deviation of 11.82, whereas the score for

Articles

FALL 2013 67

Reading and highlighting

Viewing glossary popup

Writing notes

Viewing images/graphs

Viewing glossary pages**

Asking questions*

* omitted feature in ablated Inquire** limited feature in ablated Inquire

0

15

30

45

60

75

90m

inut

es

ABLATEDINQUIRE

ABLATEDINQUIRE

INQUIRE INQUIRE

ACTIVE READING TASK HOMEWORK TASK

Figure 9. Average Student Usage Time by Activity.

full Inquire Biology was 84.4 with a standard deviationof 9.02. This suggests a high degree of subjective sat-isfaction in using Inquire Biology as well as no degra-dation in usability from the addition of AI-based fea-tures.

The students also completed a survey to providesubjective ratings of various features of Inquire Biolo-gy. The students were asked to indicate whether aparticular feature was ready for use, ready for usewith some improvements, or required majorimprovements. The concept summary pages had thehighest score with 76 percent of students rating themas ready to use in the present form, 16 percent ratingthem as requiring minor improvement, and 8 per-cent rating them as requiring major improvements.For the question-answering facility, 51 percent of thestudents rated it as ready to use in present form, 37percent rated it as requiring minor improvement,and 12 percent rated it as requiring major improve-ments.

In the posttask debrief, students in the Inquire Biol-ogy group unanimously reported that they would liketo use the system for their class. They reported thatInquire Biology “motivates you to learn more” and“helps you stay focused.” Students also reported that

they felt less distracted as they could use Inquire Biol-ogy to get additional information without losing con-text. Usage logs show students made use of questionanswering and concept summary pages during theactive reading task, which suggests that they wereengaged in the content and actively seeking out relat-ed and supporting information. A more extensiveevaluation of how Inquire Biology improves engage-ment is open for future work.

Overall, the qualitative assessment of the InquireBiology features was positive but also indicated someareas for improvement. Students reported that thecomparison answers were “well organized” and “tothe point,” but they also commented that the rangeof questions supported needs to be expanded and theanswer quality still needs to be improved.

Generalizabiltiy and ScalabilityLet us now consider how the process of knowledgebase construction generalizes and scales potentiallyto textbooks other than biology. Our most substantialexperience in using AURA is with Campbell Biology. Inaddition to biology, we have prior experience in

Articles

68 AI MAGAZINE

What are the differences between a �uid mosaic model and a sandwich model?What is a transmembrane protein?

What is the relationship between a biomembrane and a plasma membrane?What is a glycoprotein?

What is a channel protein?Is the a sandwich model generally accepted or not?

Where is a channel protein located?What is a glycolipid?

What is a sandwich model?What is cholesterol?

What are the differences between a sandwich model and a �uid mosaic model?What is a membrane carbohydrate?

What is a peripheral protein?What is an amphipathic molecule?

What is an integral protein?What is an integrin?

What is the relationship between a peripheral protein and an amphipathic molecule?What are the differences between a glycolipid and a glycoprotein?

What are the differences between a saturated fatty acid and a unsaturated fatty acid?What are the differences between a saturated fatty acid and an unsaturated fatty acid?

What is a �uid mosaic model?What is a phospholipid?

What is a plasma membrane?What is the relationship between a channel protein and an amphipathic molecule?

What are the differences between a biomembrane and a phospholipid layer?What are the differences between a lipid and a protein?

What are the differences between a peripheral protein and a transmembrane protein?What are the differences between an integral protein and a peripheral protein?

What do cholesterol maintain?What do glycoproteins do to cholesterol?

What is a carbohydrate?What is a hydrophobic core?

What is a transmembrane protein across?What is the difference between a biomembrane and a plasma membrane?

What is the relationship between a sandwich model and a �uid mosaic model?What is the structure of a glycolipid?

0 4 8 12 16 20 24

Figure 10. Number of Students Who Asked a Question

using AURA for physics and chemistry(Gunning et al. 2010). We have alsoperformed a design-level analysis forthe domains of government and poli-tics, microeconomics, and environ-mental science. Within the domain ofbiology, we have considered AURA’sapplicability to representing a middleschool textbook and to advanced text-books on cell biology and neuro-science. We can draw the followingconclusions from these studies.

Conceptual knowledge (ability todefine classes, subclasses, stating dis-jointness, defining slot values andrules), mathematical equations, andqualitative knowledge (for example,directly or inversely proportional) cutacross all domains and are widelyapplicable. Such knowledge canaccount for at least half of the knowl-edge that needs to be captured in atextbook. Inquire Biology demonstratesthat a useful enhancement to an elec-tronic textbook can be built usingthese core forms of knowledge. Theapproach is directly applicable with lit-tle changes to any textbook that iscomparable in scope to Campbell suchas Mason, Losos, and Singer (2011) ora middle school textbook (Miller andLevine 2010). The conceptual knowl-edge, qualitative knowledge, andmathematical equation features ofAURA generalize to any of the domainsmentioned above.

Outside the context of an AIresearch project, one possibleapproach to achieve scalability of theapproach in practice is for the knowl-edge representation of the content ofthe book to become an integral part ofthe book authoring process. This newrequirement is no different than thepreparation of table of contents, index,or glossary that is typically found instandard textbooks. The books of thefuture could include knowledge repre-sentation for the most salient portionsof the content, which could enablenumerous pedagogical features someof which are illustrated in Inquire Biol-ogy.

AURA does not cover all forms ofknowledge found across multipledomains. Some domains are moremathematical than others, for exam-ple, physics, algebra, or economics. Forsuch domains, the reasoning capability

of the system needs to include mathe-matical problem solving. The text-books such as chemistry and algebraalso require capturing proceduralknowledge. A typical example of suchknowledge in chemistry involves stepsfor computing pH of a buffer solution.While solutions exist for mathemati-cal problem solving and capturingprocedural knowledge, their combina-tion with conceptual knowledgerequires novel research. The back-ground knowledge provided in CLIBneeds to be extended for each text-book, requiring some research in con-ceptual modeling and application ofthis research by the domain experts inthe respective domain. New textbooksubjects also require developing newquestion types and answer-presenta-tion methods.

Related WorkThe work presented here directly over-laps with three related areas ofresearch: electronic textbooks, ques-tion-answering systems, and readingfor comprehension. In this section, webriefly situate our work in these threeareas of related research.

Electronic TextbooksTo inform the initial design of InquireBiology, in early 2010 we evaluatedexisting ebook apps and electronictextbooks to develop a list of key fea-tures and solutions to common prob-lems. Based on this analysis, we deter-mined that students would expectInquire Biology to save their places inthe book and make it easy to takenotes, highlight text, zoom in onimages, and search the concept sum-maries. We also saw many approachesto displaying the highly structuredcontent of textbooks, from scaled-down images of actual pages (Kno5) toentirely custom iPad-optimized lay-outs (Apple iBooks textbooks). Wechose to render the book content as avertically scrolling HTML page, as thatgave us an optimal combination ofreadable text, flexible layouts, goodperformance, and a reasonable level ofeffort.

Standing apart from the other iPadtextbook apps is Inkling,6 an appreleased in August 2010, around the

Articles

FALL 2013 69

time we built the first alpha versions ofInquire Biology. Inkling 1.0 had all themajor features discussed above and,like Inquire Biology, rendered content asspeedy HTML rather than the slowerscans of book pages. Since 2010,Inkling has added several features thatmay address some of the specific needswe target with Inquire Biology: provid-ing definitions of key terms throughintegrating Wikipedia into the app’ssearch feature, and shared notes thatallow students to communicate withtheir classmates and ask each otherquestions about the material. BecauseAURA’s answers never stray from thecontent of Campbell Biology, they aremore accurate and better scoped thanwhat one might get from Wikipedia orfrom one’s peers. Inkling’s work ishowever an important step forward forstudents and an indication of thedegree of rapid improvement that isoccurring in the electronic textbookdomain.

Question-Answering SystemsA survey of recent question-answeringsystems has been published in AI Mag-azine (Gunning, Chaudhri, and Welty2010). For the present discussion, wewill compare AURA to Watson andWolfram Alpha.

Watson is a recent question-answer-ing system from IBM that outper-formed humans in the game of Jeop-ardy (Ferrucci 2012). Watson is animpressive system that showed per-formance over a broad domain of ques-tion answering. One of the primarycharacteristics of the Jeopardy problemis that 94.7 percent of the answers toquestions are titles of some Wikipediapage (Chu-Carroll and Fan 2011). Theremaining 5.3 percent of the answersthat are not the titles of the Wikipediapages include multiple entities such asred, white, and blue, or even short sen-tences or phrases, such as make a scarecow. Since there are more than 3 mil-lion Wikipedia pages and a huge vari-ety of questions, this is not an easytask, yet it is constrained in a way thatmakes it amenable to solution bymachine-learning techniques. In con-trast, the answers returned by AURAcan span from a paragraph to a page —especially the answers to the compari-son and relationship reasoning ques-

text structure before reading can serveas a form of advance organizer thatcues existing knowledge, supportsstrategic reading, and permits readersto mentally record new knowledgemore effectively (Ausubel and Youssef1963; Mayer 1979). Providing readerswith opportunities to check theirknowledge as they read supports activeself-regulation of learning (Bransford,Brown, and Cocking 1999; Brint et al.2010). Prompting readers to pose ques-tions as they read is also useful formetacognitive monitoring, improvingcomprehension, and making connec-tions among ideas (Novak 1998; Press-ley and Afflerbach 1995; Udeani andOkafor 2012). In addition, couplingvisual elements in textbooks, such asdiagrams, with text can also have quiterobust effects on student learning(Mayer 2003), especially for studentswho prefer visual information (Mayerand Massa 2003).

FutureInquire Biology is an early prototype ofan intelligent textbook. Our resultsshow the promise of applying AI tech-nology to electronic textbooks toenhance learning. To fully realize thispromise, more work needs to be donein at least the following categories:improving the representation, improv-ing question-answering capability,conducting more extensive educationstudies, and improving the learninggains. We discuss each of these direc-tions in more detail.

Improving Representation and Question AnsweringAs of early 2013, Inquire Biology pro-vides very good coverage of chapters2–12 of Campbell Biology, and somecoverage of chapters 13–21, 36, and 55.It currently handles factual, compari-son, and relationship reasoning ques-tions. The knowledge representationneeds to be both deepened andexpanded to cover the remainingchapters. The current representationhandles knowledge about structureand function, process regulation, andenergy transfer. Additional representa-tion constructs are needed in CLIB forrepresenting experiments, evolution,continuity and change, and interde-

pendence in nature. Expanding theknowledge base to the full textbookrequires ensuring the scalability of theknowledge authoring system in AURA.Additional work also needs to be doneon developing new reasoning methodssuch as for process interruption rea-soning and to expand the range ofquestions that the system currentlysuggests. The usability of the systemwill also improve substantially byexpanding the range of English thatcan be accepted as input questions.

More Extensive Educational StudiesThe evaluation considered in this arti-cle lasted for only a few hours over asingle day. It remains to be shown ifsimilar improvements in student learn-ing can be obtained if the students areallowed to take Inquire Biology homeand study from it over a period of sev-eral weeks. The current study suggestedthat Inquire Biology may be especiallyuseful for lower-performing students;additional students need to be testedto validate this trend. We also need tobetter understand which specific fea-tures of Inquire Biology contributed tothe improvement in learning that weobserved.

Improving Learning GainsCurrent results show that the use ofInquire Biology improved the posttestscores by approximately 10 percent. Anatural question is whether this is themaximum improvement possible. Sev-eral new educational supports can beincluded in Inquire Biology to support abetter model of student comprehen-sion, identifying student-specific prob-lem areas to provide targeted support,and incorporating better problem solv-ing dialogue capabilities. In the longterm it could have a full-fledged intel-ligent tutoring system.

Summary and ConclusionsInquire Biology is an innovative applica-tion of AI technology to an electronictextbook. The key ideas in Inquire Biol-ogy are to provide automatically gener-ated concept summary pages and sug-gested questions, and to provideanswers to comparison and relation-ship questions. This is enabled by tak-

tions. Furthermore, these answers arenot always stated in the textbook, andeven if they are, they may be stated indifferent parts of the textbook. In addi-tion, Inquire Biology is an interactiveuser-centered application and hasmuch higher usability demands.

Wolfram Alpha is a question-answer-ing system based on Mathematica, acomputational reasoning engine.7

Wolfram Alpha relies on well-definedcomputations over curated data sets.For example, when given a questionsuch as “orange and banana nutri-tion?” it looks up the nutritional infor-mation for both orange and banana,adds them, and presents the net result.When given an equation such as g(n +1) = n2+g(n), Wolfram Alpha gives itssolution and presents a plot showingthe value of the solution as a functionof n. Wolfram Alpha is similar in spiritto AURA in the sense that it provideswell-defined computations over curat-ed data, but very different in terms ofthe curated data and the specific com-putation. For AURA, the knowledge iscurated from a biology textbook, andthe computations are based on logicalreasoning rather than mathematics.

Reading ComprehensionAchieving the potential offered byebook technology will depend on arich understanding of how studentslearn from college-level science text-books. Past research indicates that asthe information complexity of suchtextbooks increases, the explanatoryclarity, or text cohesion, declines(Graesser et al. 2004). To succeed, sci-ence students must move from decod-ing to sense-making, which involvesmaking connections between priorknowledge and new information(Kintsch 1988; Palinscar and Brown1984; Scardamalia et al. 1996), yetresearch indicates that many collegestudents find it difficult to read sciencetextbooks because of gaps in theirunderstanding of science concepts(diSessa 1993; Driver and Easley 1978;Ozuru, Dempsey, and McNamara2009).

Inquire Biology features promoteactive reading strategies that can helpreaders construct better models of theconcepts they are learning. Primingreaders to focus on their goals or the

Articles

70 AI MAGAZINE

Articles

FALL 2013 71

ing advanced AI technologies andblending them into an ebook to pro-vide a seamless user experience. Mostnatural language question-askinginterfaces suffer from the problem thatthe student does not have a clear senseof what kind of questions the systemcan handle. Inquire Biology addressesthis by the use of suggested questions.The evaluation results show that stu-dents find Inquire Biology usable andengaging, and that they learned more.Although this result is from a prelimi-nary evaluation study, it is an impor-tant and significant result as it is one ofthe first to conclusively demonstratethe usefulness of explicitly representedknowledge and question answering asstudents study from their assigned sci-ence textbook.

When students transition from printtextbooks to electronic textbooks, theywill learn more only if the new mediabetter supports their learning process.To date, most work with electronictextbooks reproduces only capabilitiessuch as highlighting and annotation,which are already available for papertextbooks. By using knowledge repre-sentation, question-answering, andnatural language generation tech-niques from AI, we have shown that anelectronic textbook can go beyondwhat is possible in paper books. Specif-ically, knowledge representation cansupport students in asking and answer-ing questions about the relationshipsamong the large number of conceptsthat are newly introduced in challeng-ing science courses. By helping stu-dents understand the relationshipsamong concepts, an AI-enriched text-book has the potential to increase stu-dents’ conceptual understanding aswell as their satisfaction with studymaterials. We hope that Inquire Biologywill provide inspiration for a variety ofAI methods to provide supports forpersonalization and leveraging onlineresources in electronic textbooks thatfacilitate the development of inquisi-tive scientists and engineers that are somuch needed.

AcknowledgementsThis work has been funded by VulcanInc. The authors wish to thank themembers of the Inquire Biology devel-opment team: Eva Banik, Roger Cor-

man, Nikhil Dinesh, Debbie Frazier,Stijn Heymans, Sue Hinojoza, EricKow, David Margolies, Ethan Stone,William Webb, Michael Wessel, andNeil Yorke-Smith.

Notes1. This data is based on our analysis of a spe-cific biology textbook and measuring thenumber of new concepts and relationshipsthat needed to be encoded from it.

2. Different groups of students and coursescan have varying requirements. For exam-ple, students studying for an advancedplacement exam need to be responsive tothe requirements of that exam. We havemade no special effort to customize InquireBiology to such specific needs.

3. See www.projecthalo.com.

4. See www.questiongeneration.org/.

5. See www.kno.com.

6. See www.inkling.com.

7. See www.wolframalpha.com/about.html.

ReferencesAgrawal, R.; Gollapudi, S.; Kannan, A.; andKenthapadi, K. 2011. Data Mining forImproving Textbooks. SIGKDD Explorations13(2): 7–10.

Ausubel, D. P. 1965. A Cognitive StructureView of Word and Concept Meaning. InReadings in the Psychology of Cognition, ed. R.C. Anderson and D. P. Ausubel, 103–115.New York: Holt, Rinehart and Winston, Inc.

Ausubel, D. P., and Youssef, M. 1963. Role ofDiscriminability in Meaningful ParalleledLearning. Journal of Educational Psychology54(6): 331–331.

Baader, F.; Calvanese, D.; Mcguinness, D. L.;Nardi, D.; and Patel-Schneider, P. F., eds.2007. The Description Logic Handbook: Theo-ry, Implementation, and Applications, 2nd ed.Cambridge, UK: Cambridge UniversityPress.

Banik, E.; Kow, E.; Chaudhri, V.; Dinesh, N.;and Oza, U. 2012. Natural Language Gener-ation for a Smart Biology Textbook. Paperpresented at the International Conferenceon Natural Language Generation, Utica Il,30 May – 1 June.

Barker, K.; Porter, B.; and Clark, P. 2001. ALibrary of Generic Concepts for ComposingKnowledge Bases. In Proceedings of the 1stInternational Conference on Knowledge Cap-ture, 14–21. New York: Association for Com-puting Machinery.

Bliss, J.; Askew, M.; and Macrae, S. 1996.Effective Teaching and Learning: Scaffold-ing Revisited. Oxford Review of Education22(1): 37–61.

Bransford, J. D.; Brown, A. L.; and Cock-ing, R. R. 1999. How People Learn: Brain,

Mind, Experience, and School. Washington,DC: National Research Council.

Brint, S.; Douglass, J. A.; Thomson, G.; andChatman, S. 2010. Engaged Learning in thePublic University: Trends in the Undergrad-uate Experience. Report on the Results ofthe 2008 University of California Under-graduate Experience Survey. Berkeley, CA:Center for the Studies in Higher Education.

Brooke, J. 1996. Sus: A Quick and DirtyUsability Scale. In Usability Evaluation inIndustry, ed. P. W. Jordan, B. Thomas, B. A.Weerdmeester, and A. L. McClelland. Lon-don: Taylor and Francis.

Chaudhri, V. K.; Heymans, S.; Tran, S.; andWessel, M. 2013. Query Answering inObject-Oriented Knowledge Bases. Paperpresented at the 6th Workshop on AnswerSet Programming and Other ComputingParadigms, Istanbul, Turkey, 24–29 August.

Chaudhri, V. K., and Tran S. C. 2012. Speci-fying and Reasoning with UnderspecifiedKnowledge Bases Using Answer Set Pro-gramming. In Proceedings of the 13th Inter-national Conference on Knowledge Representa-tion and Reasoning. Palo Alto, CA: AAAIPress.

Chaudhri, V.; John, B. E.; Mishra, S.;Pacheco, J.; Porter, B.; and Spaulding, A.2007. Enabling Experts to Build KnowledgeBases from Science Textbooks. In Proceedingsof the 4th International Conference on Knowl-edge Capture. New York: Association forComputing Machinery.

Chu-Carroll, J., and Fan, J. 2011. LeveragingWikipedia Characteristics for Search andCandidate Generation. In Proceedings of the25th AAAI Conference on Artificial Intelli-gence. Palo Alto, CA: AAAI Press.

Clark, P. E., and Porter, B. 2012. KM — TheKnowledge Machine 2.0 User’s Guide. Tech-nical Report, Dept. of Computer Science,University of Texas at Austin.

Clark, P.; Chaw, S.-Y.; Barker, K.; Chaudhri,V.; Harrison, P.; Fan, J.; John, B.; Porter, B.;Spaulding, A.; Thompson, J.; and Yeh, P. Z.2007. Capturing and Answering QuestionsPosed to a Knowledge-Based System. In Pro-ceedings of the 4th International Conference onKnowledge Capture. New York: Associationfor Computing Machinery.

Disessa, A. A. 1993. Toward an Epistemolo-gy of Physics. Cognition and Instruction 10(2–3): 105–225.

Driver, R., and Easley, J. 1978. Pupils andParadigms: A Review of Literature Related toConcept Development in Adolescent Sci-ence Students. Studies in Science Education5(10): 61–84.

Ferrucci, D. A. 2012. This Is Watson. IBMJournal of Research and Development 56(3–4).

Fitting, M. 1996. First-Order Logic and Auto-mated Theorem Proving. Berlin: Springer.

Gelfond, M., and Lifschitz, V. 1990. LogicPrograms with Classical Negation. In LogicProgramming: Proceedings Seventh Interna-tional Conference, ed. D. Warren and P.Szeredi, 579–597. Cambridge, MA: The MITPress.

Graesser, A. C.; McNamara, D. S.; Louwerse,M. M.; and Cai, Z. 2004. Coh-Metrix:Analysis of Text on Cohesion and Lan-guage. Behavior Research Methods 36(2):193–202.

Grosof, B. N. 2009. Silk: Higher Level Ruleswith Defaults and Semantic Scalability. InWeb Reasoning and Rule Systems, Proceedingsof the Third International Conference, Volume5837, Lecture Notes in Computer Science,24–25. Berlin: Springer.

Gunning, D.; Chaudhri, V. K.; Clark, P.;Barker, K.; Chow, S.-Y.; Greaves, M.; Grosof,B.; Leung, A; McDonald, D.; Mishra, S.;Pacheco, J.; Porter, B.; Spaulding, A.; Tecu-ci, D.; and Tien, J. 2010. Project HaloUpdate — Progress Toward Digital Aristotle.AI Magazine 31(3).

Gunning, D.; Chaudhri, V.; and Welty, C.,eds. 2010. Special Issue on QuestionAnswering Systems. AI Magazine 31(3).

Kintsch, W. 1988. The Use of Knowledge inDiscourse Processing: A Construction-Inte-gration Model. Psychological Review 95(2):163–182.

Mason, K.; Losos, J.; and Singer, S. 2011.Biology, 9th ed. New York: McGraw Hill.

Mayer, R. E. 2003. The Promise of Multi-media Learning: Using the Same Instruc-tional Design Methods Across DifferentMedia. Learning and Instruction 13(2): 125–139.

Mayer, R. E. 1979. Can Advance OrganizersInfluence Meaningful Learning? Review ofEducational Research 49(2): 371–383.

Mayer, R. E., and Massa, L. J. 2003. ThreeFacets of Visual and Verbal Learners: Cog-nitive Ability, Cognitive Style, and Learn-ing Preference. Journal of Educational Psy-chology 95(4): 833–833.

Means, B.; Toyama, Y.; Murphy, R.; Bakia,M.; and Jones, K. 2009. Evaluation of Evi-dence Based Practices in Online Learning:A Meta-Analysis and Review of OnlineLearning Studies. Menlo Park CA: SRI Inter-national.

Miller, K., and Levine, J. 2010. Biology, 2nded. Boston: Pearson.

National Research Council. 1999. How Peo-ple Learn: Brain, Mind, Experience, andSchool. Washington, DC: National AcademyPress.

Novak, J. D. 1998. Learning, Creating, andUsing Knowledge: Concept Maps as Facili-tative Tools in Schools and Corporations.Journal of E-Learning and Knowledge Society6(3): 21–30.

Articles

72 AI MAGAZINE

Ozuru, Y.; Dempsey, K.; and McNamara, D.S. 2009. Prior Knowledge, Reading Skill, andText Cohesion in the Comprehension ofScience Texts. Learning and Instruction 19(3):228–242.

Palinscar, A. S., and Brown, A. L. 1984.Reciprocal Teaching of Comprehension-Fostering and Comprehension-MonitoringActivities. Cognition and Instruction 1(2):117–175.

Pera, M., and Ng, Y. 2012. Personalized Rec-ommendations on Books for K–12 Students.In Proceedings of the Fifth ACM Workshop onResearch Advances in Large Digital Book Repos-itories and Complementary Media. New York:Association for Computing Machinery.

Pressley, M., and Afflerbach, P. 1995. VerbalProtocols of Reading: The Nature of Construc-tively Responsive Reading. Mahwah, NJ:Lawrence Erlbaum.

Reece, J.; Urry, L.; Cain, M.; Wasserman, S.;Minorsky, P.; and Jackson, R. 2011. Camp-bell Biology, 9th ed. Boston: Benjamin Cum-mings.

Scardamalia, M., and Bereiter, C. 2006.Knowledge Building: Theory, Pedagogy, andTechnology. The Cambridge Handbook of theLearning Sciences, 97–115. Cambridge, UK:Cambridge University Press.

Scardamalia, M.; Bereiter, C.; Hewitt, J.; andWebb, J. 1996. Constructive Learning fromTexts in Biology. In Relations and BiologyLearning: The Acquisition and Use of Knowl-edge Structures in Biology, ed. M. K. K. M. Fis-cher, 44–64. Berlin: Springer.

Sweller, J. 1988. Cognitive Load DuringProblem Solving: Effects on Learning. Cog-nitive Science 12(2): 257–285.

Udeani, U., and Okafor, P. N. 2012. TheEffect of Concept Mapping InstructionalStrategy on the Biology Achievement ofSenior Secondary School Slow Learners.Journal of Emerging Trends in EducationalResearch and Policy Studies 3(2): 137–142.

Vinay K. Chaudhri received his Ph.D. incomputer science from University of Toron-to, Canada, and is currently a programdirector in the Artificial Intelligence Centerat SRI Interntaional. His research focuses onthe science, engineering, and application oflarge knowledge base systems.

Britte Haugan Cheng is a senior educationresearcher at SRI International’s Center forTechnology in Learning. She received herPh.D. from University of California, Berke-ley’s Cognition and Development Programin the Graduate School of Education. Herresearch focuses on the design and imple-mentation of STEM learning technologies,instruction, and assessments.

Adam Overholtzer is an interaction design-er and user interface developer at SRI Inter-

national, where he designed and built theInquire iPad app. Overholtzer likes to workin pixels, in code, and in his free time, Lego.

Jeremy Roschelle specializes in the designand development of integrated interven-tions to enhance learning of complex andconceptually difficult mathematics and sci-ence; learning sciences–based research inmathematics education, on collaborativelearning, and with interactive technology;and the management of large-scale multi-year, multi-institutional research and eval-uation projects.

Aaron Spaulding is a senior computer sci-entist and interaction designer at SRI’s Arti-ficial Intelligence Center. His work centerson developing usable interfaces for AI sys-tems that meet real user needs. He holds aMaster’s degree in human computer inter-action from Carnegie Mellon University.

Peter Clark is a senior research scientist atVulcan Inc. working in the areas of naturallanguage understanding, question answer-ing, machine reasoning, and the interplaybetween these three areas. He received hisPh.D. in computer science from StrathclydeUniversity, UK, in 1991.

Mark Greaves is technical director for ana-lytics at the Pacific Northwest National Lab-oratory (PNNL). Prior to joining PNNL, hewas director of knowledge systems for Vul-can Inc., where he oversaw the AURA andInquire Biology work reported on here. Priorto joining Vulcan, he served as a programmanager at the U.S. Defense AdvancedResearch Projects Agency, where he lednumerous projects in AI and semantic webtechnology.

David Gunning is the program director forenterprise intelligence at the Palo AltoResearch Center (PARC). He directs PARC’sefforts in artificial intelligence and predic-tive analytics focused on the enterprise.These include projects in anomaly andfraud detection, contextual intelligence,recommendation systems, and tools forsmart organizations. Prior to PARC, Gun-ning was a senior research program manag-er at Vulcan Inc., a program manager atDARPA (twice), SVP of SET Corp., vice pres-ident of Cycorp, and a senior scientist in theAir Force Research Labs. At DARPA, hemanaged the Personalized Assistant thatLearns (PAL) project that produced Siri,which was acquired by Apple, and the Com-mand Post of the Future (CPoF) project thatwas adopted by the U.S. Army for use inIraq and Afghanistan. Gunning holds anM.S. in computer science from StanfordUniversity, an M.S. in cognitive psychologyfrom the University of Dayton, and a B.S. inpsychology from Otterbein College.