tutoring and educational applications 2pm: question generation based on numerical entities in basque...
TRANSCRIPT
1
Tutoring and Educational Applications
• 2pm: Question Generation Based on Numerical Entities in Basque (Itziar Aldabe, Montse Maritxalar, Ander Soraluze)
• 2:25pm: A Graph Theory Approach for Generating Multiple Choice Exams (Sarah Luger)
• 2:50pm: Generating Mathematical Word Problems (Sandra Williams)
• 3:05pm: Moderated discussion (Jack Mostow)
AAAI Symposium on Question Generation, Nov. 4-6, Arlington, VA
11/4/2011 2
Questions about Questions• Target: what does it take to answer the question?• Use: why ask the question?• Question type: cloze? wh-/how/so/…? find/compare/…?• Answer type: multiple choice? fill-in? open-ended?• Generation: how to construct question, answer, distracters?• Modality: menu? click? keyboard? speech? graphics? …• Assessment: how to score answer? how to generate feedback?• Evaluation: how well does question achieve use? how to tell?• Discussion:
– Why bother – why not just use cloze? – What kinds of difficulty are good, for what? – What’s novel? – What key idea(s) transfer?
11/4/2011 3
Question Generation Based on Numerical Entities in Basque
• Target: numerical fact (measure, date, time, number)• Use: tests• Question types: Which? When? How many?• Answer type (planned): multiple choice• Generation: find numerical entity; transform sentence• Modality: unspecified• Assessment: = correct answer?• Evaluation (human): grammatical? fluent?• Discussion: better than cloze? how?
11/4/2011 4
A Graph Theory Approach for Generating Multiple Choice Exams (Sarah Luger)
• Target: unspecified• Use: tests• Question type: unspecified• Answer type: multiple choice• Generation: none; given question, answer, and 4-5 distracters• Modality: unspecified• Assessment: correct choice?• Evaluation: difficulty = fool more good than bad students• Ideas: extract complete “virtual exams” from partial test data• Discussion: What features make distracters difficult? What kinds of
difficulty are good, for what uses? What can wrong answers reveal? Can Q-matrix or other learning identify types of students and questions? Relation to work on equating scores based on different item substs?
11/4/2011 5
Generating Mathematical Word Problems (Sandra Williams)
• Target: solve word problem• Use: practice numeracy in realistic contexts• Question type: multi-sentence problem involving two entities• Answer type: number• Generation: refactor and aggregate OWL, realize in English• Modality: unspecified• Assessment: = correct answer?• Evaluation: none yet• Ideas: Manipulate 5 difficulty factors: readability, irrelevant
numbers, extraneous information, order of numbers, conceptual difficulty of math
• Discussion: What kinds of difficulty are good, for what uses?