agile technologies for personalizing instruction faisal ahmad, sebastian de la chica, qianyi gu,...
TRANSCRIPT
Agile Technologies for Personalizing Instruction
Faisal Ahmad, Sebastian de la Chica, Qianyi Gu, Shaw Ketels, Ifi OkoyeTammy Sumner, Jim Martin, Alice Healy, Kirsten Butcher, Michael Wright
Digital Learning SciencesUniversity of Colorado at Boulder
University Corporation for Atmospheric Research
This work is supported in part by an ICS Generalization Grant, and NSF awards #0537194 and #0734875
2
Central Challenge
Enable personalized learning, while still supporting recognized learning goals
Do it at scale
How People Learn (NRC)
Extreme Diversity (KnowledgeWorks)
Disrupting Class (Christiansen)
N=1, R=G (Prahalad)
www.DLESE.org
Strandmaps.NSDL.org
Curriculum Customization
CLICK Personalization Service
4
CLICK Personalization Service
Automatically identify potential learner misconceptions by analyzing student work
Customize the selection and presentation of learning resources based on identified misconceptions
High school plate tectonics
5
Guiding Principles Personal and intentional
Build on learner understanding Learner control Learning goals organize and guide
Agile technologies Domain independent: knowledge maps for
human cognition and machine reasoning Automatic: NLP and ML Embeddable: web services, not applications Open: leverage existing web content
6
7DEMO
8
Major CLICK Components What should students know?
Domain knowledge map
What do they already understand? Compare student and domain maps
What learning activities would be useful? Select resources to address misconceptions and gaps
How to embed in learning environments? Provide web service to application and portal developers
9
Detecting Potential Knowledge Gaps
Alignment andComparison
Student Essay
(1) Student Knowledge Model (2) Domain Competency Model
Digital Library Resources
(3) Knowledge Trace
Human-Centered Methodology Expert studies to inform algorithms
(Ahmad et al 2007)
Domain knowledge map creation Student essay to student knowledge map Knowledge gap diagnosis Personal instruction plan generation
Expert scoring of intermediate results
Mixed-method learning study10
Algorithms Concept extraction (de la Chica 2008)
MEAD: multi-document summarization toolkit (Radev et al 2004)
Custom sentence scoring features: standards, gazetteer, hypertext, content word density
Eliminate redundancy, rank and choose top 5% Student essays – lexical chains (de la Chica 2008)
Knowledge gaps – NLP and graph structure comparisons (Ahmad 2008)
Personalized information retrieval – concept matrix (Gu 2008)
11
12
CLICK Personalization Web Service
Misconception diagnoses and knowledge map generation exposed via request types (Ahmad 2008)
Submit or remove a concept map Construct student map from essay Construct domain map from URLs Get student misconceptions Get important concepts Get related concepts
32 undergraduates 16 – CLICK to revise essays on Earthquakes
and Plate Tectonics 16 – control Digital Library environment
Data collected original essays, revised essays, detailed screen
capture “movies”, reflective questions, factual knowledge tests
13
Mixed-Method Learning Study
14
Essay Content Revisions
Shallow. CLICK< Control: F (1, 27) = 3.602, p = .068 (TREND)
Deep. CLICK > Control: F (1, 27) = 5.222, p = .030 (SIG EFFECT)
15
Shallow revisions Copying out of resource,
Paraphrasing, Integrated copying, Integrated paraphrasing, Concept deletion
Deep revisions Integrated sentence
paraphrasing to create new sentence, Integrated resource paraphrasing to create new sentence, Inferencing, Generation
Codes based on Wiley and Voss 1999, Constructing arguments from multiple sources
Types of Content Revisions
16
CLICK > Control: F (1, 27) = 6.490. P = 0.17 (SIG EFFECT)
Omissions Gaps in student content
knowledge such as missing details and missing concepts
Incorrect Statements Coding still underway
Process Data
17Exploration. CLICK>Control: F (1, 27) = 6.076, p = .02 (SIG EFFECT)Essay. CLICK>Control: F (1, 27) = 6.815, p = .015 (SIG EFFECT)Switches. CLICK>Control: F (1, 27) = 6.447, p = .017 (SIG EFFECT)
Exploration Episodes Exploring learning
resources and personalized feedback
Essay Episodes Revising or working with
essay
Switches Moving between essay and
exploration Integration of content
resources and developing essay
Recognizing need for outside knowledge source
Conclusions Learning - Initial CLICK results promising
Encourages deep content revisions Promotes integration between information
seeking and knowledge transformation Students more likely to recognize that they
need new knowledge, a critical element of self-directed learning
Algorithm Generalization: Promising results for “near” domain
Misconception prioritization and link generation need further work
18
Further Reading Ahmad, F., S. de la Chica, K. Butcher, T. Sumner, and J. Martin. (2007). Towards
automatic conceptual personalization tools. In Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2007): Vancouver, Canada (June 18-23), pp. 452-461.
Butcher, K. and S. de la Chica. (in press). Supporting student learning with adaptive technology: Personalized conceptual assessment and remediation. In M. Banich and D. Caccamise (Eds.), Generalization of Knowledge: Multidisciplinary Perspectives. London, England: Taylor and Francis.
de la Chica, S., F. Ahmad, J. Martin, and T Sumner. (2008). Pedagogically useful extractive summaries for science education. 22nd Meeting of the International Committee for Computational Linguistics (COLING 2008).
de la Chica, S., F. Ahmad, T. Sumner, J. Martin, and K. Butcher. (2008). Computational foundations for personalizing instruction with digital libraries. International Journal of Digital Libraries. To appear in the Special Issue on Digital Libraries and Education.
Gu, Q., de la Chica, S., Ahmad, F., Khan, H., Sumner, T., Martin, J., Butcher, K. (2008). Personalizing the Selection of Digital Library Resources to Support Intentional Learning. Research and Advanced Technology for Digital Libraries, 12th European Conference, ECDL 2008, Aarhus, Denmark, September 14-19. Lecture Notes in Computer Science, pp. 244-255.
19
Examples of “Good” Concepts
20
Plate Tectonics Weather and Climate
Good standalone concept
A gradual build-up of mechanical stress in the crust, primarily the result of tectonic forces, provides the source of energy for earthquakes; sudden motion along a fault releases it in the form of seismic waves.
The shape and position of waves in the polar jet stream determine the location and the intensity of the mid-latitude cyclones.
Good concept in context
Many places near this plate boundary are at high risk for earthquakes, including the San Francisco area, the Pacific Northwest, and Alaska, yet fully half the nation's earthquake hazard is in Southern California.
This energy is used to heat the Earth's surface and lower atmosphere, melt and evaporate water, and run photosynthesis in plants.
21
22
Detecting Potential Knowledge Gaps
Alignment andComparison
Student Essay
(1) Student Knowledge Model (2) Domain Competency Model
Digital Library Resources
(3) Knowledge Trace