from micro to macro - analyzing activity in the role sandbox
DESCRIPTION
Presented at the Third Conference on Learning Analytics and Knowledge (LAK 2013).TRANSCRIPT
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke1 These presentation slides by Dominik Renzel are licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
From Micro to Macro -Analyzing Activity in the ROLE Sandbox
Dominik Renzel, Ralf KlammaRWTH Aachen University
Advanced Community Information Systems (ACIS)Aachen, Germany
{renzel,klamma}@dbis.rwth-aachen.de
Third Conference on Learning Analytics and Knowledge (LAK 2013)April 8-12, 2013Leuven, Belgium
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke2
Responsive Open
Community Information
Systems
Community Visualization
and Simulation
Community Analytics
Community Support
Web A
nalytics
Web
Eng
inee
ring
Advanced Community Information Systems (ACIS)
Requirements Engineering
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke3
Motivation Increasing adoption of distributed learning services on the Web Large-scale Web-based learning platforms (MOOC, PLE, hybrids) Monitoring: unprecedented insights into learner behaviour
– Proprietary techniques & data models for specific scenarios (bias/limitation)– New “standards“ for learner behaviour analysis on high semantic level
But: Learner behaviour evolving & inherently hard to model!
Our proposal: step back to existing Web logsStandardized by-product of regular Web service operationWorld-wide adoptionLowest possible degree of biasGeneric low-level semantics easily liftable to higher-level semantics Analysis on multiple ecosystem levels from “micro to macro“ Highest level of data interoperability (cross-service analysis)
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke4
Focus & Level of Analysis – Bronfenbrenner‘s EST applied to LA
5 nested & interrelated ecosystems around student: Microsystem
(direct relation to peer/group) Mesosystem
(entirety of microsystems + relations) Exosystem
(networks influencing individual) Macrosystem
(entirety of societal relations incl. norms, rules, etc.) Chronosystem
(temporal dimension of development)
Existing work: often focus on specific system or subsets only… Any comprehensive LA framework…
should allow analysis & focus on all levels may not neglect micro data must apply aggregation carefully should work with historical & real-time data
Avoid “washing out“ anomalies!
Support targeted & timely intervention!
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke5
Data Interoperability
Proprietary data “standards“ for monitoring learner activity in LA– High-level semantics for convenient analysis from the beginning
danger of losing anomalies, especially interesting in LA!– Long standardization process with no guarantee for adoption
risks: repeated costly changes to LA installations necessary!– Use of competing/incompatible data formats for different LA frameworks
cross-service analysis complicated/unfeasible!
Why not build on existing Web standards?– Web logs (+ page tagging) provably powerful tools in Web Analytics– Profit from genericity & widespread use also in LA– The Web works incredibly well with these standards! LA might, too…
– Why? Existing standards are expressive
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke6
Opacity & Uncertainty
LA inherently suffers from uncertainty regarding user identity Web Analytics techniques can improve, but never solve the problem! Low-level Web log analysis
Not perfect, but always available Allows sufficiently precise identification of structural/behavioral patterns Allows inferences about opaque parts
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke7
Real-time Web Log Processing Pipeline
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke8
Example: Adding widget toWidget-based PLE (ROLE Sandbox)
Example Web log entry:<IP> <DATE> POST /space/<space-id>:;tool=<widget-url>
Geo Location:Longitude/Latitude,City,Region, Country
Widget Metadata: Domains, Activities
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke9
Analysis Techniques & Benefits for Learning Stakeholders
Analysis techniques possible with simple Web log data– Which learners/learner communities interacted how with which
tools, resources in which context where and when?– Single operations micro-level analysis– Appropriate aggregation up to macro-level analysis– Relations in log data actor networks SNA – Temporal information time series analysis– Semantic enrichment from external sources a lot more!!!
Benefits for learning stakeholders– Learning community awareness (e.g. by visualization)– Recommendations (e.g. tools, persons, resources)
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke10
Conclusions & Future Work
LA frameworks for analyzing Web-based learning services to…– Prevent bias/limitation by relying on generic Web standards– Focus & level of analysis to cover micro to macro + chrono– Enable large-scale cross-service analysis by data interoperability– Combine historical & real-time information for intervention planning– Fight uncertainty, esp. regarding learner identity
Argument: Web logs stay superior to “new standards“ Not without limitations, but always available without further instrumentation Web log processing pipeline incl. data enrichment for LA simple to setup Multiple analysis techniques applicable and rich information derivable
Next steps: Carry out analysis on ROLE Sandbox dataset recorded from Mar 2012 - now >8 Mio API requests from > 3000 IPs, >500 Widgets, >1300 PLE Spaces, >3700 shared resources Apply approach in other scenarios & projects
Lehrstuhl Informatik 5(Information Systems)
Prof. Dr. M. Jarke11
Questions?
Contact: [email protected]