Orchestrating Computer Supported Collaborative Learning (CSCL)

Prof. Yannis DimitriadisGSIC/EMIC research group

University of ValladolidJuly 19, 2012

Based on work by L.P. Prieto – S. Villagrá

University of Queensland, Centre for Innovation in Professional

Learning, Lunchtime research seminar

TEL/CSCL ecosystem

2

Outline

3

■ The two hats of this seminar■ Hat 1: Conceptual tools

■ A framework on orchestration■ Multi-level patterns as mediating tools

■ Hat 2: Technological tools for ■ Learning design■ Flexible deployment

The two hats of this seminar

4

■ Personal■ Engineering background + Interest in education

■ Audience ?■ Researchers in TEL, e-learning designers

■ TEL research■ sound pedagogy + efficient engineering

■ Quest for sustainable innovation■ good research for understanding, abstraction and

evidence ■ effective support to actors – teachers, learners or

even service and technology providers

The two hats of GSIC/EMIC

5

■ “Dark and bright side” in a group since 1994

■ Focus on CSCL: complex and innovative■ Support real-world actors and own action-research■ Inquiry cycle (design-enactment-evaluation) for

■ teaching/learning and technological support

Hat 1: Conceptual tools

6

■ Analyze and understand the field of orchestration in complex TEL ecosystems■ The “5+3 aspects” framework for orchestration

■ Collect and use knowledge in terms of good practices during the CSCL cycle■ Multi-level patterns

Orchestration: Moving target

7

■ Increasing presence in literature, but what is it about? Is there a new focus?

■ Efforts to define the field/concept/metaphor:■ Stellar NoE deliverable, CSCL 2011 and ICLS

2012 workshops, collective paper■ New proposal for “5+3” orchestration framework

and validation through expert panel

Focus of orchestration on …

8

■ Acknowledge complex integrated ecosystems for innovative TEL/CSCL environments

■ Support teachers as orchestrators (and think of how to minimize-distribute the orchestration load on teachers, learners and systems)

■ Build on pragmatism and minimalism as keys for sustainability

■ Explicit emphasis on how to flexibly manage (learning time) enactment

■ Design for adequate orchestrating and orchestrable technologies (and conceptual tools)

The “5+3” frameworkA diagramatic view

9http://prezi.com/aa2vighak7hh/orchestration-in-tel-cscl-as-easy-as-53/

The “5+3” frameworkA process view

10

Evaluating the frameworkThe setup

11

■ Research questions on■ Completeness of common understanding■ Usefulness for researchers

■ Two panels (March – May 2012)■ Pilot phase: 22 Spanish researchers of low-

medium expertise (2.58) and more technological profile (16T-7E-1M)

■ International panel: 24 researchers of medium-high expertise (4,46), high research experience in TEL (15,8 years), more balanced profile (6T-11E-7M)

Completeness of framework

12

■ Quantitative and qualitative data■ Non-normative and inclusive■ Need to clarify role of actors, technology, theory■ More than 4,5 (Lickert scale of 1-6) for all items■ Logical: 4,78 –Comprehensive: 4,52 –Relevant: 5,13

Usefulness of framework

13

■ To be referenced in papers, checklist and didactic tool for young researchers

■ But need for normative advice and real-world examples

■ Will be used in future (4.43)

■ Provided new insights (4.17)

Multi-level patterns

14

■ But the “5+3” Orchestration framework is a conceptual tool mainly for researchers

■ Tools (conceptual or technological) for orchestration should also support “teachers”■ Exploiting articulated (design) knowledge

■ (Pedagogical) patterns as mediating tools regarding various orchestration aspects■ Design and Management of learning flow,

Assessment, Adaptation …■ Related fields of

■ Learning Design, Educational Modeling, Scripting

Collaborative Learning Flow and assessment patterns

15

■ Learning Flow

■ Assessment

Aligning both sets of patterns

16

Usefulness of patterns

17

■ Are these patterns useful for practitioners?■ Evidence for learning flow patterns for OER

repurposing to collaborative activities■ Appropriation in University (Higher Education –

HE) teachers (Professional Development - PD) workshops

■ But not always■ If not included in appropriate technological tools■ Too complex (abstract) for (Primary Education –

PE) practitioners■ Do not take into account enactment issues

The need for atomic patterns

18

■ Two independent studies by GSIC and SRI using GroupScribbles in Primary and Secondary Education schools (2008-2011)

■ Studying enactment and disciplined improvisation■ Showed that other small-scale, informal, contextual,

actionable patterns emerged

A learning design in actionin a Primary Education classroom

19

Using atomic patterns in a HE PD workshop

20

Atomic patterns as mediating conceptual tools

21

■ And we need to “fill the gap”, “flesh the bones” with appropriate mediating artifacts (conceptual tools)

Evaluation of atomic patternsin PD workshops

22

■ Four (2 in PE, 2 in HE) PD workshops (2010-2011)■ WS4 data quantitative data (scale 1 to 7):

■ Enactment patterns useful (5,31)■ Feasibility (5,22) and nearness (6,06)■ 19 out of 22 teachers enriched design

■ Useful but not revolutionary■ Especially important for non-experienced or pre-

service teachers■ Need to condense and classify long list of AP■ What about inclusion in technological tools?

Approach for multi-level patterns

23

■ And the way to elicit and use multi-level patterns

Hat 1: Wrap-up

24

■ Knowledge regarding orchestration for researchers:■ “3+5 Aspects” framework■ Approach for multi-level patterns■ Atomic pattern catalogue and LD representation

■ Support teachers too with conceptual tools: ■ Classroom norms for participation, discourse

moves for discussion, decision rules for contingent teaching (SRI)

■ Multi-level patterns ■ Use in PD workshops Role playing and learning

design enrichment using cards (GSIC)

Hat 2: Technological tools

25

■ But TEL is not only about ■ Conceptual tools (patterns, moves, …)

■ It is also about ■ enacting learning in TEL complex ecosystems■ especially in HE distributed learning

environments where VLE/PLE are combined with third-party tools

■ And also support teachers (and learners) with technological tools when■ orchestrating TEL/CSCL ecosystems (less

orchestration load for teachers – more for tools)

Learning Design tools

26

■ Articulate/exploit the knowledge of patterns and support the learning design process

■ Create a model or representation of a learning design (unit of learning, script, scenario, etc.)■ That allows understanding and reflecting on the

pedagogy, making the learning design explicit, sharing it with others

■ Through the Web Collage tool■ Or other Learning tools by the Stellar Learning

Design Grid Theme Team http://www.ld-grid.org/home

Web Collage: Learning & Assessment flows

27

Teacher

Select and work individually in a

subproblem

Support individual work

Students

Work in expert groups of students

with the same subproblem

Form Jigsaw groups and explain each other the different

subproblems

Support expert group work

Support jigsaw group work

Write activity report

Review report

Work in expert groups of students

with the same subproblem

input for…

provide feedback…

http://pandora.tel.uva.es/wic2

A learning design(HE teacher in a PD workshop)

28

Web Collage:Evolution and evaluation

29

■ Evolution of the Collage tool since 2005■ Towards a more stable and user-friendly Web-

based tool that includes learning and assessment patterns, as well as support for instantiation

■ Extensive use and evaluation using mixed methods■ Multi-case studies in authentic settings and PD

workshops with HE teachers■ Wide acceptance of the pattern-based design

process

The deployment gap

30

■ But teachers wanted to complete the cycle■ deploy their design in their TEL environment,

especially their institutional VLE (e.g. Moodle)

Alternatives for deployment

31

■ Have the learning design “in mind” and then go through the VLE/PLE interface and struggle to “translate” the design to a course/lesson plan

■ Use a single system (LAMS) to do both design and deployment in the same environment

■ Use an LD Tool (LDSE) to create the interpretable script and then deploy it to one environment

■ But there are many LD tools and VLEs: ■ Provide intermediary to connect them: Glue!-PS

Glue!-PSAn architectural view

32

Glue!-PSAn architectural view - II

33

Glue!-PS:A functional view

34

Glue!-PS Validation

35

■ Proof of concept validations using the Planet Game (ICALT 2006) scenario ■ Two LD languages: LDL + IMS-LD■ Two Learning Environments: Moodle +

MediaWiki■ Design and deployment

■ by GSIC researchers of 36 learning designs created using pen and paper at two HE PD workshops (Sep-Nov 2011)

■ by 24 teachers in a follow-up PD workshop (Feb 2012)■ Immediate use in real life: 6,19 (scale 1 to 8)!

Glue!-PS and atomic patterns

36

■ Atomic patterns are especially important for flexible orchestration

■ University teachers suggested embedding atomic patterns in technological tools

■ Initial evidence of use in Glue!-PS■ Implementation of a subset of atomic patterns

without architectural changes as e.g. ■ AP: Spontaneous use of additional IT■ Actions: Add and synchronize in real-time

resources in an activity ■ Feasibility evidence in 2 cases (Mar-Apr 2012)

Hat 2: Wrap-up

37

■ Technological tools to support orchestration■ Patterns (that reflect good educational practices)

guide a flexible design and deployment process■ Glue!-PS allows to bridge the deployment gap in

distributed learning environments■ time-efficient and less error-prone■ allows sustainable migration to new VLE and

connection to various LD tools■ More naturalistic evaluation is necessary■ Important role of PD workshops for innovation and

feedback to research

Hat 2 (cont.): More tools to …

38

■ Regulate through monitoring and the detection of critical aspects of the learning designs (GLUE!-CAS)

■ Find and select appropriate tools and include them in learning designs (SEEK)

■ Integrate third-party tools in learning designs directly through the VLE interface (GLUE!)

■ Design evaluation of CSCL case studies and generate multimedia reports (CSCL-EREM)

■ ….

Hats 1 and 2: wrap-up

39

■ Support bits and pieces but also the complete cycle of orchestration ■ flexibly and sustainably

■ Design for orchestration to show added value of TEL ■ innovative pedagogies using old and new

technologies■ Integrate conceptual and technological tools for

■ researchers, teachers and students

And many open questions

40

■ Do researchers need an orchestration theory and framework and should it be normative?

■ Is the proposed approach for pattern elicitation sustainable? (crowd-sourcing ….)

■ How can we support co-orchestration with students and integrate efficiently PLE?

■ How various (Web, 3D worlds, augmented physical) spaces can be orchestrated?

■ Is there a way to solve the ethical and security problems that are involved?

■ …

Some links

41

■ Publications■ http://www.gsic.uva.es/public.php?lang=en&list_public=all

■ GSIC/EMIC channel at YouTube:■ http://www.youtube.com/user/gsicemic?

feature=watch ■ WS on orchestration at ICLS2012 (July 3)

■ https://sites.google.com/site/iclsorchestration2012/