Multidisciplinarity vs. Multivocalitythe case of “Learning Analytics”

Nicolas Balacheff1, Kristine Lund2, CNRS1,2, University of Grenoble1, University of Lyon2

Learning Analytics & Knowlege ConferenceApril 8-12, 2013

Leuven, Belgium

Laboratoire d’Informatique de Grenoble

2

Learning Analytics / Educational Data Mining

- Conceptually grounded, coined to respond to research needs

- Socially grounded, adopted as common conceptual flagships

Multidisciplinarity, multivocality and interdisciplinarity

To what extent can the different disciplines involved in the TEL community be integrated on methodological and theoretical levels?

Origin of concepts and methods in the TEL research area

Problématique,theoretical framework and methodology

In which way does each expression solve problems identified in the TEL research area and how specific are they? What relations do they have with other concepts in the domain?

The case of LA and EDM

3

The TEL dictionary perspective:

defining in order to stop reinventing the wheel

The evolution of TEL research is rapid and motivations are diverse

The language is often ill-definedDifferences in terminology: variations among communities or conceptual differencesDifficult to ensure that the wheel is not being reinvented

The case of LA and EDM- LA: introduced in 2009 -- first conference 2011 -- no identified endogenous precursors

but strong heterogeneous (analytics)- EDM: introduced in 2000 -- first conference 2008 – precursor workshops jointly held with

ITS, AIED, ICALT, etc. – evidence of historical roots in learner modeling

Different histories, but does that imply semantic differences?

http://www.tel-thesaurus.net/

4

(partial view of the links).Colors representthe clusters centered on the most important keywords

Data (orange)link Analytics (blue) and Educational Data (red)

Source: Stellar Grand Challenge problems

http://www.tel-thesaurus.net/maps/contexteGCP/

Keywords relating tolearning analytics

5

(partial view of the links).Colors representthe clusters centered on the most important keywords

Data (orange)link Analytics (blue) and Educational Data (red)

Source: Stellar Grand Challenge problems

http://www.tel-thesaurus.net/maps/contexteGCP/

Keywords relating tolearning analytics

Closer look at the DataTEL and the Productive Multivocality workshops at the Alpine Rendez Vous 2011 - although data is at the heart of both, there is almost no shared vocabulary, apart from cognates of “learning”

- in addition, there is a great difference in terms of scope in the two workshop’s objectives

6

DataTEL“The research on TEL recommender systems can contribute to decreasing the drop-out rate”

“customize existing recommendation algorithms for learning, employ recommender systems in real-life scenarios and develop suitable evaluation criteria for different kinds of recommender systems”.

Productive Multivocality“supportive structure for a dialogical interpretation of the data in order to make the community and stakeholders aware what results converge among the different data sets and different interpretations and in order to identify open questions”.

The TEL dictionary perspective:

defining in order to stop reinventing the wheel

Data is at the core of both communities, but in different ways- One focuses on improving algorithms to treat data (recommender systems)- The other focuses on interpretation of shared data

Sharing data is a potentially productive move for TEL research, but not an easy one

What « data » means might be the next question

a challenge illustrated in the second part of this communication

7

Initial conditions for Productive Multivocality using the pivotal moment as

a boundary object

X 5 Editors: Suthers, D., Lund, K., Rosé, C., Law, N. & Teplovs, C.

8

Multidisciplinarity, interdisciplinarity and multivocality

• Neither theoretical perspectives nor actual results from different participating disciplines are integrated during multidisciplinarity

subject approached from different angles, using different disciplinary perspectives (van den Besselaar & Heimerik, 2001)

Each research group stays within their own boundaries (Choi & Pak, 2001)

• Interdisciplinary research integrates contributing disciplines by creating its own theoretical, conceptual and methodological identity

analyzes, synthesizes and harmonizes links between disciplines into a coordinated and coherent whole (van den Besselaar & Heimerik, 2001)

• Multivocal research performing multiple analyses from different epistemological and methodological

frameworks on a shared corpus (e.g. group interactions in pedagogical contexts) Productive : analytical concepts were refined, epistemological positions were made

explicit, and the conditions under which learning occurs were characterized, but with different perspectives, thus allowing discussion about learning

9

Our argument

• The LA community is much like the CSCL community Multidisciplinary with a potential for interdisciplinarity

• (Our version of ) multivocality is closer to interdisciplinarity than to multidisciplinarity

We will use an example from the Productive Multivocality Initiative to illustrate this

Multivocality and interdisciplinarity are approaches that move research fields forward

– The communities researching “Learning Analytics” are nicely positioned to benefit from such approaches, much in the same way that CSCL has been

10

How multivocality can tend towards interdisciplinarity

• Step 1 3 researchers each designate the moment they call pivotal

– Different visions of learning are made explicit– “Moments” are of differing length (cf. unit of analysis/interaction)

1 Trausan-Matu

2 Shirouzu

3 Chiu

1 Trausan-Matu

1 Trausan-Matu

“Fold, then cut out the 3/4 of 2/3 of the origami paper”

11

Multivocality without convergence

• The comparison of two researcher’s pivotal moments lead to progress in each other’s problématiques, but not to integrating on either a theoretical or methodological level

Taking another researcher’s pivotal moments and interpreting them in one’s own framework (e.g. Chiu : breakpoints in frequency of new ideas corresponded to when and how the pedagogical designer’s intentions were actualized by students’ behavior - Shirouzu)

Neither methodological nor theoretical convergence is achieved, but a discussion has begun

12

Multivocality with convergence

• The comparison of two other researcher’s pivotal moments lead to progress in one researcher’s problématique, but also to integrating one of the researchers’ approaches on a methodological level

Trausan-Matu extended the definition of an analytical concept (e.g. Bakhtinian “voices” include gestures)

extended the domain of application (e.g. from just chat to face-to-face interactions)

Deeper theoretical integration is more difficult– We do not always aspire to that because tension can be productive

13

“Data” as a boundary object for learning analytics and educational data mining

What is shared by a teacher having to manage a lesson on ratio and proportion, and by the dean of the university having to ensure the success of the freshman class?

How far is the meaning of “learning” the same in both cases?

We join Siemens and Baker in a call for cooperation with the suggestion of an analysis of…

- the nature of data

- the problématiques driving underlying commonalities and differences

perhaps for the sake of a new theoretical, conceptual and methodological identity for both Learning Analytics and Educational Data Mining

14

LA“Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long and Siemens, EDUCAUSE 2011).

EDM“Educational Data Mining is a term used for processes designed for the analysis of data from educational settings to better understand students and the settings which they learn in.” (Desmarais and Baker, TEL Dictionary 2012)

The TEL dictionary perspective:

defining in order to stop reinventing the wheel

- Are the Learning Analytics tools imported from analytics sufficient for relevantly analyzing learning data?

- Should all data attached to the activities of a student be considered as learning data?

- Isn’t Learning Analytics reducing successful learning to the academic success of students in their institutions, limiting de facto the problématique of TEL research?

- Compared to the classical problématique of “learner modeling”, what are the specific contributions of “Learning Analytics”?

15

AfterwordFrom…

- 42 papers from LAK12

- 24 papers from LAK11 76 docs / articles

- 10 papers from JETS12

article'ssubject

article'sbody

"learning analytics"

10 564

"educational data mining"

3 40

LA "learninganalytic“

learning (1000.0) learn (909.0) analytic

(498.0) learner (372.0) activity (276.0) research (257.0) context (230.0) social (222.0) design (218.0) provide (214.0) community

(205.0) knowledge (203.0) practice (196.0)

tool (192.0) individual (180.0) datum (175.0)

model (169.0) development (160.0) environment (159.0) EDM "educationaldatummining"

cluster (140.0) use (125.0) datum (94.0) teacher (91.0) student (85.0) user (81.0) clustering (76.0) project (72.0) mining(66.0) system (64.0)

theme (62.0) pattern (57.0) lecture (57.0) study (55.0) video (51.0) online (49.0) feature (43.0)

model (43.0) class (41.0) group (40.0) question

(39.0) final (39.0) classroom (38.0) tool (37.0) level (35.0) teaching (33.0)

instructional (32.0)

1. Latent Dirichlet Allocation on the preprocessed corpus (stopwords elimination, discarding parentheses and phrases of less than 2 words & lemmatizing) -- to enforce the search for the specific concepts in the topic model – to consider them compound words.

after this step emerges: "learninganalytic" (without s) and "educationaldatummining" (with datum instead of data)

2. Trained incremental LDA topic models starting from only the LAK corpus (1.3MB) with 20 topics (a topic contains all words, with their corresponding weights in descending order with concepts semantically related that emerge from co-occurence relations

3. adding chunks of TASA corpus results to come

Thanks! To Mihail Dascalufor bringing his expertise within

very short delay! Work in progress!