adolfo ruiz calleja "using social and semantic tech"

SEEK-AT-WD: A social-semantic infrastructure for educators to discover and

select ICT tools in the Web of Data

Adolfo Ruiz-Calleja

08th, October 2014

Outline GSIC @ University of Valladolid

Introduction:

Context and problem definition

Approach: a social-semantic solution in the Web of Data

Contributions:

SEEK-AT-WD infrastructure

Applications demo!!

Evaluation

Conclusions

2


Introduction:



Contributions:


Applications demo!!

Evaluation

Conclusions

3

GSIC @ University of Valladolid

4

Scripting

Tool integration frameworks

Orchestration

Augmented reality

Semantic search

Web of Data

Social Network Analysis

Monitoring

Learning flow

Cloud computing

E-Portfolios

CSCL-EREM

CSCL


Introduction:



Contributions:


Applications demo!!

Evaluation

Conclusions

5

Marie

Offers tools

Marie

Google

Context: Educational ICT tools registries

6 6

Google

Docs

MediaWiki

Hot

potatoes

Group

Scribbles

Wikimedia

foundation

Learning situation

Educational ICT tool pool

SELECT...tool...write ... collaboration...

peer review...

1.- Google Docs

2.- MediaWiki

Activities

People

Google

Docs

?

I need a tool for my students to write a

document collaboratively in a peer review activity

Query: write, collaboration, peer review

Results

Educational ICT tool search system

(semantic or keyword-based)

Registry of

educational ICT tools

(semantic or not)

Administers and publishes tool descriptions

Registry administrator

Marie

Offers tools

Marie

Google

7 7

Registry of


(semantic or not) Google

Docs

MediaWiki

Hot

potatoes

Group

Scribbles

Wikimedia

foundation

Learning situation


Activities

People

Google

Docs

Educational ICT tool search system


(Non-semantic)

(Semantic)

Sisoft

Context: “Web 1.0 approach”

The institution assumes all the sustainment effort

No information about the educational use of the tools

Publish tool

descriptions

Publish tool

descriptions

Educators community

Administers


Marie

Offers tools

Marie

Google

8

Google

Docs

MediaWiki

Hot

potatoes

Group

Scribbles

Wikimedia

foundation

Learning situation


Activities

People

Google

Docs

Social annotation

application

Social annotation

application Educational ICT tool search system


Social registry of


(semantic or not) (Semantic)

(Non-semantic)

Context: “Social Web approach”

The cold-start problem

The isolation of registries

This thesis proposal

9 9

Federate a social registry of educational ICT tools with third parties

– Import tool descriptions from other registries

– Allow third parties to freely use its data

– Avoids the isolation of educational ICT tool registries

Create an initial collection of ICT tool descriptions using third-party data

– Palliate the cold-start problem

10

Linked Data

Use of semantic technologies to publish and federate data on the Web

– In addition to enable semantic searches

Use of open standards

– RDF [W3C04] to structure the data

– SPARQL [W3C08] to query the data

Data is federated reusing and linking third parties URIs and vocabularies

Linked Data [BL06]: “Set of best practices

for publishing and connecting structured

data on the Web” [Biz09]

The Web of Data

11

Web of Data: “Extension of the Web with a global

data space based on open standards” [Hea11]

The Web of Data: descriptions of educational ICT tools

12

Domain-specific datasets

Cross-Domain datasets

Few hundreds of descriptions

Small educational communities

No information about educational use

Several thousands of descriptions

Huge general purpose communities

No educational vocabularies

Administers


Educators community

Web community Marie

Offers tools

Marie Google

13 13

Social-Semantic registry

of educational ICT tools

in the Web of Data

Semantic educational ICT

tool search system

Google

Docs

MediaWiki

Hot

potatoes

Group

Scribbles

Wikimedia

foundation

Learning situation


Activities

People

Google

Docs

Social annotation

application

ICT tool

descriptions

Web of Data

Are

federated

A social-semantic approach in the WoD (I)

Publish tool

descriptions

14

Approach Expected sustain team

Expected advantages

Social-Semantic

in the Web of Data

Registry administrator +

Community of users +

Web community

Issue 1: Is the information currently published on the Web of Data useful to inform educators about ICT tools?

Issue 2: Does a social-semantic approach allow the collection of educational ICT tool descriptions in a sustainable way?

- Use of third-party non-educative data to palliate the cold-start problem

- Use of the tool descriptions updated by the web community

- Still enable educators to publish and enrich tool descriptions

Limitations for the sustainment of educational ICT tool descriptions

Approach Sustain team Problems detected

Web 1.0 Registry administrator

- The institution assumes all the effort - Few tool descriptions - Not educational experiences using tools - The isolation of registries

Social Web Registry administrator

+ Community of users

- The cold-start problem

- The isolation of registries

A social-semantic approach in the WoD (II)


Introduction:



Contributions:


Applications demo!!

Evaluation

Conclusions

15

A social-semantic infrastructure

16

Offer a collection of educational ICT tool descriptions

Publish educational ICT tool descriptions as Linked Open Data

Provide semantic structure to the tool descriptions using an educational ontology

Allow educators to share their experiences

Automatically obtain tool descriptions from the Web

Social- semantic infrastructure

obtain ICT tool descriptions

Educational data consuming applications

Make ICT tooldescriptions available Educators

community

Enrich ICT tool descriptions

Educational annotation applications

publ

ish

info

rmat

ion

abou

t IC

T to

ols

Web community(tool providers,

Wikipedia publishers...)

Web of Datapublish

data

ICT tool descriptions

Obtain descriptions

Educationalontology

Make ICT tooldescriptions availableas Linked Open Data

SEEK-AT-WD requirements

17

Offer a collection of educational ICT tool descriptions

Publish educational ICT tool descriptions as Linked Open Data

Provide semantic structure to the tool descriptions using an educational ontology

Allow educators to share their experiences

Automatically obtain tool descriptions from the Web



Educators community

Web of Datapublish

data



Obtain descriptions

Convert descriptions

Make descriptions available


publ

ish

info

rmat

ion

abou

t IC

T to

ols



SEEK-AT-WDSEEK Ontology

Linked Open Data interface

Data publicationinterface

SEEK Ontology: Conceptualization

18

Reused ontologies:

– Ontoolcole [VG08]

– Review vocabulary [Hea07]

– DBpedia ontology [Aue07], FOAF, DC... and others

Educational context

– [Kur08] [IEE02] [Jov07] [JA06]...

– Competency questions

SEEK Ontology: Example (I)

19

“I need free tools that allow the construction of multimedia documents”

“I need a latex editor for Windows”

“I need a tool to support the synchronous communication in an on-line debate”

Some information needs that can be

expressed using the SEEK Ontology

SEEK Ontology: Example (II)

20

Tool:GoogleDocs EducationalReview:Review000

Label Google Docs Publisher Marie

Description “Is a free office suite…” Text “A very useful tool that…”

Type Asynchronous text editor Rating 4

Type Group tool MaxRating 5

Supports task Writing MinRating 1

Supports task Commenting HasEducationalContext ContextAr

Manages artifact Text document EducationalContext:ContextAr

Manages artifact Spreadsheet Course Computers Architecture

Operating system Web application AreaOfKnowledge Engineering

License Propietary software LearningGoal RequirementAnalysis

Developer Google TeachingTechnique Peer review

Has review Review000 DeliveryMode Blended

Was used in context ContextAr VLE Moodle

SEEK-AT-WD infrastructure: Software architecture

21

Publish

data

Web of Data

SEEK-KB


Crawler

SPARQL endpoint

Web interface

Data publication interface

Educators community



SEEK-AT-WD



Obtain descriptions



Obtain ICT tool descriptions

Publish informationabout ICT tools

SEEK-AT-WD infrastructure: Obtaining ICT tool descriptions

22

Crawling algorithm

– DBpedia and Factforge are accessed

– Follow your nose [Hea11]

– Automatically discover data sources

Alignment to the SEEK Ontology

– Ontology mappings [Cho06]

– Exploiting the links of the Web of Data

SEEK-AT-WD infrastructure: Implementation

23

114 formal relationships stated between the SEEK Ontology and DBpedia vocabulary

– 599 additional relationships crawling the Web

– No need to manually create additional mappings to extract data from Factforge

SEEK crawler was implemented as a Java application

SEEK-KB was implemented as an RKBexplorer registry

– Includes web and SPARQL interfaces

– Robust and reliable registry

– Available at: http://seek-rkbexplorer.com

SEEK-AT-WD dataset

24

Retrieved from ICT tool

descriptions Educational

contexts Educational

reviews

Web of Data 7935 0 0

Community of educators

49 68 176

TOTAL 7984 68 176

September 2013

Applications that interact with SEEK-AT-WD

25



Educators community

Web of Datapublish

data



Obtain descriptions




publ

ish

info

rmat

ion

abou

t IC

T to

ols








Educators community

Web of Datapublish

data


U-Seek

Obtain descriptions




publ

ish

info

rmat

ion

abou

t IC

T to

ols





We-Share

Applications: DEMO

26


Introduction:



Contributions:


Applications demo!!

Evaluation

Conclusions

27

Evaluation issues and methodology

28

Is it possible to

develop a sustainable

registry of educational

ICT tools?

Mixed methods [Tas03][Cre03]

CSCL-EREM [JA09]

Adaptation of the mixed evaluation method by [Mar03]

I 2: Does a social-semantic approach allow the collection

of educational ICT tool descriptions in a sustainable way?

I 1: Is the information already published on the

Web of Data able to support educators when discovering

ICT tools?

ID Happening Date

1 Tool descriptions from the Web Spring 2011

2 SEEK-AT-WD administration January 2012 – March 2013

3 Data published on We-Share Winter 2013

1 2

3


29


30

Aim: analyze whether the data available on the Web is useful for educators to discover ICT tools

Preliminary dataset of educational ICT tools from the Web of Data

– Ontoolcole-based dataset [VG08]

– 2753 different ICT tool descriptions

31

Tool descriptions from the Web: Happening description (I)

Evaluation Tool descriptions from the Web: Happening description (II)

[WOZ-x] Wizard of Oz experiment [Kel84]

– 12 participants (43 questions)

– Reuse of 33 questions from Ontoolcole evaluation [VG08]

[EXP-y] Expert panel with 8 TEL practitioners and

researchers

[INT-z] Interviews to 4 participants

49

Tool descriptions from the Web: Selected findings and evidence

33

It is possible to generate a useful dataset of educational ICT tools out of data from the Web

– A dataset with 2753 tool descriptions was automatically created

– [INT-1] The most important thing I would expect to

retrieve is a short description of each tool

– [INT-2] If a search engine starts with such amount of

content, it obviously encourages you to use it

Mappings add a significant value

– [EXP-1]An important added value is that it is possible

to answer Ontoolcole questions using external non-educative data

Tool descriptions from the Web: Selected findings and evidence

34

Most educators’ information needs are satisfied using external data

– [WOZ-1] Median of results = 134 (SIQR = 58)

– [WOZ-2] 27 out of 32 information needs were

satisfied with 5 random results

The results retrieved by the repository are precise enough

– [WOZ-3] 57% of satisfactory results

– [INT-3] I got 54 results for my query. If 10 out of

these 54 satisfy my query I think it would be enough

35

Evaluation SEEK-AT-WD administration: Happening description Aim: discuss whether the web community can

contribute to SEEK-AT-WD sustainment and whether its administration effort may hinder its sustainability

[DAG-x] Analysis of the data gathered by SEEK-AT-

WD

– March 2013

[ADL-y] Analysis of SEEK-AT-WD administration log

– Log from January 2012 to March 2013

SEEK-AT-WD administration: Selected findings and evidence (I)

36

SEEK-AT-WD automatically generates a dataset that covers almost all SEEK Ontology categories

– [DAG-1] 3/46 tool categories could not be mapped to

concepts defined in external vocabularies

– [DAG-2] Several tens of tool descriptions in each

category

– [DAG-3] +6000 tool descriptions imported (March

2013)

– [DAG-4] Noisy data appears (as expected)

SEEK-AT-WD administration: Selected findings and evidence (II)

37

The web community contributes to the sustainment of SEEK-AT-WD

– [DAG-5] New tool descriptions are imported (e.g.

Google Drive or Academic Room)

The SEEK-AT-WD administration tasks can reasonably be done by a technical administrator

– [ADL-1] SEEK-AT-WD administration tasks took 24

hours during the first 15 months

– [ADL-2] Flexibility of the SEEK Ontology and

mappings

Data published on We-Share: Happening description Aim: evaluate the possibility of collecting useful data

using We-Share

23 participants with experience using ICT in classroom

– No previous knowledge of the applications

[UWS-x] Use of We-Share to publish data

[COM-x] Comparison of two versions of U-Seek

– Based on the TREC workshops [Lag98]

– Using and not using social information

– Metrics: satisfaction on the tools discovered and confidence on the tools selected

[QUE-Y] Questionnaires

38

Data published on We-Share: Selected findings and evidence (I)

39

U-Seek and We-Share allow educators to interact with SEEK-AT-WD

– [COM-1][USW-1] No important usability problems

were detected due to the origin of the data they manage

– [COM-2][USW-2] Most participants agreed that

these are useful and easy-to-use applications

– [COM-3] In 80 out of 82 observations using U-Seek,

the participants satisfied their information needs

– [QUE-1] The main problem for me is the classification

of the taxonomies […] But maybe I can solve it with a bit of practice

Data published on We-Share: Selected findings and evidence (II)

40

Educators find useful the data added using We-Share

– [COM-4] No significant difference on the satisfaction

of the tools discovered

– [COM-5] Significant difference on the confidence of

the tools selected when social information is available (90% confidence interval)

– [QUE-2]At first, I did not understand the exact meaning

of the term ‘educational context’. After watching some other reviewed tools, it became clearer

– [QUE-3] I discovered the option ‘Has reviews’!!! (very

useful!!!)

Wrap up: Issue 1

41

Wrap up: Issue 1

42

The data published on the Web is useful to palliate the cold-start problem

– Educators can discover tools using data from the Web once related to an educational vocabulary

– The support provided by this data to the educators can be improved

Educators can take advantage of the tool descriptions available on the Web of Data using interactive applications

Is the information already published on the Web of Data able to support educators when discovering ICT tools?

Wrap up: Issue 2

43

Wrap up: Issue 2

44

SEEK-AT-WD automatically updates its dataset obtaining information from the Web of Data

SEEK-AT-WD administration tasks are reasonable to be assumed by a technical administrator

Educators could be expected to sustain SEEK-AT-WD data

– We-Share is perceived as a useful and easy-to-use application

– The information published on We-Share improves the utility of SEEK-AT-WD

Does a social-semantic approach allow the collection of educational ICT tool descriptions in a sustainable way?

Wrap up: Some caveats

45

…but there are some (expected) caveats

– Three happenings with a limited number of participants

• But triangulation increases the trustworthiness of the findings and conclusions

– The SEEK Ontology had an important impact on the results obtained

• It does not describe purpose-specific tools

• Its users are higher-education educators

– Limited amount of social-information available

• Although enough to increase the utility of SEEK-AT-WD


Introduction:



Contributions:


Applications demo!!

Evaluation

Conclusions

46

Conclusions (I)

47

There are data sustainability problems in current registries of educational ICT tools

SEEK-AT-WD follows a social-semantic approach

– Takes advantage of the Linked Open Data to create and update a collection of ICT tool descriptions

– Allows the community of educators to further enrich its dataset

U-Seek and We-Share are two applications that allow educators to interact with SEEK-AT-WD

– Educators find them useful and easy to use

Conclusions (II)

48

SEEK-AT-WD presents sustainability advantages derived from following a social-semantic approach

– Advantages for educators

• Palliate the cold-start problem

• Obtain updated descriptions of ICT tools

• Obtain information published by other communities

…and also

• Semantic searches are allowed

• Share their experience using tools in the classroom

Conclusions (III)

49

SEEK-AT-WD presents sustainability advantages derived from following a social-semantic approach

– Advantages for registry administrators

• Can reasonably be administered by a technician

– Advantages for the web community

• Third parties can make use of the data managed by SEEK-AT-WD

• Open educational-specific data on the Web

Conclusions (IV)

50

Problems derived from the social-semantic approach

– Importation of noisy data from the Web

• Convenient to curate SEEK-AT-WD dataset

• The precision of the search results decreases

– Educators need to understand SEEK Ontology

– Other problems related to the creation of a community of users

• Engagement

SEEK-AT-WD: A social-semantic infrastructure for educators to discover and

select ICT tools in the Web of Data

Adolfo Ruiz-Calleja

08th, October 2014

adolfo ruiz calleja "using social and semantic tech"

Education

socialsemantic solution

social web approach

web of datacontributions

socialsemantic infrastructure

problem definitionapproach

educational use

peer review activityquery

parties uris