CUbRIK Presentation 1

Humanist-Machine Interaction for the digital humanitiesBuilding the Social graph of the History of European Integration

DHLU 2013

www.cubrikproject.eu

About CUbRIK

European Community's Seventh Framework Program FP7-ICT

15 European partners Multimedia search

processing: Puttinghumans in the loop

Demos: History of Europeand Fashion

CUbRIK Presentation 2

CUbRIK Presentation 3

Why CUbRIK approach

Human in the loop added value: Verification of identities/places/events ambiguous and

temporal only possible by putting humans in the loop Integration of multiple perspectives

CUbRIK as an open toolbox allows follow-up and extension through third parties

“Vertical” integration:GUI, components, crowdsourcing

integrated in a platform

Specification process: User pull & technology push

CUbRIK Presentation 4

The situation today

More and more sources such as images, texts and videos are being digitalized

How can we extract high-level information from them?

Harnessing the effectiveness of machines and the expertise of (expert) crowds

For starters: 3000 images representing the history of European integration, more documents to come soon

CUbRIK Presentation 5

CUbRIK Review 6

History of Europe App

Name: HoE App Domain: digital humanities Target users: researchers in digital humanities Rationale: software-enhanced social hermeneutics

based on heterogeneous multimedia archives

Researcher approach Researchers manually verify information about historical photos Neglects aggregate information from different sources that could lead to new

insights

CUbRIK solution: Full pipeline for the discovery of relationships in heterogeneous data Integration of human and machine computation Visualization of relationships Discursive interface enabling the mapping of multiple perspectives

27/11/2013

CUbRIK Review 7

Main Challenges

The main challenges in the HoE app: Detection and identification of identities/places/events in time Verification of identities/places/events in time Analysis of relationships (e.g. co-occurrences) Rights aware crawling and storage Verification of provenance and license information

Approach: Crowd-sourced verification of detected faces (false

positives/negatives) Verification of identities through/places/events in time social

networks of experts Visual knowledge discovery/exploration Integrated rights aware crawling and storage Integrated license and provenance management

27/11/2013

CUbRIK in a nutshell

CUbRIK Presentation 8

Pipelining the CUbRIK components:Conflict resolution

CUbRIK Presentation 9

Raw content

High level features (automatic annotations)

Conflict(e.g., “Image contains

‘Romano Prodi’ ”Confidence = low)?

Conflict store Conflict manager

Conflict resolution task store

Conflict resolution task: conflict,

required skill, priority, ..

CUbRIK app for Conflict resolution

Game Q&ACrowdtask

Pipelining the CUbRIK components:Human input from click-workers

Great choice for simple tasks: Face detection: false positives, false

negatives Monetary motivation, via

www.microtask.com

Poor performance on complex tasks: Low resolution images Different angles etc. Actors recurring over time

CUbRIK Presentation 10

Pipelining the CUbRIK components:Human input from experts

Capable of complex tasks: In-depth knowledge of key actors Context knowledge allows inferences

But: Different motivational models! Public goods Reputation

CUbRIK Presentation 11

Pipelining the CUbRIK components:Towards a community of experts

Time for a Demo!

CUbRIK Presentation 12

Usage for historians

No one truth in history but interpretation, context and discussion

Therefore need to represent ambivalence, contradictions and discussion

Close ties between data representation (Social graph) and their original context (primary sources)

CUbRIK Presentation 13

Conclusion & Outlook

Challenges What is truth? Humanities vs. Computer

Science Gathering requirements for tools that haven‘t

been developed yet Engaging crowds Image copyrights

Refinement of the application Additional datasources Improvement of the interface Integration of the different components

CUbRIK Presentation 14

WWW.CUBRIKPROJECT.EU

Visit us on

CUbRIK Presentation 15

@CUBRIKPROJECT

Or follow us on Twitter