humanist machine interaction with histograph

CUbRIK Presentation 0

Humanist-Machine Interaction for thedigital humanitiesBuilding the Social graph of the History of European Integration

DHLU 2013

www.cubrikproject.eu

About CUbRIK

European Community's Seventh FrameworkProgram FP7-ICT

15 European partners Multimedia search

processing: Puttinghumans in the loop

Demos: History of Europeand Fashion

CUbRIK Presentation 1

CUbRIK Presentation 2

Why CUbRIK approach

Human in the loop added value: Verification of identities/places/events ambiguous and temporal only

possible by putting humans in the loop Integration of multiple perspectives

CUbRIK as an open toolbox allowsfollow-up and extension throughthird parties

“Vertical” integration:GUI, components, crowdsourcing

integrated in a platform

Specification process: User pull & technology push

CUbRIK Presentation 3

The situation today

More and more sources such as images, textsand videos are being digitalized

How can we extract high-level information fromthem?

Harnessing the effectiveness of machines andthe expertise of (expert) crowds

For starters: 3000 images representing thehistory of European integration, more documentsto come soon

CUbRIK Presentation 4

5

History of Europe App Name: HoE App Domain: digital humanities Target users: researchers in digital humanities Rationale: software-enhanced social hermeneutics

based on heterogeneous multimedia archives

Researcher approach Researchers manually verify information about historical photos Neglects aggregate information from different sources that could lead to new insights

CUbRIK solution: Full pipeline for the discovery of relationships in heterogeneous data Integration of human and machine computation Visualization of relationships Discursive interface enabling the mapping of multiple perspectives

CUbRIK Presentation

6

Main Challenges The main challenges in the HoE app:

Detection and identification of identities/places/events in time Verification of identities/places/events in time Analysis of relationships (e.g. co-occurrences) Rights aware crawling and storage Verification of provenance and license information

Approach: Crowd-sourced verification of detected faces (false positives/negatives) Verification of identities through/places/events in time social networks of

experts Visual knowledge discovery/exploration Integrated rights aware crawling and storage Integrated license and provenance management

CUbRIK Presentation

CUbRIK in a nutshell

CUbRIK Presentation 7

Pipelining the CUbRIK components:Conflict resolution

CUbRIK Presentation 8

Raw contentHigh level features

(automatic annotations)

Conflict(e.g., “Image contains

‘Romano Prodi’ ”Confidence = low)?

Conflict store Conflict manager

Conflict resolution task store

Conflict resolution task: conflict,

required skill, priority, ..

CUbRIK app for Conflict resolution

Game Q&ACrowdtask

Pipelining the CUbRIK components:Human input from click-workers

Great choice for simple tasks: Face detection: false positives, false negatives Monetary motivation, via www.microtask.com

Poor performance on complex tasks: Low resolution images Different angles etc. Actors recurring over time

CUbRIK Presentation 9

Pipelining the CUbRIK components:Human input from experts

Capable of complex tasks: In-depth knowledge of key actors Context knowledge allows inferences

But: Different motivational models! Public goods Reputation

CUbRIK Presentation 10

Pipelining the CUbRIK components:Towards a community of experts

Time for a Demo!

CUbRIK Presentation 11

Usage for historians

No one truth in history but interpretation,context and discussion

Therefore need to represent ambivalence,contradictions and discussion

Close ties between data representation (Socialgraph) and their original context (primarysources)

CUbRIK Presentation 12

Conclusion & Outlook

Challenges What is truth? Humanities vs. Computer Science Gathering requirements for tools that haven‘t been

developed yet Engaging crowds Image copyrights

Refinement of the application Additional datasources Improvement of the interface Integration of the different components

CUbRIK Presentation 13

WWW.CUBRIKPROJECT.EUVisit us on

CUbRIK Presentation 14

@CUBRIKPROJECTOr follow us on Twitter