humanities computing

Humanities computing

Claudine Chionh

University of Melbourne

2 March 2010 / LUV meeting

Claudine Chionh (University of Melbourne) Humanities computing 1/ 48 2 March 2010 1 / 48

Overview

The Founders and Survivors project

Humanities computing

Why FOSS?

Challenges

Drupal in action: Data migration

Claudine Chionh (University of Melbourne) Humanities computing 2/ 48 2 March 2010 2 / 48

Founders and Survivors

The Claudine

Woolwich 24/aug/1821 to Hobart 15/dec/1821 – 113 days at sea

160 male convicts boarded, 159 survived/landed (not a bad record)

Claudine Chionh (University of Melbourne) Humanities computing 3/ 48 2 March 2010 3 / 48

Founders and Survivors

Journal

Claudine Chionh (University of Melbourne) Humanities computing 4/ 48 2 March 2010 4 / 48

Founders and Survivors

Conduct registers

Claudine Chionh (University of Melbourne) Humanities computing 5/ 48 2 March 2010 5 / 48

Founders and Survivors

Archives of Tasmania convict index

Claudine Chionh (University of Melbourne) Humanities computing 6/ 48 2 March 2010 6 / 48

Founders and Survivors

Founders and Survivors

Claudine Chionh (University of Melbourne) Humanities computing 7/ 48 2 March 2010 7 / 48

Founders and Survivors

Van Diemen’s Land

Transportation period, 1803-1853

∼ 1 million rows of data

Quantifiable data: conduct registers, BDM. . .

Text: journals, newspaper reports

Claudine Chionh (University of Melbourne) Humanities computing 8/ 48 2 March 2010 8 / 48

Founders and Survivors

Genealogists

What happened to convicts after they were freed?

Links with genealogists for lives of convicts and their families.

Claudine Chionh (University of Melbourne) Humanities computing 9/ 48 2 March 2010 9 / 48

Founders and Survivors

The ‘factory plan’

Claudine Chionh (University of Melbourne) Humanities computing 10/ 48 2 March 2010 10 / 48

Humanities computing

Old questions, new tools?

Digitisation

Analyse large[r] amounts of material

Public access and collaboration

Claudine Chionh (University of Melbourne) Humanities computing 11/ 48 2 March 2010 11 / 48

Humanities computing

The Valley of the Shadow

Claudine Chionh (University of Melbourne) Humanities computing 12/ 48 2 March 2010 12 / 48

Humanities computing

The Valley of the Shadow

Claudine Chionh (University of Melbourne) Humanities computing 13/ 48 2 March 2010 13 / 48

Humanities computing

Old Bailey Online

Claudine Chionh (University of Melbourne) Humanities computing 14/ 48 2 March 2010 14 / 48

Humanities computing

Perseus Digital Library

Virgil’s Aeneid

Claudine Chionh (University of Melbourne) Humanities computing 15/ 48 2 March 2010 15 / 48

Humanities computing

‘Libraries’: Literary and linguistic applications

Index Thomisticus (1946)

Perseus

Claudine Chionh (University of Melbourne) Humanities computing 16/ 48 2 March 2010 16 / 48

Humanities computing

‘Archives’: Historical applications

Digitisation

Data analysis

Collaboration

Claudine Chionh (University of Melbourne) Humanities computing 17/ 48 2 March 2010 17 / 48

Humanities computing

Digitisation

Documents

Images

Linked/cross-referenced presentation of sources

Claudine Chionh (University of Melbourne) Humanities computing 18/ 48 2 March 2010 18 / 48

Humanities computing

Tasmanian Police Gazette, 1861-1933

Claudine Chionh (University of Melbourne) Humanities computing 19/ 48 2 March 2010 19 / 48

Humanities computing

Surgeon’s journals

Claudine Chionh (University of Melbourne) Humanities computing 20/ 48 2 March 2010 20 / 48

Humanities computing

Conduct registers

Claudine Chionh (University of Melbourne) Humanities computing 21/ 48 2 March 2010 21 / 48

Humanities computing

Data analysis

Claudine Chionh (University of Melbourne) Humanities computing 22/ 48 2 March 2010 22 / 48

Humanities computing

GIS

Claudine Chionh (University of Melbourne) Humanities computing 23/ 48 2 March 2010 23 / 48

Humanities computing

Collaboration

Claudine Chionh (University of Melbourne) Humanities computing 24/ 48 2 March 2010 24 / 48

Why FOSS?

Why FOSS?

Community of developers

Access

Values

Claudine Chionh (University of Melbourne) Humanities computing 25/ 48 2 March 2010 25 / 48

Why FOSS?

Community of developers

Mutual support

Don’t reinvent the wheel

Using and adapting tools

Claudine Chionh (University of Melbourne) Humanities computing 26/ 48 2 March 2010 26 / 48

Why FOSS?

Access

Make archival sources and research results accessible to general public

Sharing data with other researchers

Claudine Chionh (University of Melbourne) Humanities computing 27/ 48 2 March 2010 27 / 48

Why FOSS?

Values

Public interest

Free access, free expression

Dialogue

Public participation

Claudine Chionh (University of Melbourne) Humanities computing 28/ 48 2 March 2010 28 / 48

Challenges

The Two Cultures

The Two Cultures and the Scientific Revolution

CP Snow, Rede Lecture, 1959

Literature/humanities vs science/tech

Claudine Chionh (University of Melbourne) Humanities computing 29/ 48 2 March 2010 29 / 48

Challenges

Many cultures?

Translating between academics, IT professionals, diverse public audience

Different priorities, research questions

Claudine Chionh (University of Melbourne) Humanities computing 30/ 48 2 March 2010 30 / 48

Challenges

Geeks and non-geeks

Non-geeks may not understand the values behind FOSS

Technology as magic

Claudine Chionh (University of Melbourne) Humanities computing 31/ 48 2 March 2010 31 / 48

Challenges

Where do developers belong?

Identity crisis

‘Digital humanities professional’?

Background – IT or academic?

Autonomy

Career progression

Where do humanities computing projects belong?

Claudine Chionh (University of Melbourne) Humanities computing 32/ 48 2 March 2010 32 / 48

Data migration

Screenshot: Index record

Claudine Chionh (University of Melbourne) Humanities computing 33/ 48 2 March 2010 33 / 48

Data migration

Why Drupal?

Modular

Define our own content types and views

Define user roles

Workflow

Claudine Chionh (University of Melbourne) Humanities computing 34/ 48 2 March 2010 34 / 48

Data migration

Index data

Access –> Excel –> CSV –> Drupal

Database on the web

Claudine Chionh (University of Melbourne) Humanities computing 35/ 48 2 March 2010 35 / 48

Data migration

Content Construction Kit

http://drupal.org/project/cck

Define your own data structures in Drupal

Claudine Chionh (University of Melbourne) Humanities computing 36/ 48 2 March 2010 36 / 48

Data migration

Rules

http://drupal.org/project/rules

More powerful than core Trigger and Action modules

Generate a title for each node

{index number} | {convict name} ({ship name})

Claudine Chionh (University of Melbourne) Humanities computing 37/ 48 2 March 2010 37 / 48

Data migration

Views

http://drupal.org/project/views

Define your own views of content

Claudine Chionh (University of Melbourne) Humanities computing 38/ 48 2 March 2010 38 / 48

Data migration

Table Wizard

http://drupal.org/project/tw

Expose a MySQL table or CSV file to Views

Claudine Chionh (University of Melbourne) Humanities computing 39/ 48 2 March 2010 39 / 48

Data migration

Table analysis

Claudine Chionh (University of Melbourne) Humanities computing 40/ 48 2 March 2010 40 / 48

Data migration

Migrate

http://drupal.org/project/migrate

Map structure of external table to a Drupal data structure

Migrate Extras http://drupal.org/project/migrate_extras tomigrate to CCK fields

Claudine Chionh (University of Melbourne) Humanities computing 41/ 48 2 March 2010 41 / 48

Data migration

Content set

Claudine Chionh (University of Melbourne) Humanities computing 42/ 48 2 March 2010 42 / 48

Data migration

Migrate dashboard

Claudine Chionh (University of Melbourne) Humanities computing 43/ 48 2 March 2010 43 / 48

Data migration

Drush

Web-based dashboard good for testing on small samples

Drush: the Drupal Shell http://drupal.org/project/drush

(out of memory issues)

Run drush migrate-import {content set} from cron

Approx. one week to migrate ∼ 80,000 records

Claudine Chionh (University of Melbourne) Humanities computing 44/ 48 2 March 2010 44 / 48

Where to from here?

Next stage of project

(manually) linking official index with public submissions

later life stories

WWI links

Claudine Chionh (University of Melbourne) Humanities computing 45/ 48 2 March 2010 45 / 48

Where to from here?

Links

The Valley of the Shadow http://valley.lib.virginia.edu/

Old Bailey Online http://www.oldbaileyonline.org/

Perseus Digital Library http://www.perseus.tufts.edu/

Index Thomisticus http://www.corpusthomisticum.org/it/

Claudine Chionh (University of Melbourne) Humanities computing 46/ 48 2 March 2010 46 / 48

Where to from here?

Links

Founders and Survivors http://www.foundersandsurvivors.org/

Mapping Our Anzacs http://mappingouranzacs.naa.gov.au/

Australian Newspapers (National Library)http://newspapers.nla.gov.au/

Essays in Humanities Computinghttp://www.digitalhumanities.org/Essays/

Claudine Chionh (University of Melbourne) Humanities computing 47/ 48 2 March 2010 47 / 48

Where to from here?

Questions/advice?

http://claudine.github.com/2010/02/17/

drupal-table-wizard-migrate.html

http://www.slideshare.net/claudinec

Claudine Chionh (University of Melbourne) Humanities computing 48/ 48 2 March 2010 48 / 48