Download - Part of the UK Data Archive and the Arts and Humanities Data Service. Funded by the Joint Information Systems Committee and the Arts and Humanities Research

Good Design - © History Data Service 1

Part of the UK Data Archive and the Arts and Humanities Data Service. Funded by the Joint Information Systems Committee and the Arts and Humanities Research Board.

Databases of Historical Sources: Principles of Good

Design

Mark Merry

History Data Service


History as a product

“History may be thought of as either product or process. As a product, a piece of history consists of a representation of a past reality based upon the interpretation of a body of known facts. Such representations of past realities are always bounded: they treat a subject chosen by the historian which might be static (the situation at point x) or dynamic (how the situation changed between points x and y).”

Harvey and Press, Databases in Historical Research (1996)

Chapter 1


What is a database?

• “Basically a computerized record keeping system - that is, a system whose overall purpose is to maintain information and to make that information available on demand”

C.J. Date, An Introduction to Database Systems.

Vol I. Seventh edition (1999)

• A Database Management System (DBMS) is the computer application built around a database to provide flexible ways of storing, manipulating, and examining the data

• A DBMS on a personal computer will provide facilities for:– inputting, sharing, modifying, retrieving and deleting data– querying the data (SQL)– producing reports based on the data– building ‘front-ends’ for users


What can databases do for us?

• Store and organise large amounts of information• Provide a surrogate for the original (inaccessible, fragile)

source• Group physically dispersed material together at one virtual

location• Provide an environment for manipulating and analysing the

content of the original source• To search/filter/summarise complex information quickly

– Analysis of large amounts of data– Analysis of complex interrelated data

• Downside: the time and effort needed to convert original sources into a database


Source → Digitisation → Resource

• The ‘input channels’ of digitisation (keyboard, scanner etc.) are narrow and can only capture a small proportion of the source’s information content

identify aspects of source to digitise

chose digitisation method

chose data model


Organising data

• The field is the basic unit of data in a database. A field stores a single piece of information of a particular data type

• Fields are combined to form a record. A record matches an entity

• A set of records with the same fields are collected together in a table


Data models

• Data models are the abstract definitions of structures and relationships used to organise data

• A DBMS will implement a particular data model• Data models can be characterised by how they organise the

connections between different records:– flat file– relational– mark-up (hierarchical)

• Most DBMS’s available for personal computers are either flat file or relational


Relationships

• One to one relationships connect one entity to one other entity

• One to many relationships connect one entity to one or more other entities

• Many to many relationships connect many entities to many other entities


Relationships example

H'hold_n ADDRESSA010 PINNER COMMONA011 PINNER COMMONA012 PINNER COMMONA013 PINNER WOODA014 PINNER GREEN TOLL

H'hold_n SURNAME FORENAME OCCUNAMEA010 SNOOK GEORGE POLICEMANA010 SNOOK ANNA010 SNOOK SARAH HANNAHA011 DEAN JAMES SAWYERA011 DEAN MARGARETA012 ROBERTSON MARIA INDEPENDENT LADYA012 EDMONDS EMILY SERVANTA013 CRAWLEY GEORGE AG LABA013 CRAWLEY MARY ANNA013 CRAWLEY CAROLINEA013 CRAWLEY ELIZABETH

OCCUCODEOCCUNAME(blank) SCHOLAR AT HOME2PP13 SCHOOL MISTRESS2PP13 SCHOOLMISTRESS4DS1 SERVANT4DS3 SERVANT AND GROOM3AG1 SHEPHERD


Databases of historical sources

• An historical source based database is a representation of the original source, but it is not an exact replica of the original source– information may be left out– extra information may be included

• An historical database should:– try to reflect the source accurately and completely – improve the usability of the source– integrate the source with other data (additional sources, coding etc.)

• NB: these are conflicting aims!• An historical source based database mixes elements of a

primary source with elements of a secondary source


Building databases from historical sources

• Historians work with information they do not control– incomplete, poorly structured information of varying quality– sources intended for a different purpose– multiple sources not intended to be used in an integrated way

• Nature of historical sources– ambiguity: the meaning of material may be unclear or dependent on

its context– repetition: data is often repeated in different guises– variation: the same item can be referred to using a variety of terms

and spellings– variable structure: even apparently well organised sources often

have margin notes and other types of ‘random’ additional data


Source → information content

• Simplify the source– Ignore unwanted information– Exclude certain types of information– Select information directly from the source or define a set of

summarised information based on the source

• Model the information content subset– Break information content into discrete elements of information– Describe the characteristics of each information element– Describe how information elements relate to each other

• Successful source analysis requires a good understanding of the source and of the purpose of the database


Source → database content

fonts

columnstext

spacing

page size

date issue

fold linemarginalia

headlines


Software & hardware

• Technical decisions are often the least important• Remember that there is nearly always more than one way of

doing something with a computer• Define what you need to do, then seek technical advice• Seek a second opinion!

– Technical support staff will often suggest what is most convenient for them, not necessarily you

– Commercial companies obviously have their own motives

• Look for software that supports common standards• Avoid little-used software with proprietary features• Recognise that hardware may need to be replaced in 2 or 3

years


The ‘Three Layer Model’

Standardisation Layer•provides a foundation for analysing the data

•codes and standardisation rules are applied

Source Layer•an accurate digital representation of the source

•defines level of detail captured

Interpretation Layer•incorporates researcher’s knowledge and judgement

•Links records and forms aggregates


‘Three Layer’ design examples

Source Standardise Intrepretation6 mnths 0.5 infant

ag. lab. agricultural labourer farm occupations

J. SmithJohn A. Smith, bakerJ.A. SmithSmith & Son Bakers

J. ? SmithJ. A. SmithJ. A. Smith? ? Smith

John Smith, Baker

MdlboroughMdsbroMeddlesbroMedelsbro

Middlesbrough Middlesbrough,Yorkshire


Some simple design hints

• The smallest unit of data should match the smallest unit of analysis– if you want to look at people by last name then have separate first

and last name fields, not just a name field

• Don’t mix data types– separate numbers and words– identify numbers being used as words (addresses)

• Document everything you do, either in the database or with the database– data entry, data standardisation and coding, data transformations,

limits of data, issues of plausibility/probability, coping with uncertainty etc

– keep information that tracks the origin and history of the database.

• Add information, don’t delete information• Have a backup procedure!


Further Information

• History Data Service

web - http://hds.essex.ac.uk

email - [email protected]• Michael J. Hernandez, Database Design for Mere Mortals : A Hands-On

Guide to Relational Database Design, Addison-Wesley, 1997• Charles Harvey & Jon Press, Databases in Historical Research,

Macmillan Press, 1996• C. J. Date, An Introduction to Database Systems, Addison-Wesley, 1999

(7th ed.)• SearchDatabase.Com: http://searchdatabase.techtarget.com/• Concordia University: http://www.cuaa.edu/computing/softrain

/access/access15.shtml • University of Newcastle Database Service: http://www.ncl.ac.uk/ucs

/databases/

http://searchdatabase.techtarget.com/





http://www.cuaa.edu/computing/softrain/access/access15.shtml








http://www.ncl.ac.uk/ucs/databases/








Part of the UK Data Archive and the Arts and Humanities Data Service. Funded by the Joint Information Systems Committee and the Arts and Humanities Research Board.

web - http://hds.essex.ac.uk/

email - [email protected]


Source Layer

• Acts as the reference version of the original source.– An accurate representation of the source, including errors, omissions

etc.– Contents determine the highest level of detail available about the

source in the database– Includes a reference to the non-digital original source– Includes a unique identifier for each item

• Implementation:– as long text fields containing full text transcriptions– as ‘blob’ fields containing scanned images


Standardisation Layer

• Organises the information into discrete units with fully defined contents– Separates information in the source into separate fields according to

data type and data content– Simplifies the data by standardising and coding it– Normalises the data– Includes links back to the source layer

• Implementation:– Possibly as addition columns in source layer tables– Probably as separate tables with, ideally, a one-to-one relationship

to records in the source layer


Interpretation Layer

• Creates historical entities from the data and the knowledge and expertise of the historian– Incorporates interpolations and extrapolations from the data in the

standardisation layer– Selectively includes and excludes information from the

standardisation layer– Links separate records to form entities such as ‘individuals’ or

‘households’

Download - Part of the UK Data Archive and the Arts and Humanities Data Service. Funded by the Joint Information Systems Committee and the Arts and Humanities Research

Top Related