prestocentre-data - new york universitybesser.tsoa.nyu.edu/howard/talks/12prestocentre-data.pdf ·...

5/21/12

1

Audiovisual as Research Data

Howard Besser, Director NYU Moving Image Archiving & PreservaDon

hEp://www.nyu.edu/Dsch/preservaDon/

1 Besser-‐Screening The Future-‐Data

Audiovisual as Research Data-‐

•  Making works already in our collecDons (for other purposes) useable for future research

•  Issues with taking in new works that are products of the research environment –  The tsunami of digital informaDon that will need to be preserved, as the largest funding agencies change their mandates

– What does moving image research data look like, and what do we need to worry about?

–  Is sensor output really “moving image” material?

2 Besser-‐Screening The Future-‐Data

Making exisDng works more useable for future research

Besser-‐Screening The Future-‐Data 3

Researching LeZ-‐handedness Current Biology, Volume 17, Issue 18, R793-‐R794, 18 September 2007


Mitchell & Kenyon Football Match, 1902 hEp://www.bfi.org.uk/features/mk/


Problems w/using 1902 films to research leZ-‐handedness

•  Filming wasn’t set up as a research project, so there’s methodological problems

•  Archive didn’t index for hand-‐waving, so finding films that contain this is hit-‐and-‐miss

•  Need to keep hedging on your findings


5/21/12

2

If Archives want to enable Research

•  At minimum, we need to provide beEer tools for Discovery

•  example from Center for Home Movies-‐


Center for Home Movies —Summit


Portal Project— Taxonomies example (1/3)

•  Possible Genre categories: –  Milestone film (weddings, bar mitzvahs, Christmases, birthday parDes,

etc.)

–  Family at Leisure

–  Family business/Livelihood

–  Public events –  Travelogues/Community portraits –  Amateur Dramas

–  Art Film



•  Possible Tropes by Human Behavior –  Coming of Age/Childhood Development /Milestone Moment

–  The "Check out our stuff" moDf

–  Self-‐documentaDon/Self-‐aggrandizement

–  "Oh no he didn't!” –  Images of the Beloved –  “Mugging” for the Camera

–  Trainspoong

–  “Look Ma, no hands”.

–  "Look, an animal! And it's so close I can feed it a potato chip!"

–  Human Diversity as Curiosity



•  Possible Tropes by Recurring Imagery – Parades – Public Events – The Road – Gardens – Special Weather – DemoliDons and ConstrucDons


•  Once you provide access to home movies by these taxonomies, you enable research

•  And not just research in social science areas like history, sociology, gender studies, etc. But also literary studies-‐


5/21/12

3

Literary Studies research

•  For each Trope, Genre, etc., think about applying literary studies techniques:

•  How would we define each of these? What core elements need to be present?

•  How frequently does each occur, and under what condiDons?

• What types of variants are there within a given trope or genre?

•  How do the tropes or genres relate to each other? • What other tropes or genres have not yet been defined?


Last 15 years—shiZ towards Ephemeral media from everyday life – Works that document everyday life

•  Amateur films

•  Home movies •  Anthropological footage •  Daily news and Newsreels •  Other raw footage and out-‐takes

– Works that reveal the prioriDes of powerful social forces •  Industrial films

•  Government films

– Works created to influence us •  EducaDonal films •  AdverDsements

Besser-‐IFA Hidden Archives, 3/25/11 14

Orphans

Besser-‐IFA Hidden Archives, 3/25/11 15

Challenges for Scholarship •  Changing paradigms; Re-‐shaping of film history

•  Broadening what consDtutes the content/subject of study

•  What is collected by archives heavily influences what is studied (favoring a “great works” approach)

•  What is easiest to study becomes favored (length of Dme to analyze, jusDfying your work and producDvity to campuswide tenure commiEees, qualitaDve vs quanDtaDve, …)

16 Besser-‐IFA Hidden Archives, 3/25/11

The growth in interest in “everyday” moving images began to get us closer to what we need

for Research •  Older discovery pracDce had us treat a moving image work as a whole, with metadata applied to the enDre work (rather than pieces) –  Cinema: Dtle, director, country, year –  BroadcasDng: network/staDon, Dtle, year

•  Only the old discovery pracDces for News (another type of “everyday” work) someDmes let us treat parts of the whole with Dme-‐code based metadata –  Date, key-‐words (oZen derived from logs)


Research requires being able to aEach metadata to Dme-‐codes


5/21/12

4

ScienDfic data has tradiDonally existed in Silos

•  Movement to require researchers to make their data available to others


Science Commons Jesse Dylan Video hEp://sciencecommons.org/index.php (“new insights” 1:15)


General “Open Access” Ideas (paralleled ideas of Data Sharing )

•  UN Economic and Social Council (2000) – “universal access to knowledge and informaDon”

•  Budapest Open Access IniDaDve (2001) –  “to accelerate progress in the internaDonal effort to make research

arDcles in all academic fields freely available on the internet”

•  Vienna DeclaraDon (2005) –  10 Theses on Freedom of InformaDon

•  Kronberg DeclaraDon on the Future of Knowledge AcquisiDon and Sharing (2007) –  UNESCO High Level Group of Visionaries on Knowledge AcquisiDon and

Sharing Besser-‐Screening The Future-‐Data 21

Data Sharing-‐-‐History

•  Bromley Principles (1991) -‐-‐ Full and Open Access to "Global Change" Data

•  Human Genome Project (1990-‐2000/2003/2006) –  Bermuda principles (1996)—DNA sequence Data released into public DBs w/i 24 hours of generaDon

–  Fort Lauderdale Statement (2003)—Wellcome Trust

•  DeclaraDon on Access to Research Data From Public Funding (2004)—OECD

•  UK establishes “Open Data InsDtute” with £10 million funding (Nov 2011)


Shareable Research Data became Mandated by Funders

•  Wellcome Trust—2007/2010 –  requires all funded researchers to maximise access to research data

with as few restricDons as possible

–  requires data management & sharing plans for proposals generaDng data that could be shared for added value

–  commits to meet costs for data sharing acDviDes outlined in the plans

•  Joint Statement by InternaDonal Public Health Funders—Jan 2011 Lancet –  promote greater access to and use of the research data that they fund

•  Research Councils of UK “Common Principles on Data Policy” (2011)-‐

•  NSF-‐-‐January 2011-‐ Besser-‐Screening The Future-‐Data 23

Research Councils of UK “Common Principles on Data Policy”

•  Making research data available to users is a core part of the Research Councils’ remit

•  Publicly funded research data are a public good, produced in the public interest, which should be made openly available with as few restricDons as possible

•  Data with acknowledged long-‐term value should be preserved and remain accessible and usable for future research

•  To enable research data to be discoverable and effecDvely re-‐used by others, sufficient metadata should be recorded and made openly available to enable other researchers to understand the research and re-‐use potenDal of the data. Published results should always include informaDon on how to access the supporDng data.

•  It is appropriate to use public funds to support the management and sharing of publicly-‐funded research data.


5/21/12

5

NSF Mandate


Challenges for Data Sharing (Wellcome Trust)

•  Infrastructural: sustaining the infrastructure required for long-‐term data storage and curaDon

•  Cultural: incenDves and recogniDon for researchers whoshare their data

•  Technical: developing data standards, metadata, plaxorms needed for inter-‐operability

•  Professional: training and career development of data specialists and bioinformaDcians

•  Ethical: protecDng the confidenDality of research parDcipants


Data-‐sharing mandates

•  Will mean a tsunami of new digital works entering repositories (including a significant amount of moving image material)

•  The library preservaDon community is now highly engaged in planning for this – “Data CuraDon” appears to be the hoEest current topic in digital preservaDon

– Huge efforts are going into creaDng models for “data management plans”


Data Management Plans


Example of moving image research data


Example of video research data: NYU’s Infant AcDon Lab

hEp://psych.nyu.edu/adolph/

•  “we study behavioral flexibility-‐-‐how people learn to adapt to changes in their bodies and skills and to variaDons in the environment”

•  “We challenge infants, children, and adults with novel predicaments such as crawling over bridges, squeezing through apertures, brachiaDng over monkey bars, and reaching for targets with the body in moDon. We observe people's acDons using computerized video coding (openSHAPA) and state-‐of-‐the-‐art recording technologies including gait carpets, moDon tracking, head-‐mounted eye-‐tracking and remote eye-‐tracking.”


5/21/12

6

Infant AcDon Lab trying to examine






NYU’s Infant AcDon Lab (1 min)


Infant AcDon Lab coding


Reviewing audio


5/21/12

7

Tracing paths


OpenSHAPA Databrary hEps://openshapa.org/share/


OpenSHAPA Databrary Mission

•  “an open data-‐sharing project within the developmental science community. We will create a community of open collaboraDon and data sharing by removing professional barriers to sharing and by developing two enabling soZware tools”

•  “Databrary is a proposed web-‐based environment in which researchers can release raw video data, other data streams, and associated metadata under a creaDve commons license, browse others' data, and perform further analyses on others' data that can extend or even transform the community's understanding.”


Major requirements for video research data

•  Linking metadata to Dme-‐codes •  Tools for easy coding/mark-‐up/annotaDon

•  Tools for viewing and subsequent mark-‐up by other researchers


Mangold’s Interact commercial video coding product (1:15)


Coding tools need

•  to be able to examine audio and video channels (separately) with tools that will easily show changes (eg. Volume)

•  to be able to match to Dme-‐code, and segments of Dme-‐code may need to be referenced as events

•  to include annotaDons and transcripDons •  ability to add new coding at any point during the life-‐cycle of

the data

•  to perform analysis, like extracDng staDsDcs (eg. frequencies and duraDons)

•  someDmes to draw diagrams (path taken by baby)

•  export to other soZware


5/21/12

8

OpenSHAPA environment


OpenSHAPA Spreadsheet & video


Researcher-‐cited Challenges of Sharing Video Data and Associated Metadata •  Permission, access, and security •  Licensing •  AEribuDon/citaDon •  Data coding tools •  Types of metadata


Challenges video as data pose for us

•  Linking complex sets of metadata (whole spreadsheets or selected cells, annotaDons, sensor data, diagrams, etc.) to Dmecode periods, and being able to reference Dmecode periods as “events”

•  IdenDfying and grouping metadata by contributor (as data is re-‐used over its lifecycle, it will accrete metadata from various sources)

•  Privacy/security—anonymizing personal info in ways that can’t be reconstructed using other sources, and limiDng access to legiDmate researchers who have currently valid “protecDon of human subject” agreements (maintaining mulDple access levels, and tying those to aggreements that need to be periodically renewed)

•  IntegraDon and interacDon with open-‐source coding/annotaDon/viewing tools


Final QuesDon: Is sensor output really Audiovisual?

•  Yes & No

•  The data does not become a video unDl it’s Rendered

•  There are oZen can be many choices in how to Render – Speed (fps), though there may be direcDons

– Colors – Example-‐


Data as visual image (ImageQuery 1986)

48 Besser-‐Visual Archives, 14/6/11

5/21/12

9

SubducDon at Tonga-‐earthquake sounds visualizaDon hEp://www.youtube.com/watch?v=tV6TUDpXohM


NASA ExpediDon 28 ISS flyover


Final Comments (1/2)

•  People can use our collecDons for research, as long as they can find what they need

•  The metadata we provide will enable certain types of research (and make other types of research very difficult)

•  The more types of metadata we provide, the wider the research that will be done – Must consider automaDc image indexing, crowd-‐sourcing metadata, etc.


Final Comments (1/2)

•  In handling media born as data, we need to make sure that we can: –  link various media types to our moving image Dme-‐code –  provide privacy/security controls –  form ways that future researchers can re-‐use older research

–  integrate annotaDon and viewing tools –  find ways to group metadata by contributor, by event, etc. –  make sure that older data can be referenced in persistent ways


Audiovisual as Research Data

•  Howard Besser (2011). “Large datasets of visual daily culture: Why they're important; Who's studying them; and the Challenges for scholars, scholarship, pedagogy, and stewards” Texas InsDtute for Literary and Textual Studies, AusDn, March 12 hEp://besser.tsoa.nyu.edu/howard/Talks/11Dlts.pdf

•  hEp://sciencecommons.org/index.php

•  hEp://www.wellcome.ac.uk/About-‐us/Policy/Spotlight-‐issues/Data-‐sharing/

•  hEp://www.rcuk.ac.uk/research/Pages/DataPolicy.aspx

•  hEp://besser.tsoa.nyu.edu/howard/Talks/

•  hEp://www.nyu.edu/Dsch/preservaDon/


prestocentre-data - new york universitybesser.tsoa.nyu.edu/howard/talks/12prestocentre-data.pdf ·...

Documents