xxbiennialicisinnew orleans - databrary.org · best practices for sharing ... establish file naming...

XX"Biennial"ICIS"in"New Orleans

Hilton'New'Orleans'Riverside'Hotel'May'26628,'2016'

Program'Chairs:'Lisa'Oakes,'Vanessa'LoBue,'Marianella'Casasola'

For'addiGonal'details'(as'they'become'available):''hLp://www.isisweb.org/view/0/conferences.html'

Science,'music,'and'food'meet'in'NOLA''

Invited(speakers(include:(

Paul Bloom Helen Neville

Nim Tottenham

Jim Rehg Brian Scassellati

Laurie Santos and many more….

Linda Smith

Felix Warneken

Thursday, March 19, 15

Open Video Data Sharing

The Databrary Project

NSF BCS-1238599NICHD U01-HD-076595


Special promises of video

Captures richness & complexity of behavior in real timeWidely used in developmental, education, social, & behavioral scienceCheap & easy to collect Self-documentingHigh potential for reuse


Special challenges of video

Identifiable data needs special policiesFaces, voices, names, places, classroom

Technical issues of sharingLarge files to store/stream, formatsSearch for videos, inside videos

Cultural barriers to sharingWe haven’t done it, we don’t know how to do it, no incentives to do it


Policies


Ensure that participants' wishes about sharing their data are respected just as you’ve done for excerptsTreat Databrary data with the same high standard of care that you treat data collected in your own labTake responsibility for your students’ use of Databrary data just as you do for your own data

Do what you’ve always done!3 ethical principles


Databrary as a walled garden


Restricted access

Authorized researchersReal researcher/professor with “PI status” at a university or other institutionGoverned by an IRB (ethics board)Developmental psychologists, learning/education researchers, clinical researchers

Authorized affiliatesStudents/staff of authorized researchers


Why restrict access?

Extends the zone of trust beyond the PI’s lab to other PIs in other labs Developmental researchers are more comfortable sharing their data & asking participants to share with other researchers like themselvesParticipants comfortable sharing with other researchers


Access model

Unique from other sharing agreementsUser & contributor privileges to library

Research & non-research usesIRB approval not required by Databrary for authorization to access data

Requires institution to co-sign researcher’s agreement

They (not us) are responsible for youThursday, March 19, 15

It is OK to share identifiable data if participants agree!

Raw video filesExcerptsMetadata (birth date, test date, location, ethnicity, disability status, etc)Codes of behaviors


5 Databrary release levels

Did not ask Functions as private



Did not askPrivate

Only researchers on protocol



Did not askPrivate Shared

Only authorized researchers



Did not askPrivate Shared Shared + Excerpts

Authorized researchers can show excerpts for research & educational purposes)



Did not askPrivate Shared Shared + Excerpts Public

Available to everyone



Did not askPrivate Shared Shared + Excerpts Public


Permission to share vs consent to participate

Consent to participate is separate from permission to shareConsent obtained before sessionRelease to share obtained after session

Clear to participants what was recorded


Asking participant release to share

databrary.org/user-guide/getting-started/release-script/example-videos.html


http://databrary.org/user-guide/getting-started/release-script/example-videos.html




Just do it!

Asking to share does not obligate you to shareCan’t share without askingWe’ll store all the files, so long as participants are askedGoing forward, incorporate Databrary release into your protocol


Databrary garden

Get authorized so you can play in the garden


In the Databrary garden


Browse videos

Get inspired!

Learn about procedures

See displays

Learn about coding rules

Nose around

See what colleague is up to

Curious

Get ideas

In the Databrary gardennon-research uses


Decide whether a study is worth doing

Collect preliminary data for grant proposal

Replicate, expand, or review work based on viewing the procedure or codes

In the Databrary gardenpre-research uses


Share excerpts to promote your own work

Find excerpts for teaching

Practice codingShow excerpt in talk

Illustrate an idea

In the Databrary gardenteaching & didactic uses


Use Databrary as your lab serverShare videos with lab staff & collaboratorsUse Databrary to back up & preserve your dataComplete enrollment for grant progress report

In the Databrary gardendata management


Reuse data to ask new questions

Add new codes

Perform integrative analyses

Grow sample size

Include Ss from other populations

In the Databrary gardenreuse data for research


Browse videos Find illustrative excerpts for teaching

Share excerpts to promote your work

Get inspired

Share with lab students & collaborators

Use Databrary as lab serverLearn about procedures

Use Databrary to back up & preserve your data

See displays

Re-use data to ask new questions


Add new codes


Practice coding

And much more!

In the Databrary gardenso many wonderful things to do!


Curation? Upload as you go

Don’t wait to upload until study is doneAfter the fact curation is painfulWe will help, but it will hurt at a little

Upload as you go is easy curationUpload videos & tag with basic metadata at end of each sessionYou’ve got to put it somewhere anywayData management & backup

Databrary = your lab serverThursday, March 19, 15

Best practices for sharing

Upload as you go spreadsheet requests info useful for search

Child’s language(s), session languageData collection setting, locationDisability statusEven if you don’t use this info

You have this info at your fingertips at end of session

No work to enter it now, is work laterThursday, March 19, 15


Accessibility helps everyoneAvoid lab-specific acronyms & terms

ET CWBaby OHFirst Steps

VT14

SOM SE18MA

PedestalCLB

Emo Lights

SFR

18MA-NoCurve

HK emb



Accessibility helps everyoneAvoid lab-specific acronyms & termsUse plain EnglishEstablish file naming conventions

Databrary will keep you organizedYou will never lose a file


Curation? Upload as you go

Describe study design (conditions, groups, etc.)Enter (type/paste/import) session metadataUpload new video, associate it with sessionKeep track of which data were enteredExport previously entered data for



Browse videos

Get inspired


See displays


Find illustrative excerpts for teaching


Practice coding


Use Databrary as lab server




Browse videos

Get inspired


See displays






Practice coding



Browse videos

Get inspired


See displays




Practice coding






Browse videos

Get inspired


See displays




Practice coding




Re-use data to ask new questions


Add new codes


Pre-research uses

Browse videos, nose aroundSee what colleagues are up toSee how to do a procedureLearn how someone coded their videosSee displaysGet inspired

Browse videos

Get inspired


See displays


Nose around


5 AimsCreate community of researchers committed to sharing Create policies for sharing Enhance Datavyu video coding tool Provide data management tools Build Databrary repository for sharing & storing video


Aim 1: Community Building

Help at every stepMinimize transaction costsSolve pain pointsIncentives/disincentives matterThe customer is always right


Aim 1: Community BuildingData will follow the peopleGet people in first


Getting people in

Registration takes 2 minStaff handles authorization with institutions

http://databrary.org/register




Getting people in

Promote via professional societiesISIS 2014 Berlin, boothSRCD 2015 Philly, preconference & boothAERA 2015, boothCDS 2015 Columbus, preconference & boothISIS 2016 New Orleans, preconference & booth


Getting people in

Boilerplate language for data management plansCollaborating on grant submissions

Autism Intervention Trial (U.K.)IU CyberlearningNSF 15-523


http://databrary.org/access/policies/dmp-template.html

http://databrary.org/access/policies/dmp-template.html

Aim 2: Policies for sharing

Consent to shareBuilds on informed consent & video/photo releaseSeparate from consent, after the sessionTemplate language on website

Share only with authorized researchersInstitutional agreement to permit access, supervise use, co-signed by researcher


http://databrary.org/access/policies/release-template.html

http://databrary.org/access/policies/release-template.html

Researchers promise to:

Treat Databrary data ethicallyRespect participants' wishesAuthorize people carefully


http://databrary.org/access/responsibilities/investigators.html

http://databrary.org/access/responsibilities/investigators.html

Aim 3: Enhance Datavyu.org

Free, open source multi-platform video coding/tagging/annotation toolUser guideHelp forumWorkshops for new users


http://datavyu.org

http://datavyu.org

Aim 4: Data management

“After-the-fact”Conventional routeSharing awaits completion of studyRequires curation after study is completed or later

Numerous disincentives for researchersHigh cost in time, effort, $Reopening file drawer = high pain


Aim 4: Data management

“Upload-as-you-go”New routeSelf-curation, upload after each sessionSharing while study is in progress or with one button click

Only incentives for researchersDatabrary as lab server & backupData management built in


Upload-as-you-go design

Familiar spreadsheet for data entryEnter data as collected or from paperSupport standard session-level metadataExport metadata for further analysis, post-processing


Illustration

http://databrary.org/volume/8


https://nyu.databrary.org/volume/8




Session timeline

https://nyu.databrary.org/volume/9/slot/6113?segment=-&asset=9607&select=Thursday, March 19, 15

https://nyu.databrary.org/volume/9/slot/6113?segment=-&asset=9607&select=




Aim 5: Databrary


Databrary.org

Beta release early 2014, public release October 201471 authorized researchers, 35 contributors40 institutions from U.S., Canada, South America, and Europe


Databrary.org

5700 videos; 1600 hrs; 1100 participantsRaw data, excerpts, displays

2,200 other files


Representative NSF-funded projectsTamis-Lemonda (0721383)

https://databrary.org/volume/8LoBue (1247590)

https://databrary.org/volume/30Messinger (0808767, 1052736)

https://databrary.org/volume/15Karasik (1349044)

https://databrary.org/volume/11Barr (1023772)

https://databrary.org/volume/40Gilmore (1147440)

https://databrary.org/volume/23
















5 potential new avenuesExpand target community Create policies for wider video sharing Build web-based Datavyu Enhance Databrary Plan for sustainability


Avenue 1: Expand community

New groups of researchersEducation/teachers, computer science, cognitive/vision science, linguistics, anthropology, ethology/animal behavior

Regional workshopsPartnerships with professional societiesLinks to other repositories (imaging, etc)


Avenue 2: Create policies for wider video sharing

Permissions for classrooms & other groupsAuthorize teachers, cliniciansTarget valuable existing dataPlay leadership role in policies for dealing with identifiable data


Avenue 3: Web-based DatavyuEliminate video downloadsSimplify data management & collaborationEnable frame-by-frame coding/tagging, editing; hyperlinked codebooks Support multiple, synchronized video viewsSpatial coding


Avenue 3: Web-based DatavyuAllow group-sourced coding/taggingReduce hardware/operating system dependencies Support import/export with existing coding tools


Avenue 4: Enhance Databrary

Search, filtering, downloadsSearch inside videos by time-locked tagView multiple video/data streams

Non-video data streams (physiology, imaging)

User metricsEnable collaboration and dialogue


Avenue 5: Sustainability

Pilot and test modelsWhen/how to charge for accessInstitutional subscriptions


xxbiennialicisinnew orleans - databrary.org · best practices for sharing ... establish file naming...

Documents