Content Working Group
2015 NDSA Web Archiving Survey Report HighlightsNicholas Taylor (@nullhandle)Web Archiving Service ManagerStanford University Libraries
Archives 2016: Web Archiving RoundtableAugust 3, 2016
Content Working Group
NDSA Web Archiving Survey Working Group
Jefferson BaileyInternet Archive / Archive-It
Edward McCainUniversity of Missouri
Abbey PotterLibrary of Congress
Abbie GrotkeLibrary of Congress
Christie MoffattNational Library of Medicine
Nicholas TaylorStanford University Libraries
Content Working Group
NDSA Web Archiving survey background2011• 78 respondents• program info• tools/services• access• policies
2013• 92 respondents• program info
• staff, metrics, skills, content concerns
• tools/services• access/discovery
• new options
• policies• embargo, social
media, robots.txt, resources
2015• 106 respondents• program info
• areas of progress, collaboration interests + barriers
• tools/services• replication targets
• access/discovery• researchers
• policies
Content Working Group
Respondent Characteristics
“Lego People” by Scoobay under CC BY-NC-SA 2.0
Content Working Group
increasing proportion of universities
Archive
Universi
ty
Commercial
Consortium
Fed Gov
Loca
l Gov
State Gov
Hist. S
ociety
K12 School
Museum
Public Li
braryOther
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
2011 2013 2015
Content Working Group
SAA WebArch RT tops group affiliations
group 2011 2013 2015
8% 7% 8%
31% 33% 39%
45% 50%
Content Working Group
growth in staffing at the middle
.25 FTE56%
.5 FTE25%
1 FTE
6%
1 to 3 FTE
7%
3+ FTE7%
2013
.25 FTE58%
.5 FTE13%
.75 FTE5%
1 FTE5%
1 to 3 FTE13%
3+ FTE6%
2015
Content Working Group
Maturity and Progress
“Apple Mouse Evolution” by raneko under CC BY 2.0
Content Working Group
shift from pilot to production (all responses)
Planning Pilot Production Discontinued0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
2011 2013 2015
Content Working Group
shift from pilot to production (longitudinal responses)
2011 2013 20150
2
4
6
8
10
12
14
16
18
Planning Pilot Production
Content Working Group
increased perceptions of progress
Significant progress
29%
Some progress43%
About the
same15%
Slightly worse
off2%
Much worse
off2%
Other10%
2013
Significant progress
49%
Some progress22%
About the same19%
Other10%
2015
Content Working Group
perceived progress on data capture, appraisal, vision
Vision + O
bjective
sPolic
y
Resources +
Workfl
ow
Risk M
anagement
Appraisal +
Selecti
on
Scoping
Data Capture
QA + Analysis
Storage + O
rganization
Preservation
Metadata/Descr
iption
Access/
Use/Reuse
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
40.00%
17.65%
34.12%
5.88%
45.88%
37.65%
58.82%
22.35%
10.59% 12.94%
24.71%
16.47%
Content Working Group
low perceived progress on access, metadata, QA
Vision + O
bjective
sPolic
y
Resources +
Workfl
ow
Risk M
anagement
Appraisal +
Selecti
on
Scoping
Data Capture
QA + Analysis
Storage + O
rganization
Preservation
Metadata/Descr
iption
Access/
Use/Reuse
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
15.85%
28.05%24.39% 24.39%
12.20%9.76%
12.20%
36.59%
21.95%
29.27%
37.80%
52.44%
Content Working Group
Archiving Focus
“Ant Farm Media Van v.08 (Time Capsule) in Bellewether at Southern Exposure” by Steve Rhodes under CC BY-NC-SA 2.0
Content Working Group
more archiving of own content
Own content 3rd-party content Both Other0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
2011 2013 2015
Content Working Group
access, cost, quality are top valued metrics
Access
+ use Cost
Data volume
Institutional b
uy-in
Loss
Quality
Risk M
anagementOther
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
2013 2015
Content Working Group
tools, appraisal, QA are top valued skills
Appraisal/s
election
Archivi
ng tools
Collaboration/co
mmunication
Domain experti
se
Metadata
Quality assu
rance
Software deve
lopment
Web tech
nologiesOther
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
2013 2015
Content Working Group
collaboration interest in QA, capture, metadata
Risk m
anagement
Capture optimization
Collecti
on development
APIs
Metadata standards QA
Tool deve
lopmentOther
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
40.91%
52.27%
31.82%27.27%
51.14%
64.77%
46.59%
3.41%
Content Working Group
“Photocopier” by Joriel "Joz" Jimenez under CC BY-NC-ND 2.0
Tools and Services
Content Working Group
more using multiple archiving solutions
Local External Both0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
25.40%
60.32%
14.29%
19.51%
63.41%
15.85%
4.81%
63.29%
30.38%
2011 2013 2015
Content Working Group
flat data transfer from service provider
Transfer data Do not transfer data0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
19.15%
80.85%
20.29%
79.71%
20.27%
79.73%
2011 2013 2015
Content Working Group
transferring data locally + to other services
Local External Both0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
58.82%
47.06%
5.88%
Content Working Group
trust provider + building locally for data transfer
Trust data ca
pture servi
ce provid
er
Building lo
cal in
frastr
ucture
No place to
store/m
aintain it
Not sure w
hat we’d do w
ith it
once w
e got itOther
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
2011 2013 2015
Content Working Group
Archiving Policies
“Handle With Care” by ServInt under CC BY-NC-ND 2.0
Content Working Group
more conditional handling of robots.txt
Always Never Sometimes/it depends Don't know0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
2011 2013 2015
Content Working Group
policies of other orgs key for policy-making
ARL Code of B
est Practi
ces
Oakland Arch
ive Polic
y
Secti
on 108 Study G
roup
Legal c
ounsel
Statutory
authority
Policies o
f other o
rgs
Previous N
DSA su
rveys
Other0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
2013 2015
Content Working Group
Landscape Summary
“Mt Baldy from Box Springs Mountain wi Theodolite” by signal mirror under CC BY 2.0
Content Working Group
overall picture• generally positive perceptions of progress• more moves to production• growing proportion of universities• growing proportion of focus on own content• staffing largely remains fractional w/ minor growth• majority use external service but hints of hybrid
approaches• more comfort w/ conditional policy approaches
Content Working Group
recurring themes: progress• data capture• highest perceived progress• 2nd highest collaboration interest
• appraisal• 2nd highest perceived progress• 2nd highest valued skill
Content Working Group
recurring themes: opportunities• quality assurance• highest collaboration interest• 3rd highest desired skill• 3rd lowest perceived progress
• access• lowest perceived progress• tied for highest valued metric
• metadata• 2nd lowest perceived progress• 3rd highest collaboration interest
Content Working Group
Nicholas Taylor@nullhandle
“Thank You” by vistamommy under CC BY 2.0