“storage is cheap” and other lies lance stuchell, university of michigan library curating and...
TRANSCRIPT
![Page 1: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/1.jpg)
““Storage is Cheap” and Other LiesStorage is Cheap” and Other Lies
Lance Stuchell, University of Michigan LibraryCurating and Managing Research Data for Re-UseICPSR, July 31, 2013
“Hard Drives” by Michael Muni CC-BY-NC-ND
![Page 2: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/2.jpg)
Overview
• Instability of storage hardware costs • Check out David Rosenthal’s blog at
http://blog.dshr.org/
• Costs of storing digitized moving images
• Effects of storage costs on digital preservation, formats, etc.
![Page 3: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/3.jpg)
“Storage is Cheap”
• Perception is based on “Kryder's Law”• A 2005 Scientific American article about Mark
Kryder, Seagate's Senior VP of Research • Magnetic disk density increases quickly• Disk density closely tied to pricing • 30-year history of disk prices dropping about
40% per year
• Disk costs were affordable & predictable• 30% of total storage costs
![Page 4: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/4.jpg)
The Party's Over
• Mid 2011: Kryder’s Law slows• Latest projections ~ 20% density growth
• Late 2011: Flooding in Taiwan causes shortfall of 70 million disk drives• Prices remain over 50% higher• Not expected to return to pre-flood levels
until 2014
• Changes in technology David S. H. Rosenthal, “Storage Will Be A Lot Less Free Than It Used To Be,” 2012.http://blog.dshr.org/2012/10/storage-will-be-lot-less-free-than-it.html
![Page 5: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/5.jpg)
“Optimistically, for the rest of this decade the rapid decrease in cost per bit of storage that
has been a constant of the last three decades will be much slower; it might even stop.”
David S. H. Rosenthal, et al., “The Economics of Long-Term Digital Storage” 2012.http://www.lockss.org/locksswp/wp-content/uploads/2012/09/unesco2012.pdf
![Page 6: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/6.jpg)
A/V Digitization at MLibrary
Digitization of Special Collections material
Audio:• Uncompressed BWF master files• 1 original audio object ≈ 5 GB storage
Moving Image:• Uncompressed preservation master• Compressed production master• 1 original video object ≈ 40.4 GB Storage
![Page 7: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/7.jpg)
A/V Storage Estimates
![Page 8: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/8.jpg)
A/V Storage EstimatesTB/Year
![Page 9: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/9.jpg)
Initial Costs Estimates
Value Storage: $250/TB per year
![Page 10: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/10.jpg)
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
![Page 11: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/11.jpg)
Initial Costs Estimates
Value Storage:Tape Backups:
$250/TB per year$1,825/TB per year
2018: $245,3252023: $485,200
2013: $32,450
![Page 12: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/12.jpg)
Reaction
![Page 13: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/13.jpg)
Current Costs Estimates
Value Storage:Tape Backups
(MLib):
$250/TB per yearEquipment costsCheaper most years
![Page 14: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/14.jpg)
Current Cost Estimates
![Page 15: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/15.jpg)
Current Cost Estimates
![Page 16: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/16.jpg)
Current Cost Estimates
![Page 17: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/17.jpg)
Total 11 Year Storage Cost
$569,368
![Page 18: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/18.jpg)
Current Cost Estimates
![Page 19: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/19.jpg)
Current Cost Estimates
![Page 20: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/20.jpg)
Looking Ahead
• Assuming steady pricing• Instability in per-bit storage
• Will not fall under 20% growth until 2018
• ITS storage costs may not fall at same rate
• Long-term costs of cloud are not known• Amazon costs haven't decreased at HD rate1
• Tape storage for preservation copies?
• Economies of scale• Digital Preservation Network
1Rebecca Pool, “Is cloud storage the answer to preservation?” Research Information, 2/14/2013
![Page 21: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/21.jpg)
Ramifications
• Preservation• Some moving image material is not
retained in uncompressed formats• Still image formats are balance of
preservation and size
• Appraisal and re-appraisal • What are we keeping and for how long?
• Costs of starting up
![Page 22: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/22.jpg)
Storage Takeaways
• HD costs are no longer predictable • Budgets often neglect storage
• Ignoring significant and ongoing costs
• Storage costs are community wide problem• Question of scale• Best practice may not be possible
• Backups can cost more than primary storage
• Archival storage ain’t cheap!
![Page 23: “Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard](https://reader036.vdocument.in/reader036/viewer/2022070408/56649e655503460f94b5fff7/html5/thumbnails/23.jpg)
Thanks!!