Download - David Reinsel - Entering the Era of Big IT
Copyright IDC. Reproduction is forbidden unless authorized. All rights reserved.
David Reinsel
E ntering the Era of Big ITE ntering the Era of Big IT
Group Vice President Storage, Semiconductors, GR C Infrastructure , Pricing
Feb-11© IDC
Fueling the Digital Universe
Big IT
Important areas of opportunities on which to focus
AgendaAgenda
Cloud• Public/Private• Security/Privacy
Content • How Much?• What Kind?
Context • Metadata• Sensors
What is driving Big IT?
Where does it all come from?
Feb-11© IDC
The Long View - ITThe Long View - IT
-10%
-5%
0%
5%
10%
15%
1996 1998 2000 2002 2004 2006 2008 2010 2012e 2014e
Source : IDC Worldwide Black Book, Q2 2010; growth in constant currency
Worldwide IT Spending Growth 1996-2014 (%)
Feb-11© IDC
Digital Universe Growth vs Storage Capacity ShipmentsDigital Universe Growth vs Storage Capacity Shipments
0EB
100EB
200EB
300EB
400EB
500EB
600EB
700EB
800EB
900EB
2005 2006 2007 2008 2009
Source: IDC Digital Universe Study, sponsored by EMC, 2010
WW Content Creation Vs . Storage Shipments
MemoryNV FlashTapeO pticalHard Disk DrivesWW Content Creation
Feb-11© IDC
Fueling the Digital UniverseFueling the Digital Universe
MediaCamerasDataMusicVoice CaptureOther
5
56.7%31.5%
7.2% 2.8% 1.5%0.4%
2009 0.8 ZB
Source: IDC Digital Universe Study, sponsored by EMC, May 2010
Feb-11© IDC
Digital Universe Growth –Long-term Content CreationDigital Universe Growth –Long-term Content Creation
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
2005 2010 2015 2020
WW Content Creation
Source: IDC Digital Universe Study, sponsored by EMC, 2010
35 Zetabytes
Feb-11© IDC
Data-Intensive Social NetworksData-Intensive Social Networks
7
400M ‘ ’ users60M status updates/day10B comments/mo20M videos uploaded/mo3B photos uploaded/mo
24 hrs of video uploaded/min
55M tweets/day
70M members1 new member every sec
Assumption Annual Footprint
active 10M B /user 4,000TB
0.25K B /update 5TB /yr
0.125K B /update 15TB /yr
8M B /v ideo 1,920TB /yr
100K B /photo 3,600TB /yr
12.5M B /video 3,787TB /yr
3M photo uploads/day
125Bytes/tweet 2.5TB /yr
1M B /photo 1,096TB /yr
0.1M B /profile 7TB
0.1M B /profile 3TB /yr
~14,436 Terabytes of Content
Creation in 2010
Feb-11© IDC
Next Agenda ItemNext Agenda Item
Cloud• Why?• How Big?
Content • How Much?• What Kind?
Context • Metadata• Sensors
Where does it all come from?
What is Driving Big IT?
Feb-11© IDC
Consumption is a Big Driver of BIGConsumption is a Big Driver of BIG
9
Consumption• Drives Traffic (ad revenue)
• Drives eCommerce• Drives Social Networks
• Drives Mobility (devices and apps)
•Drives Personal Services•Demands Analytics
Feb-11© IDC
Consumption VS CreationConsumption VS Creation
10
10M B /user 4,000TB
0.25K B /update 5TB /yr
0.125K B /update 15TB /yr
8M B /v ideo 1,920TB /yr
100K B /photo 3,600TB /yr
12.5M B /video 3,787TB /yr
125Bytes/tweet 2.4TB /yr
1M B /photo 1,096TB /yr
0.1M B /profile 7TB
0.1M B /profile 3TB /yr
624PB/yr 260B pg . views/mo
192PB/yr 2B vid . views/mo
9,131PB/yr 2B vid . v iews/day
0.5PB/yr 4.4B pg . views/mo
56PB/yr 30.4M views/day
0.2PB/yr 1.9B pg . v iews/mo
Feb-11© IDC
Finding Answers where there are yet to be Questions…
From Science Projects to Social Networks to Smart Technology
MOREAPPLICATIONS
MOREON-DEMAND
ACCESS
MOREDEVICES
MORECONTENT
Big Data is the Created Content nor is it even its Consumption – It’s the analysis of all the data surrounding or swirling around it
Big Data is the Created Content nor is it even its Consumption – It’s the analysis of all the data surrounding or swirling around it
notnot
Feb-11© IDC
Consumption is a Big Driver of BIG!Consumption is a Big Driver of BIG!
12
Cloud • Virtualization• Convergence
Mobility • Context• Commerce
Smart • Sensors• Decisions
Feb-11© IDC
What’s So Appealing About CloudWhat’s So Appealing About Cloud
13
Source: IDC’s IT Cloud Services Survey, Q2 2010 (# 225021)
N = 219
Feb-11© IDC
Netflix and the CloudNetflix and the Cloud
14
Feb ’09 – 10 Million Users (2nd
Datacenter?)
Feb ’07 – 1 Billionth DVD
Late ’08 – 1 Data Center
May ’08 – 1st Set-top box to stream
Feb-11© IDC
Netflix and the CloudNetflix and the Cloud
15
Feb ’09 – 10 Million Users (2nd
Datacenter?)
Feb ’07 – 1 Billionth DVD
Apr ’09 – 2 Billionth DVD
Late ’08 – 1 Data Center
May ’08 – 1st Set-top box to stream
Late ’10 –
-Anonymous
Amazon AWS
“What I need is an exact list of specific unknown problems we might encounter”
Feb-11© IDC
Netflix and the CloudNetflix and the Cloud
16
1. We needed to re-architect, which allowed us to question everything, including whether to keep building out our own data center solution.
2. Letting Amazon focus on data center infrastructure allows our engineers to focus on building and improving our business.
3. We’re not very good at predicting customer growth or device engagement.
4. We think cloud computing is the future.Driving Netlix’s Decision
Feb-11© IDC
Consuming the Right Content at the Right TimeConsuming the Right Content at the Right Time
17
Cloud• Virtualization• Convergence
Mobility • Context• Commerce
Smart• Sensors• Decisions
Feb-11© IDC
Devices Communicating at any given time (2012)– excluding enterprise datacenters
Devices Communicating at any given time (2012)– excluding enterprise datacenters
Mobile Devices (4.2B)
Computers (1.9B)
Entertainment (1.3B)
Home Networking (1.0B)
Toys/Appliances (0.8B)
VoIP (0.7B)
Industrial/Auto (0.2B)
Over 10 bi l l i on devi ces connected and sendi ng/r equest i ng dat a
Feb-11© IDC
Fueling the Digital Universe (2020)Fueling the Digital Universe (2020)
MediaCamerasDataMusicVoice CaptureOther
19
48.8%
30.8%
20.1%
0.0% 0.1%0.2%
2020 35 ZB
Source: IDC Digital Universe Study, sponsored by EMC, May 2010
Feb-11© IDC
Digital Universe Growth –Content vs # of FilesDigital Universe Growth –Content vs # of Files
1
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
1,000,000,000
10,000,000,000
100,000,000,000
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
WW Content CreationWW Files
Source: IDC Digital Universe Study, sponsored by EMC, 2010
# of F iles (M)
200 Trillion in 200915 Q uintillion in 2020!
35 Zetabytes
Feb-11© IDC 21
Feb-11© IDC
Capturing Real-time Value in a Connected WorldCapturing Real-time Value in a Connected World
Soft drink system that allows consumers over 100 different product variations
“Made to order” drinks
Sensors report machine state
Data instantly reported to prod development and marketing (real time research)
Analytics applied to predict demand profiles
Feb-11© IDC
LinkedIn Swarm Analysis – Real Time Visual AnalyticsLinkedIn Swarm Analysis – Real Time Visual Analytics
23
LinkedIn Swarm visualizes LinkedIn’s most recent company and titles searches, jobs posted, blog entries and shared as a moving “tag cloud”, or rather a “tag swarm” and it cycles through a variety of topics automatically.
Source: http://blog.linkedin.com/2011/01/25/linkedin-swarm/
Feb-11© IDC
Consuming the Right Content at the Right time - CONTEXTConsuming the Right Content at the Right time - CONTEXT
24
Cloud •Virtualization•Convergence
Mobility•Context•Commerce
Smart •Sensors•Decisions
Feb-11© IDC
Smart in the Wake of TragedySmart in the Wake of Tragedy
25
Aug 1, 2007, I-35W Mississippi river bridge collapsed during rush hour.
5th busiest bridge in the state, carrying 140,000 vehicles daily.
13 people killed and 145 injured.
The new bridge opened September 19, 2008 at a cost of over $200M
Feb-11© IDC
Smart TechnologySmart Technology
26
323 sensors with nearly 500 channels of information flow possible.Data file sizes which are being stored each day:Dynamic data = ~500 MB/day -- SOFO Static = ~7 MB/day -- VW Static = ~0.5 MB/day These amounts are based on hourly static readings, and dynamic readings at 100 Hz for accelerometers and 4 Hz for the LPs and strain gages.
Feb-11© IDC
Last Agenda ItemLast Agenda Item
Cloud
• Public/Private
• Security/Privacy
Content
• How Much?• What Kind?
Context
• Metadata• Sensors
Where does it all come from?
What is Driving Big IT?
Feb-11© IDC
There is a real cost to store all this Data – MORE EFFICIENCY is neededThere is a real cost to store all this Data – MORE EFFICIENCY is needed
Source: IDC September, 2010 - Doc #225016, “A Plateau in Sight for the Rising Costs Power and Cool the World's External Storage?”
$0M$500M$1,000M$1,500M$2,000M$2,500M$3,000M$3,500M$4,000M$4,500M$5,000M
0PB20,000PB40,000PB60,000PB80,000PB
100,000PB120,000PB140,000PB160,000PB180,000PB
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
e20
11e
2012
e20
13e
2014
e
Installed base of Ext. Storage WW Cost to Power and Cool
$60.79/yr ??
$1.7M = The av g . cost to acquire 1PB of enterprise external storage in 2010
$130K = The cost to power and cool 1PB of external storage in 2010
Installed PBs vs . Cost to Power and Cool(C o n te x t = W W E n terpris e E x tern a l S tora g e)
Feb-11© IDC
0%
10%
20%
30%
40%
50%
20102020
LockdownConfidentialCustodialCompliancePrivacy (email address on a Youtube upload)
(emails that might be discoverable in litigation)
(account information – could lead to ID theft)
(trade secrets, customer lists, secret memos)
(financial transactions, medical, military)
The Need for Information SecurityThe Need for Information SecurityPercentage of the Digital UniversePercentage of the Digital Universe
Source: IDC Digital Universe Study, sponsored by EMC, May 2010
Feb-11© IDC
Embedded 25+ billion
Other CE 2.0 billion
Phones 2.6 billion
PCs 1.9 billion
Billions of Interactions….Billions of SensorsBillions of Interactions….Billions of Sensors
Billions of C onnected Devices , 2020
200B Big T hings
1-5 x People
<1 x People
< .5 x People
TAM
Feb-11© IDC
• enabling massive connectivity• grows at a 40% CAGR to 35ZB in 2020 (Meta data
fastest growing segment – massive management of files)• aware is critical to enabling consumption of the right data
at the right time. (Meta data will be critical in defining context and optimizing commerce opportunity)
• A drive for more • The dire need for • Big Data apps must be • Ability to manage, analyze, and make decisions on data that is
captured via , devices, and machines – but must be smart about it.
Summary and ConclusionSummary and Conclusion
One of the top dynamics driving Big IT:
Opportunities upon which to focus:
CONSUMPTIONCloudContent
Context
efficiencysecurity and privacy
cloud-friendly
BILLIO NS of sensors