are you ready for the era of big data and extreme information management?
DESCRIPTION
This is the first part of a 5-part series on the Information Challenges facing organizations. A white paper describing these challenges can be found here - http://pages2.aiim.org/CIPWebPage_InfoProWP.htmlTRANSCRIPT
The New Normal: Get Ready for the Era of Extreme Informa7on Management
John Mancini President, AIIM @jmancini77 DigitalLandfill.org
Giving Credit Where Credit is Due… I didn’t make up the term “Extreme Informa7on Management” I first heard it used by Gartner. But I like it a lot.
Big Data is not just “more data.” -‐-‐Thornton May
The Chessboard Fable
We are reaching the 2nd half of “Moore’s Chessboard,” drama7cally changing what informa7on means to our organiza7ons and how it must be managed.
aiim.org/futurehistory
Era
Years
Typical thing
managed
Best known company
Content mgmt focus
Mainframe
1960-‐1975
A batch trans
IBM
Microfilm
Mini
1975-‐1992
A dept process
Digital Equipment
Image Mgmt
PC
1992-‐2001
A document
Microso`
Document Mgmt
Internet
2001-‐2009
A web page
Content Mgmt
Social and Cloud
2010-‐2015
An interac7on
Social Business Systems
Systems of Record
Systems of Engagement
Considera*on Systems of Record Systems of Engagement
Focus Transac7ons Interac7ons
Governance Command & Control Collabora7on
Core Elements Facts & Commitments Ideas & Nuances
Value Single Source of Truth Discovery & Dialog
Standard Accurate & Complete Immediate & Accessible
Content Authored Communal
Primary Record Type Documents Conversa7ons
Searchability Easy Hard
Usability User is trained User “knows”
Accessibility Regulated & Contained Ad Hoc & Open
Reten7on Permanent Transient
Policy Focus Security (Protect Assets) Privacy (Protect Users)
Systems of Engagement • For the past decade, companies have been accumula5ng data
in what we call a system of record. Those who survive going forward will also have systems of engagement – h=p://www.aiim.org/futurehistory -‐-‐ start with evalua5ng how you can have a relevant conversa5on with each individual customer across all channels. And insuring you have the analy5cal capability and the data to support that analysis. That is where the linkage is between the system of record data to system of engagement. On the technology side, we believe the future of handling this volume lies in leveraging the capability of the cloud. • Yuchon Lee, Vice President, IBM
Our 5-‐Point Manifesto 1. Commit to the cloud. 2. Mobilize everything. 3. Make the business
social. 4. Digi7ze anything that
moves. 5. Prepare for
informa7on management on a massive scale.
According to IDC – Between now and 2020… • 44X growth in informa7on
• 75X growth in informa7on “containers”
BUT… • 1.4X growth in IT professionals
The drama7c changes in the consumer space provide a hint as to what is coming…
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Source = hnp://www.mbaonline.com/a-‐day-‐in-‐the-‐internet/
Prepare for extreme informa7on management.
We are moving from the Systems of Record era in which our focus was on high-‐value informa7on assets to the Systems of Engagement era in which volume and complexity and velocity are increasingly drama7cally.
Structured Informa7on i.e., “data”
HIGH DENSITY
Managed via tradi7onal BI and Data
Warehousing
Value of Informa7on per Unit to Organiza7on
Original concept – Freeform Dynamics
Systems of Record
1
Structured Informa7on i.e., “data”
Unstructured Informa7on i.e., “content”
Managed in ECM & ERM systems
HIGH Value/Byte
Managed via tradi7onal BI and Data
Warehousing
Currently unmanaged
Value of Informa7on per Unit to Organiza7on
Original concept – Freeform Dynamics
2
Systems of Record
1
Structured Informa7on i.e., “data”
Unstructured Informa7on i.e., “content”
Managed in ECM & ERM systems
Managed via tradi7onal BI and Data
Warehousing
Currently unmanaged
Value of Informa7on per Unit to Organiza7on
BIG DATA
Original concept – Freeform Dynamics
2
3
Social, images, audio, video, text, office apps, web traffic, print streams, email, documents
BIG CONTENT
Volume, Velocity, Variety, Complexity 2.5 quin7llion bytes/day
Systems of Record Systems of Engagement
1
Internet of things – e.g., climate data, transac7on records, phone
GPS data – intelligent, interconnected, and everywhere
4
LOW Value/Byte HIGH Value/Byte
0% 20% 40% 60% 80% 100%
Structured data
Unstructured data
5 Fully 4 3 2 1-‐Poorly Unsure
Considered overall, to what degree does your organiza7on exploit its informa7on assets for
analysis and decision making purposes?
Source: Online survey of Register readers, 122 respondents, first half of November 2011, Freeform Dynamics
The New Normal • Volume: Enterprises are awash with ever-‐growing data of all types, easily
amassing terabytes—even petabytes—of informa7on. – Turn 12 terabytes of Tweets created daily into improved product sen7ment analysis – Convert 350 billion meter readings per annum to bener predict power consump7on
• Velocity: Some7mes 2 minutes is too late. For 7me-‐sensi7ve processes such as catching fraud, big data must be used as it streams into your enterprise in order to maximize its value. – Scru7nize 5 million trade events per day to iden7fy poten7al fraud – Analyze 500 million call detail records per day in real-‐7me to predict customer churn
faster
• Variety: Big data is any type of data -‐ structured and unstructured data such as text, sensor data, audio, video, click streams, log files and more. New insights are found when analyzing these data types together. – Use 100’s of live video feeds from surveillance cameras to monitor points of interest – Take advantage of the 80% data growth in images, video and documents to improve
customer sa7sfac7on
Source = IBM
The New Normal
• The vast majority of the world’s informa7on is unstructured.
• Unstructured informa7on growing 15X faster than structured.
• Raw compu7ng power growing so fast that an off-‐the-‐shelf box approaching the compu7ng power of a super computer 5 years ago.
• “Democra7za7on” of informa7on access.
Source = Understanding Big Data: Analy5cs for Enterprise Class Hadoop and Streamng Data
Irra7onal thinking
• Get rid of as much as you can: – Li7ga7on risk – Compliance risk – Storage cost
High Value/Byte
• Save everything that you can: – Might need it “someday” – Poten7al aggregated value – Disposi7on uncertainty
Low Value/Byte
Welcome to the Era of Extreme Informa7on
AIIM.org/research
5 Things to Remember About Extreme Informa7on
#1: Most organiza7ons do not have the basics in place.
• 61% report “content chaos” re unstructured informa7on. • 80% have no extended search across mul7ple repositories. • For 70%, harder to find your own stuff than stuff on the web.
#2: No one except “the industry” cares about “structured” vs. “unstructured.”
61% would find it “very useful” to link structured and unstructured datasets.
#3: Manual records management is dead.
• 66% have an informa7on management strategy, but only 22 percent use it.
• 79% have an informa7on reten7on policy, but only 32 percent enforce it.
• 58% say that a single enterprise records management model underlying all content systems is their goal, yet only 9% have achieved this.
Source: AIIM, Process Revolu7on: Moving Your Business from Paper to PC to Tablet
#4: The future is in metadata.
• “The solu7on to the over abundance of informa7on is more informa7on.” – David Weinberger
• “Data that is seman7c means exactly the same thing to any system or person who uses it.” – David Siegel
The New Normal • The standardiza7on of data sets across industries, the
separa7on of data from its descrip7on, and the exposure of this informa7on in the cloud create enormous opportuni7es.
• We can now analyze problems that were previously undiges7ble do to the sheer scale of compu7ng power required to address them. – The cloud, HADOOP, and MapReduce driving division of vast data into small pieces and parsing compu7ng across large numbers of computers.
• We can now solve the metadata problem (i.e., there is none!) for vast landfills of unstructured informa7on – We can now use seman7c technology to apply metadata where it didn’t previously exist.
For example…
Individual Paper records • Copied and aggregated paper • Manual compliance and “reading rooms”
Individual Computerized records • Aggregated computerized records • Spot audits and online viewing via EDGAR
Separa7on of data from viewing • XBRL standards – values, tags, dic7onaries • Internal process standardiza7on
Industry and regulatory standardiza7on • Adop7on as GAAP • Mandated by 110 countries
Standardized data moves to the cloud • Automated compliance and availability • Big data analy7c opportuni7es
Source = Pull: The Power of the Seman7c Web
A Short History of Financial Repor7ng…
Mining social streams… for predic7ons about the next hit…
nextbigsound.com
Mining social streams…for the best food Dish7p.com
Mining social streams…for drug info Treato.com
Using text analy7cs to mo7vate voters
Big Data Process Applica7ons • Financial Services
– Fraud detec7on – 360° View of the Customer
• Transporta7on – Logis7cs op7miza7on – Traffic conges7on
• Health & Life Sciences – Epidemic early warning – ICU monitoring
• Telecommunica7ons – Geomapping / marke7ng – Network monitoring
• U7li7es – Weather analysis – Smart grid management
• Intelligence – System Log Analysis – Cybersecurity
• Retail – 360° View of the Customer – Real-‐7me promo7ons
• Law Enforcement – Mul7modal surveillance – Cyber security detec7on
Source = IBM
The combina7on of seman7cs and accessibility of data in the cloud is revolu7onary. Across industries Across geography
This revolu7on will once and for all require the elimina7on of paper and dictate the management of unstructured informa7on assets.
#5: We need T-‐Shaped people to address this “extreme informa7on” opportunity.
AIIM.org/Cer7fica7on
The emerging informa7on professional
• The vast majority of organiza5ons see the need to manage informa5on as an enterprise resource rather than in separate "silos," departments or systems, but they don't know how to begin to address the challenge, as it is so large...
• Professional roles focused on informa5on management will be different to that of established IT roles.
• An "informa5on professional" will not be one type of role or skill set, but will in fact have a number of specializa5ons. – Deb Logan and Regina Casonata, Gartner
Who are these people?
Inform
a7on
Professio
nals
Risk/Liability Focus
IT Legal professional
Records Manager
Digital Archivist
Value Focus
Business Process Owners
Business Analyst
Knowledge Manager
Informa7on/Data Scien7st
Governance Focus Ent Informa7on Manager
Info/Data Stewards
Ent Informa7on Architect
Social Focus Informa7on Curators
Community Managers
Most roles from Deb Logan and Regina Casonata, Gartner
AIIM.org/cer7fica7on
White paper here – hnp://pages2.aiim.org/CIPWebPage_InfoProWP.html Free prac*ce exam/assessment -‐-‐ hnp://www.AIIM.org/CIP-‐prac7ce-‐exam
AIIM.org/training
We need T-‐Shaped Professionals BROAD DEEP