examining the big data frontier

27
Frontier Examinin g the March 26, 2014 #gltrain

Upload: govloop

Post on 26-Jan-2015

110 views

Category:

Technology


0 download

DESCRIPTION

Brief report on the Examining the Big Data Frontier GovLoop Guide. Best Practices, Common Challenges and much more!

TRANSCRIPT

Page 1: Examining the Big Data Frontier

Frontier

Examining

the

March 26, 2014#gltrain

Page 2: Examining the Big Data Frontier

Welcome

Steve ResslerFounder & PresidentGovLoop

- Tweets: #gltrain- Slides, Video archive & other resources will be emailed to

you later this week.- Check out the GovLoop VIP programhttp

://www.govloop.com/TrainingVIPs- Be sure to fill out the evaluation to obtain your 3 CPE’s at

the end of the event.

Page 3: Examining the Big Data Frontier

The Data Revolution

“As we sit on the cusp of remarkable innovations, we must remember that this time, modern innovations are powered by data.”

Page 4: Examining the Big Data Frontier

Our Goals - GovLoop Research• Our research: empower and educate • Our basic formula:

– Survey of GovLoop Audience– Case study from state, local and federal government– Best practices, challenges– Cheat sheet

• Our mission: Help you do your job better

Page 5: Examining the Big Data Frontier

What You’ll Find in the Report • A local government spotlight

showing how the city of Louisville, KY., has leveraged data to improve services

• A federal government case study highlighting the Army’s Enterprise Management Decision Support program.

• Industry insights on the current big data landscape.

• 8 Strategies and best practices for smart big data adoption and analysis.

• GovLoop’s big data cheat sheet.

Page 6: Examining the Big Data Frontier

GovLoop Survey Data

Page 7: Examining the Big Data Frontier

Defining Big Data “Big Data refers to the massive amounts

of data that collect over time that are difficult to analyze and handle using common database

management tools.” – PC Magazine

The Method for an Integrated Knowledge Environment open-source project. The MIKE project argues that big data is not a function of the size of a data set but its complexity. Consequently, it is the high degree of permutations and interactions within a data set that defines big data.

The National Institute of Standards and Technology. NIST argues that big data is data which “exceed(s) the capacity or capability of current or conventional methods and systems.” In other words, the notion of “big” is relative to the current standard of computation.

Definitions from: The Big Data Conundrum: How to Define It?

Page 8: Examining the Big Data Frontier

A Working Definition

“Big data is a term describing the storage and analysis of large and or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.”

See full study: Undefined By Data: A Survey of Big Data Definitions

- Jonathan Stuart Ward and Adam Barker, School of Computer Science, University of St. Andrews, UK.

Page 9: Examining the Big Data Frontier

Do we care?

• Leveraging data in new ways to meet mission need.

• Unlocking new insights by synthesizing data across your department

• Collaborating and sharing resources

Page 10: Examining the Big Data Frontier

Convergence of Technology

You can’t talk about big data without thinking about cloud and mobile.

Page 11: Examining the Big Data Frontier

3 Big Data Case Studies

Page 12: Examining the Big Data Frontier

Social Welfare

• The oldest case is the Famine Early Warning Systems Network (FEWS NET) developed by U.S. Agency for International Development in 1986.

• The $25 million dollar program helps optimize the distribution of up to $1.5 billion dollars per year in USAID Food for Peace assistance.

Page 13: Examining the Big Data Frontier

Smarter Healthcare

• Predicting the likelihood of hospitalization or death within 90 days, the Patient Care Assessment System (PCAS) calculates the Care Assessment Needs (CAN) Score.

• This score allows the Veterans Health Administration (VHA) to focus care teams and proactively care for their patients.

• The system collects 120 unique elements for 5.25 million patients and is supported by an 80-terabyte corporate data warehouse.

Page 14: Examining the Big Data Frontier

Biometric Intelligence

• Since 2003, US armed forces have collected biometric information from non-US citizens in Iraq and Afghanistan.

• It identifies enemy combatants and permits access into controlled areas.

• The system, known as The Automated Biometric Identification System (ABIS), stores 4.4 million unique identities and has identified over 3,000 enemy combatants, added 190,000 identities to the watch list, and protected the welfare of the United States and its allies.

Page 15: Examining the Big Data Frontier

Big Data’s Challenges – From the Survey

• Data governance• Data locked in legacy systems• Unclear mission and goals• Lack of support • Lack of clarity on metrics

Page 16: Examining the Big Data Frontier

Big Data’s Core Challenge

How do we find the needle in the haystack…when the haystack keeps

getting exponentially bigger?

Page 17: Examining the Big Data Frontier

Our Big Data Best Practices

Page 18: Examining the Big Data Frontier

1: Executive Leadership

Page 19: Examining the Big Data Frontier

2: Business Before Treasure

Page 20: Examining the Big Data Frontier

3: Know Thyself: Define Use Cases

Page 21: Examining the Big Data Frontier

4: Leverage Existing Resources & Augment

Page 22: Examining the Big Data Frontier

5: Integrate Legacy Systems

Page 23: Examining the Big Data Frontier

6: Partner With Industry

Page 24: Examining the Big Data Frontier

7: Focus on Governance and Data Quality

Page 25: Examining the Big Data Frontier

8: Have a Deep Bench - Train

Page 26: Examining the Big Data Frontier

Where to begin?

• What problem are we trying to solve?• How do we engage the right people?• How do we break down silos? • What kinds of data do we need?• What do I need to be able to do with the data? • Who needs access and when? • Can I leverage existing technology my agency has? • What does success look like?

Page 27: Examining the Big Data Frontier

Before we wrap up…

Today, public service offers a unique

opportunity

Photo license: by Pa1nt, FlickR Creative Commons