government big data success stories

CTOlabs.com

Government Big Data Success Stories

Bob Gourley http://ctolabs.com Jan 2012

CTOlabs.com

About This Presentation:

• Four areas of significant activity in Big Data solutions• Top Federal Mission Needs for Big Data• Five Exemplars of Big Data Solutions

2

CTOlabs.com

Most active fed solution areas: Federal integrators: Spending internal research and development

funds to create prototypes and full solutions relevant to fed missions

DoD and IC agencies: Using Big Data approaches to solve “needle in the haystack” and “connect the dots” problems

National Labs: Bioinformatics solutions have been put in place by federal researchers

OMB and GSA: Ensuring sharing of lessons and solutions. Key exemplars around web search methods. Solutions inside government agencies and on citizen facing properties

3

Big Data solutions are already making a difference in government service to citizens.

CTOlabs.com

Top Federal Mission Needs for Big Data Financial fraud detection across large, rapidly changing data sets

Cyber Security: rapid real time analysis of all relevant data

Rapid return of geospatial data based on query

Location based push of data: Focused on emergency response

Real time return of relevant search: USA.gov is exemplar

Real time suggestion of topics: USA.gov is exemplar

Real time suggestion of correlations: DoD has many use cases

Bioinformatics: Human Genome

Bioinformatics: Patient location, treatment, outcomes

4

These needs must be met in an era of significant downward pressure on budgets. Scalable systems with well thought out governance & extensive automation are key.

CTOlabs.com

Five Exemplars USA Search: Best in class hosted search services over more than 400

gov sites. Great use of CDH3.

GCE Federal: Cloud-based financial management solutions. Apache Hadoop, Hbase, Lucene for Dept of Labor.

PNNL Bioinformatics: Leading researcher Dr. Taylor of PNNL is advancing understanding of health, biology, genetics and computing using Apache Hadoop/MapReduce/HBase.

SherpaSurfing: Use of CDH as a cybersecurity solution. Ingest packet capture in any format, analyze trends, find malware, alert.

US Department of State: Bureau of Counselor Affairs. Large data with important applications for citizen service and national security.

5

Each of these are making a difference for government missions right now.

CTOlabs.com

Questions/Comments?

CTOlabs.com

This Presentation Prepared By:Bob GourleyCTOlabs.com

http://twitter.com/bobgourley

CTOlabs.com

Backup SlidesGSA

USA Search

CTOlabs.com 9

CTOlabs.com

USA Search

Program of General Services Administration’s (GSA) Office of Citizen Services and Information Technologies.

Hosted search services for USA.gov and over 500 other government websites.

Solves big data challenges with open source capabilities.

CDH3 since fall 2010. HDFS, Hadoop and Hive used in cost effective, resilient, scalable solution.

Search Results. Search Suggestions. Trend analysis. Analytic dashboards.

10

Bottom Line: USA Search brings the best of the open source community to multiple government missions, including direct citizen support

CTOlabs.com 11