big data symposium: analytics and applications for federal big data – bureau of justice statistics

14
Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ March 5-6, 2013 http://semanticommunity.info/Big_Data_Symposia 1

Upload: ghita

Post on 23-Feb-2016

40 views

Category:

Documents


0 download

DESCRIPTION

Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics. Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

1

Big Data Symposium:Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

Dr. Brand NiemannDirector and Senior Enterprise Architect – Data Scientist

Semantic Communityhttp://semanticommunity.info/

AOL Government Bloggerhttp://gov.aol.com/bloggers/brand-niemann/

March 5-6, 2013http://semanticommunity.info/Big_Data_Symposia

Page 2: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

2

Begin With the End in Mind• Ms. Jo Strang, Associate Administrator, Safety, Federal Railroad

Administration, Department of Transportation “Open Gov 2.0 and Safety.Data.Gov”– New safety data sources and challenge from the National Institute of

Justice and new data from Open FEMA• Open FEMA

– Could not find with Google search– Start with http://www.fema.gov/

• See http://semanticommunity.info/Big_Data_Symposia/FEMA.gov

• Safety.Data.Gov– Could not find at that Web site– Start with Bureau of Justice Statistics

• http://www.bjs.gov/

Page 3: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

3

My Process• Found New Releases and Female Victims of Sexual Violence, 1994-2010 at

BJS.gov• Built a Knowledge Base of the Web Site, Metadata, and Data Sources for:

– Press Release– PDF (1.4M)– ASCII file (34K)– Comma-delimited format (CSV) (Zip format 26K)– Help for using BJS products– About the Source Data: National Crime Victimization Survey (NCVS)

• Did Extensive Pre-Conditioning of the CSV Spreadsheets for Use and Display of the Data Sets

• Imported the Data Sets Into Spotfire and Created a Guided Analysis• Documented My Data Science Work in a Story and PowerPoint Slides• Provided Conclusions and Recommendations

Page 4: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

4

Bureau of Justice Statistics

http://bjs.ojp.usdoj.gov/index.cfm

Female Victims of Sexual Violence, 1994-2010

Page 5: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

5

Female Victims of Sexual Violence, 1994-2010

http://bjs.ojp.usdoj.gov/index.cfm?ty=pbdetail&iid=4594

Build a Knowledge Base:Press ReleaseReport (PDF and ASCII)HelpAbout the Source Data

Page 6: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

6

Knowledge Base*: MindTouch

http://semanticommunity.info/Big_Data_Symposia/BJS

*Well-defined URLs for everything:PDFTextCSVImagesSee next slide!

Page 7: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

7

Knowledge Base*: MindTouch

*Well-defined URLs for everything:PDFTextCSVImages

http://semanticommunity.info/Big_Data_Symposia/BJS#Report_PDF

Page 8: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

8

Knowledge Base in Spreadsheet: Excel

http://semanticommunity.info/@api/deki/files/23274/BJSFVSV19942010.xlsx

This is Linked Open Data! My 5 Steps to Getting to 5 Stars!

Page 9: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

9

Spreadsheet in Dashboard: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?BJSFVSV19942010-Spotfire

Readme.txt to Master Data Management and Unified Data Architecture :Figures: 3; Tables: 11; and Appendix Tables 16

Page 10: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

10

Spreadsheet in Dashboard: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?BJSFVSV19942010-Spotfire

From 1995 to 2005, the total rate of sexual violence committed against U.S. female residents age 12 or older declined 64% from a peak of 5.0 per 1,000 females in 1995 to 1.8 per 1,000 females in 2005. It then remained unchanged from 2005 to 2010.

Sexual violence against females includes completed, attempted, or threatened rape or sexual assault. In 2010, females nationwide experienced about 270,000 rape or sexual assault victimizations, compared to about 556,000 in 1995.

Page 11: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

11

Spreadsheet in Dashboard: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?BJSFVSV19942010-Spotfire

Males had lower rates of rape or sexual assault than females from1995 to 2010

Due to the relatively small number of sample cases, coupled with a low rate of victimization, estimates of male sexual violence from the NCVS cannot be used reliably for further disaggregation by victim and incident characteristics. Therefore, this report focuses exclusively on females.

Page 12: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

12

Spreadsheet in Dashboard: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?BJSFVSV19942010-Spotfire

The percentage of sexual violence reported to police increased to a high of 56% in 2003 before dropping to 35% in 2010, a level last seen in 1995

The percentage of victimizations known to police because they were reported by another household member declined from 26% in 1994-98 to 10% in 2005-10, while the percentage reported by an official other than the police increased from 4% to 14%.

Page 13: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

13

Spreadsheet in Dashboard: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?BJSFVSV19942010-Spotfire

All the tables were carefullyformatted in the spreadsheetfor display on Spotfire.This is what Data Science does!

Page 14: Big Data Symposium: Analytics and Applications for Federal Big Data – Bureau of Justice Statistics

14

Conclusions and Recommendations

• Built a Knowledge Base of the Web Site, Metadata, and Data Sources with Well-Defined URLs for Everything

• Did Extensive Pre-Conditioning of the 30 CSV Spreadsheets for Use and Display of the Data Sets

• Made the Reame.txt File a Master Data Management, a Unified Data Architecture, and Linked Open Data: My 5 Steps to Getting to 5 Stars

• Imported the Data Sets Into Spotfire and Created a Guided Analysis That Augments the Original Report

• Documented My Data Science Work in a Story and PowerPoint Slides• The Fairfax County Domestic Violence Fatality Review Team 2012

Annual Report is next