shepherd - university of virginia · mind that people can learn a lot by hearing what went wrong...

48
KEY FACTORS IN KEY FACTORS IN THE FUTURE OF THE FUTURE OF COMPUTING COMPUTING in the Government Enterprise Lewis Shepherd Chief Technology Officer Microsoft Institute for Advanced Technology in Governments

Upload: others

Post on 22-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

KEY FACTORS IN KEY FACTORS IN THE FUTURE OF THE FUTURE OF COMPUTINGCOMPUTINGin the Government Enterprise

Lewis ShepherdChief Technology Officer

Microsoft Institute for Advanced Technology in Governments

Page 2: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

““What technologies What technologies will be required will be required

over the next 10 years over the next 10 years to protect U.S. interests?to protect U.S. interests?””

An Exercise in Prediction, with theAn Exercise in Prediction, with theIntelligence Community as an ExampleIntelligence Community as an Example

What if we had asked that question, What if we had asked that question, 10 years ago10 years ago? ?

Page 3: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

““Asymmetric adversaryAsymmetric adversary”” = an information challenge (= an information challenge (““hard targethard target””))

Seeming irrelevance of traditional methods for new targetsSeeming irrelevance of traditional methods for new targets-- Order of battle (counting military elements)Order of battle (counting military elements)-- StateState--toto--state analysisstate analysis-- ““KremlinologicalKremlinological”” approaches approaches

Challenges of IT during wartimeChallenges of IT during wartime-- Stress on systems infrastructure of 2 wars Stress on systems infrastructure of 2 wars -- Stress on software (linkStress on software (link--analysis, SNA, analysis, SNA, ““searchsearch””))-- Stress on collection capacity (sensor grids, Internet)Stress on collection capacity (sensor grids, Internet)-- Stress on analystsStress on analysts’’ –– and technologistsand technologists’’ –– imaginationimagination

Some Surprises PostSome Surprises Post--9/119/11

Page 4: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Limits of Limits of ““SearchSearch”” for Prediction for Prediction

We don’t have a “Search”capability to reach inside enemy minds … yet

We donWe don’’t have a t have a ““SearchSearch””capability to reach inside capability to reach inside enemy minds enemy minds …… yetyet

Page 5: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

IT Challenge: LowIT Challenge: Low--Observable AdversaryObservable Adversary

Our databases had no fields for box-cutters, IM accountsOur databases had no fields for box-cutters, IM accounts

How does THIS … … help perform analysis on THIS?How does THIS … … help perform analysis on THIS?

Page 6: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Case Study: Intelligence CommunityCase Study: Intelligence Community

The IC’s post-9/11 challenge

Some identified solutions:

1. Grid/Cloud computing

2. Secure SOA platform

3. Web 2.0 tools

(Intellipedia, A-Space)

Implementation challenges

PD1

Page 7: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Slide 6

PD1 As with the rest of this template, this is a suggested agenda that may or may not fit your situation. Please feel free to make changes as you see fit, including adding and changing pages. In general, the idea is to educate folks on the technology challenge you faced, how you addressed it, and the business benefits the project ultimately delivered - or failed to deliver, as the case may be. Keep in mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. Paul Desmond, 10/3/2007

Page 8: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

What Drives the Future of Enterprise ComputingWhat Drives the Future of Enterprise Computing

The value for a new user of a service depends on the number of existing users of the service…

“Critical mass” can lead to “Bandwagon effect”…

Page 9: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

SideSide--Effects of Network EffectEffects of Network Effect

Exponential growth of networks, systemsRequires Scale

Exposes networks to “edge audiences”Requires Security

Derives new wisdom from growing “crowd”Makes Smart Systems

Page 10: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Scale

Page 11: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Scale: a Challenge for Large Commercial EnterprisesScale: a Challenge for Large Commercial Enterprises

“GOVERNMENT NO EXCEPTION”

Page 12: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Remote Office IT Remote Office IT ScenariosScenarios

Page 13: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

No InfrastructureNo Infrastructure

Page 14: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Microsoft Inc. as an Enterprise ExampleMicrosoft Inc. as an Enterprise Example

435 million unique users435 million unique users

6 billion instant 6 billion instant messages (IMs) per daymessages (IMs) per day

280 billion page views per 280 billion page views per dayday

29 billion E29 billion E--mails sent mails sent per day per day

141,000 end users141,000 end users260,000 computers260,000 computers550 Buildings in 98 550 Buildings in 98 countries countries 358,000 SharePoint sites358,000 SharePoint sites2,500 internal applications 2,500 internal applications 2,500,000 internal E2,500,000 internal E--mails mails per dayper day136,000 E136,000 E--mail Server mail Server accounts accounts 1,000,000 remote 1,000,000 remote connections per monthconnections per month

Page 15: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Defense Intelligence Agency Defense Intelligence Agency as an Enterprise Exampleas an Enterprise Example

One of 16 agencies in the Intelligence Community

9,000+ personnel

DIA IT systems support the entire intelligence community

100,000+ users of DIA’s Top Secret network, apps, data

Global reach through IT support of all DoD Commands

Pacific Command, European Command, etc.

The only true “all-source” agency in the IC

Collection (signals intell, human intell,

measurements & signatures, etc)

Page 16: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

The Challenge: Stovepiped Analytic CapabilitiesThe Challenge: Stovepiped Analytic Capabilities

Page 17: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Security

Page 18: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

The Security Side of The Security Side of ““Enterprise 2.0Enterprise 2.0””

Page 19: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Secret to a Walled Garden: ControlSecret to a Walled Garden: Control

Definition: On the Internet, a walled garden is an environment that controls the user's access

to Web content and services. In effect, the walled garden directs the user's navigation within particular areas, to allow access to a

selection of material, or prevent access to other material. [SearchSecurity.com]

Page 20: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Why WalledWhy Walled--Garden Content & Systems?Garden Content & Systems?

Rationale on the Internet: MoneyPaid-Access Content RevenueMember-Fee RevenueExclusive Ad Revenue (Segmented Eyeballs)Value of Intellectual Property

“Enterprise” Rationale: SecurityTrade Secrets in Operational DataCompetitive AdvantagesRegulatory Control over Data

“GOVERNMENT NO EXCEPTION”

Page 21: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Smart Systems

Page 22: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Need for Analytic ReformNeed for Analytic Reform

Traditional IC output: ~50,000 stand-alone reports/yearMany redundanciesProduced in agency/organization silosLack of collaborative capabilities across (and within) agencies“Intelink” (the IC-wide shared domain) seen as a backlot

Forcing Function: 9/11 Commission ReportKey Recommendation: From Need-to-Know to Need-to-Share!

Realization: “Something that’s 80 percent accurate, on-time, and sharable, is better than something that is perfectly formatted, but too much, too late, and over-classified.”

Chris Rasmussen, NGA

Page 23: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Birth Pangs of IC Web 2.0: 2004Birth Pangs of IC Web 2.0: 2004--20052005

Early Efforts were internal, agency-specific projectsCIA’s internal blogs, 2004DIA’s internal “IntelliPedia” wiki, 2004NGA’s internal blogs, early 2005DIA’s AJAX mashups in “Lab X,” 2004-05CIA’s del.ici.ous lookalike, Tag/Connect, 2005

A “Wisdom of Crowds” Culture was forming by 2005Joint trips to outside conferencesCross-agency collaboration on metadata taggingFormation of “IC Enterprise Services” group, or ICES

Tipping Points, sparked by ICES: August 2005 launch of “Intelink Blogs”April 2006 launch of IC-wide Intellipedia

PD3

Page 24: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Slide 22

PD3 As with the rest of this template, this is a suggested agenda that may or may not fit your situation. Please feel free to make changes as you see fit, including adding and changing pages. In general, the idea is to educate folks on the technology challenge you faced, how you addressed it, and the business benefits the project ultimately delivered - or failed to deliver, as the case may be. Keep in mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. Paul Desmond, 10/3/2007

Page 25: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

One thing we learned wikiOne thing we learned wiki--wikiwiki……

Page 26: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Key Distinctions, Intellipedia vs WikipediaKey Distinctions, Intellipedia vs Wikipedia

Business Practices of intelligence analysis & reporting demanded certain technical features:

Not open to the public, only users with access to the IC’s Top Secret network (JWICS), accounts created by ICES.

No anonymity. All edits and additions are traceable.

Intellipedia does not enforce a “neutral point of view”Actually intended to represent various points of view; viewpoints

are attributed to the agencies, offices, and individuals participatingConsensus may or may not emerge!

Page 27: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

IntellipediaIntellipedia’’s Hockeys Hockey--Stick GrowthStick Growth

Page 28: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

The TopThe Top--Secret Wiki Gets ClonedSecret Wiki Gets Cloned

Summer 2007, ICES introduced 2 new Intellipedia versions:• One on the SECRET network “SIPRNET”• One on a “Sensitive But Unclassified” network “DNI(U)” (a

protected trunk apart from the regular Internet)

Rationale:• Many military intelligence analysts (and most soldiers) only

have access to SIPRNET• Many DHS personnel and Law Enforcement have no

clearances whatsoever for classified information• Many IC personnel like to work at home on research and

topical news items

Page 29: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint
Page 30: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Walled Gardens Within Walled Gardens: Walled Gardens Within Walled Gardens: Relative Value of Classified InformationRelative Value of Classified Information

Relative Number of Users,Also Relative Volume of Data

Relative Growth in Intellipedia Pages

Page 31: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Anticipate a Network Effect for DNI(U)?Anticipate a Network Effect for DNI(U)?

Expect increasing rates of growth for DNI(U) usage and information sharingImproved realtime Internet data-miningAwareness of value of collaboration outside traditional IC boundaries (DHS, LE, foreign partners)Improved Web 2.0 tools deployed on DNI(U) to mirror those on JWICS and the Internet

Page 32: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Intellipedia Totals on All Three NetworksIntellipedia Totals on All Three Networks

64,782 users2.3 million edits

Page 33: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Bottom Line: Knowledge Work is Universal Bottom Line: Knowledge Work is Universal

Page 34: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

New IC Focus: New IC Focus: ““Analytic TransformationAnalytic Transformation””

Launched by ODNI, April 2007Both “analysis” side and “techie” sideDDNI/A and DNI CIO are the two project owners

Several key programs:Community-wide “IC Data Layer” to aggregate access to “all” databases (no one knows the true number)A-Space, a16-agency “collaborative environment for analysis”

DNI assigned job of ICDL and A-Space to DIA on behalf of full IC - because of our SOA work

Page 35: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

DIADIA’’s Alien: s Alien: AlAlll--Source Source IIntelligence ntelligence EnEnvironmentvironment

SOA Planning Begun 2005SOA Planning Begun 2005: Full web: Full web--services frameworkservices framework

Alien is a framework, not a single toolAlien is a framework, not a single tool

Reliant on globally networked set of data centersReliant on globally networked set of data centers

New bestNew best--ofof--breed analytic software breed analytic software

Alien data services Alien data services –– tying data togethertying data together

Message traffic and other text sourcesMessage traffic and other text sources

Traditional singleTraditional single--INT databasesINT databases

Integrated security architecture Integrated security architecture for single signfor single sign--onon

Page 36: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Alien allows tools to exploit semanticallyAlien allows tools to exploit semantically--enhanced dataenhanced data

34

METS: Metadata Extraction & Tagging Service“Black-box Tagging Factory” combines 13 separate

best-of-breed entity-identifiers, natural-language processors, disambiguators, tagging engines.

Page 37: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Key Desired Features of AKey Desired Features of A--SpaceSpace

Wikis, blogs, social networking, personalized RSS feeds, collaborative cloud-based word processing, mash-ups, and content tagging…

… all built atop an underlying SOA.

Page 38: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

AA--Space: think Space: think ““iGoogle,iGoogle,”” ““Live SpacesLive Spaces””

Page 39: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

37

Metrics (a key postMetrics (a key post--9/11 recommendation)9/11 recommendation)

Page 40: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

AA--Space Pilot Schedule: Bridge Too Far?Space Pilot Schedule: Bridge Too Far?

Pilot Awarded Sep 14, 2007Pilot Development and Integration Sep 14-Nov 23, 2007Pilot Development Freeze Nov 23, 2007 Integration Testing and IPAT Nov 26-30, 2007Functional Testing (Approved Users) Dec 3-7, 2007 Final Clean Up Dec 10-12, 2007 C&A DIA* Dec 13-14, 2007 C&A DNI* Dec 17-19, 2007Installation at DIA’s main Data Center Dec 20-28, 2007Prototype available to IC users Dec 31, 2007

Page 41: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Time overrunsBudget overrunsIncomplete featuresIncomplete functions

Cancelled prior to completionAbandoned

Source: CIO Executive Board research; Standish Group 2004 CHAOS Report

On timeOn budgetDesired featuresDesired functions

Average IT Project SuccessAverage IT Project Success

Lesson: Many Enterprise IT Projects Fall Short of Expectations

“GOVERNMENT NO EXCEPTION”

Page 42: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Other Government Examples: epa.wik.isOther Government Examples: epa.wik.is

http://epa.wik.is/

Page 43: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

41

epa.wik.is goes mashup bigtimeepa.wik.is goes mashup bigtime

Extensibility: Integration with Yahoo!, Windows Live, Google, Flickr, WidgetBox, YouTube, and much more.

“Data reuse in mashups will revolutionize EPA data architecture, data management, and data reuse applications!”

EPA Architect Brand Niemann

Page 44: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

NearNear--Future IT Enablers for the ICFuture IT Enablers for the IC

Semantic Web - Global all-source system enabling rich ontological information management

autonomously and presumptively alerting analystsautomatically populating knowledge basescueing other military and IT systems

GIGINT - ability to mine and control the Global Information Grid without human intervention, including the billions of sensor/ RFID/nano/autonomous devices communicating with the Grid.

Gartner: By 2013, more than 200 billion processors will be in daily use around the world

Semantic Web Semantic Web -- Global allGlobal all--source system enabling rich source system enabling rich ontological information managementontological information management

autonomously and presumptively alerting analystsautonomously and presumptively alerting analysts

automatically populating knowledge basesautomatically populating knowledge bases

cueing other military and IT systems cueing other military and IT systems

GIGINT GIGINT -- ability to mine and control the Global Information Grid ability to mine and control the Global Information Grid without human intervention, including the billions of sensor/ without human intervention, including the billions of sensor/ RFID/nano/autonomous devices communicating with the Grid. RFID/nano/autonomous devices communicating with the Grid.

Gartner: By 2013, more than 200 billion processors will be in Gartner: By 2013, more than 200 billion processors will be in daily use around the worlddaily use around the world

Page 45: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

43

Virtual WorldsVirtual Worlds

New methods of modeling, simulation, and collaboration are being created for analysts and collectors

“Knowledge Walls” and Crisis Centers can be built more cheaply in a Virtual World, still using real-time feeds

Page 46: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

1.1. SOA environments driven entirely by business processesSOA environments driven entirely by business processes

2.2. CrossCross--Domain capabilities as embedded, intuitive servicesDomain capabilities as embedded, intuitive services

3.3. Rapid increases in speed/volume of sensor and analytic feedsRapid increases in speed/volume of sensor and analytic feeds

4.4. Stateless devices (the ultimate thin client Stateless devices (the ultimate thin client ““computercomputer””))

5.5. Wideband agile human interfaces, and true video teleWideband agile human interfaces, and true video tele--presencepresence

6.6. The The far edges of technological support for analysisfar edges of technological support for analysis::

Support to prediction;Support to prediction;

Crisis uncertainty management; Crisis uncertainty management;

Dynamic retasking of machines by machines...Dynamic retasking of machines by machines...

Research UnderwayResearch Underwayfor Future Enterprise Effectivenessfor Future Enterprise Effectiveness

Page 47: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

IT Portfolio Lifecycle

Acc

ount

abili

ty

0

100

Business LeadersBusiness Leaders

CIO

Lesson: Joint Leadership ResponsibilityLesson: Joint Leadership Responsibility

“GOVERNMENT NO EXCEPTION”

Page 48: Shepherd - University of Virginia · mind that people can learn a lot by hearing what went wrong with your project, so don't be afraid to mix the bad with the good. ... 358,000 SharePoint

Lewis ShepherdMicrosoft Institute for Advanced

Technology in Governmentswww.ShepherdsPi.com