utilizing web 2.0 in soa for enhanced data sharing among federal agencies
DESCRIPTION
Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies. Nick Duan, Ph.D. Sr. Software Architect McDonald Bradley, Inc. Oct. 2, 2007. MBI At a Glance. Information technology firm, founded in 1985 Headquartered in Herndon, VA Focus on DoD, IC, and Federal Civil - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/1.jpg)
Utilizing Web 2.0 in SOA for Enhanced Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal AgenciesData Sharing Among Federal Agencies
Nick Duan, Ph.D.Sr. Software ArchitectMcDonald Bradley, Inc.
Oct. 2, 2007
![Page 2: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/2.jpg)
MBI At a GlanceMBI At a Glance
• Information technology firm, founded in 1985
• Headquartered in Herndon, VA• Focus on DoD, IC, and Federal Civil• About 300 associates
– Experts and Industry leaders in enterprise technology and subject matter expertise
• Top Secret Facility clearance with many cleared professionals
CONTRACTOR OF THE YEAR
![Page 3: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/3.jpg)
MBI Core Competency AreasMBI Core Competency Areas
Service-Oriented Enterprise
Data Interoperability
Information Assurance
Enterprise Governance and Management
![Page 4: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/4.jpg)
Overview
• The Ultimate Quest of Sharing Data and Services• SOA Implementation – The Scalability Problem• SOA Enablement via Web 2.0
– Simple, Agile, Efficient
• Current Web 2.0 Technologies• A Web 2.0-based Framework for Enhanced Data
Sharing• A Sample Case Study• Future Tasks• Summary and Conclusions
![Page 5: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/5.jpg)
The Ultimate Quest: Sharing Data and Services
• Millions of data records are published on the Internet by the government– Census data, weather data, housing data, map
data, budget data, airport data, etc..• Data utilization is limited
– Manual search and retrieval, lack of standards and quality control, limited corporation among agencies, long development cycles
• Unleash the potential– A robust and scalable SOA framework to add
values beyond search
![Page 6: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/6.jpg)
General Concepts in SOA for Data and Service Interoperability
• Data Structuralization and Standardization– Data format, metadata standards, authoring tools
• Service Standardization– Protocol standard at transport level and message level
• Infrastructure Services– Search and Discovery– Service Registration– Metadata Registry and Discovery
• Security– Transport level and message/content level
• Governance Rules and Policies– Getting the right data at the right time
![Page 7: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/7.jpg)
The SOA Scalability Problem
• Most SOA implementations today are based on Web Services with complex WS-* stack– Out-of-the-box adoption of the commercial SOA
models and products– Complexity: Distributed Object Computing (DOC)
model does not scale across enterprises– Centralized service registry and repository with
complex governance policies suitable for developing monolethic, closely-coupled systems
– Topdown, ”Creational” development – Lack of overall and consistent conceptual data models
across multiple domains• $Millions spent with limited results
![Page 8: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/8.jpg)
Using Web 2.0 to Achieve Simplicity, Agility and Efficiency
• What is Web 2.0?– Collection of web technologies that leverage existing
web tools and infrastructure• Asynchronous JavaScript and XML (AJAX)• REpresentational State Transfer (REST)• Really Simple Syndication (RSS)• Rich Client Applications, Semantic Web, Wiki, Blogs, etc
– Client-side and Server-side Data Mash-ups• Examples: GoogleMaps, Yahoo Maps, Yahoo!Pipes,
Amazon, eBay, MyFaceBook• Key Advantages of Web 2.0
– Data and service mash-ups at various levels, enabling non-transactional data retrieval
– De-centralized, bottom-up, evolutional approach to enable rapid service offering
![Page 9: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/9.jpg)
Web 2.0 in ActionWeb 2.0 in Action
![Page 10: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/10.jpg)
Asynchronous JavaScript and XML
• Microsoft introduced XMLHttpRequest in IE in late 90’s– Allows JavaScript object to connect with back-end web
servers directly– Enables partial update of web pages from multiple web sites – Simple, efficient object serialization via JSON
<script type=”text/javascript”>
........
</script>
XMLHttpRequestXMLHttpRequest
Web ServerWeb Browser
Server-sideComponent
Server-sideComponent
Server-sideComponent
Full Update
Partial Update
![Page 11: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/11.jpg)
REpresentational State Transfer
• As an architecture style first introduced by Roy Fielding in 2000
• Basic design concept of a good protocol: – Separating transport control from content!– Utilizing the full spectrum of HTTP methods:
• POST, GET, PUT, DELETE– Representing states via resource URLs
• No need for cookies
• Resource-Oriented Computing (ROC): a virtual system comprised of multiple, interconnected web resources
![Page 12: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/12.jpg)
REST Examples
• Retrieving part list info using GET (http://www.parts-depot.com/parts)
<?xml version="1.0"?><p:Parts xmlns:p="http://www.parts-depot.com" xmlns:xlink="http://www.w3.org/1999/xlink"> <Part id="00345" xlink:href="http://www.parts-depot.com/parts/00345"/> <Part id="00346" xlink:href="http://www.parts-depot.com/parts/00346"/> <Part id="00347" xlink:href="http://www.parts-depot.com/parts/00347"/> <Part id="00348" xlink:href="http://www.parts-depot.com/parts/00348"/></p:Parts>
• Retrieving individual part (http://www.parts-depot.com/parts/000345) <?xml version="1.0"?>
<p:Part xmlns:p="http://www.parts-depot.com" xmlns:xlink="http://www.w3.org/1999/xlink"> <Part-ID>00345</Part-ID> <Name>Widget-A</Name> <Description>This part is used within the frap assembly</Description> <Specification xlink:href="http://www.parts-depot.com/parts/00345/specification"/> <UnitCost currency="USD">0.10</UnitCost> <Quantity>10</Quantity></p:Part>
![Page 13: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/13.jpg)
Benefits of Using REST
• Statelessness: Clients are completely de-coupled from the Server
• Uniform Interface: HTTP GET, POST, PUT and DELETE• Uniform State Representation: Define a virtual machine
whose states are represented by interconnected resources via URL
• Unrestricted Scalability: The Web is the limit
![Page 14: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/14.jpg)
Data and Service MashupsData and Service Mashups
• Wikipedia: A mashup is a web application that combines data from more than one source into a single integrated tool
• In general, mashup is the concept of integrating data from multiple data sources to create presentation in a new dimension– Data mashup for presentation– Service mashup for reuse
• Client-side mashup– JavaScript, HTML/DHTML, AJAX, AdobeFlash, Customized APIs
• Server-side mashup– Proxy or gateway services providing data mapping, conversion,
and mediation functions
![Page 15: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/15.jpg)
It's all about Mashups
Server-site MashupsServer-site Mashups
Client-side MashupsClient-side Mashups
Federal Data ProvidersFederal Data Providers
![Page 16: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/16.jpg)
Client vs. Server-side Mash-ups
• Client-side mash-ups: data mash-ups within a browser– Simple, easy, cost-effective, end user driven– Limited functionality, primarily presentation use only,
constraint by security
• Server-side mash-ups: value-added, aggregated services for data integration– Feature rich, sophisticated functionality, less security
constraints– Complex, expensive, business driven
• Create the best combination to achieve maximum effectiveness and reuse
![Page 17: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/17.jpg)
• Server-centric• Thin/simple client• SOAP, WSDL, UDDI• Heavy WS-* stack • Standard-committee
driven• Top-down, creational• Long dev cycle
Moving SOA to a New Dimension
• User/business-centric• Rich client• REST, AJAX, RSS,
REST Registry • Leveraging existing
HTTP/HTTPS • User and community
driven• Bottom-up, evolutional• Short development
cycle
From Web 1.0/SOA 1.0 From Web 1.0/SOA 1.0 to Web 2.0/SOA 2.0to Web 2.0/SOA 2.0
![Page 18: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/18.jpg)
SOA Landscape with Web 2.0
FAA NOAA Census Bureau
USGS States Local
End Users
Aggregated Content and Service Providers
Open Infrastructure Services
YahooPipes
GoogleMaps
Amazon S3
Programmable Web
Yahoo News
Google RSSInternet
Data Providers
Open Source
![Page 19: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/19.jpg)
A Case Study
• Case Scenario – Emergency Planning – In case of a category 5 hurricane, which area
(states/counties) may be affected– Which hospitals are to be evacuated?– Which airports may be used for delivering emergency
supplies?
• Data Contents and Providers– Real-time forecast data in RSS from NOAA (NWS, NHC)– County data from US. Census Bureau in CVS– Airport information from FAA in Excel/CVS– Hospital information from AHD in Excel
![Page 20: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/20.jpg)
Plant 1 Plant 2 Plant 3 Plant 4
0
2
4
6
8
10
12
Estimated Generators Needed
Category 3
Category 4
Category 5
Interaction of Services
Aggregation & Mediation Service
Data MashupsService Registry and Discovery
![Page 21: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/21.jpg)
21
Rich User Functions beyond Search
Geo-spatial Analysis: Locate the hospitals in the impact region
Plant 1 Plant 2 Plant 3 Plant 4
0
2
4
6
8
10
12
Estimated Reduction in Electricity
Category 3
Category 4
Category 5
Converting GIS data intoGoogle-compliant lat/long
Converting/Correlating hospital data AJAX-specific format
Statistical Analysis: Estimated number of evacuees per hospital
Analytics capabilities provided by service aggregation
Aggregated Service
![Page 22: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/22.jpg)
Future TasksWhat are needed for SOA 2.0
Future TasksWhat are needed for SOA 2.0
• Aggregated services to enable rapid data and service integration– Data mapping, mediation,
• Government-sponsored infrastructure services to meet specific need of agencies– Service Registry and Discovery– Standard Metadata Registry– Standard cross domain taxonomies– Security if needed
• Government standard for data quality– Tolerable error rate, availability, security, etc.
![Page 23: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/23.jpg)
Summary and Conclusions
• The majority of data published by the federal government have not been fully utilized
• Vast potential of data and service sharing can be unleashed via SOA
• A simple, agile and efficient SOA framework can be achieved using Web 2.0 technologies for enhancing data and service interoperability
• Data and service mashups are the key towards rapid development and service offerings
![Page 24: Utilizing Web 2.0 in SOA for Enhanced Data Sharing Among Federal Agencies](https://reader035.vdocument.in/reader035/viewer/2022062422/56813168550346895d97de7a/html5/thumbnails/24.jpg)
Q & AQ & A
??