grid wrapper for ibm websphere information integrator - isegserv
TRANSCRIPT
Grid Wrapper Grid Wrapper for for
IBM Websphere IBM Websphere Information IntegratorInformation Integrator
Patrick Dantressangle 8/2005Patrick Dantressangle 8/2005 IBM Hursley – Emerging Technology ServicesIBM Hursley – Emerging Technology Services [email protected]@uk.ibm.com
AgendaAgenda
The Data GRID: virtualization of dataThe Data GRID: virtualization of data OGSA-DAI and DAISOGSA-DAI and DAIS Websphere II & GRID Websphere II & GRID OGSA-DAI Wrapper features: an example OGSA-DAI Wrapper features: an example
of GRID technology with Websphere II.of GRID technology with Websphere II. Conclusion/questionsConclusion/questions
The GridThe Grid
Distributed Computing Over a Distributed Computing Over a Network of Heterogeneous Resources Network of Heterogeneous Resources
Using Open StandardsUsing Open Standards
Grid ComputingGrid Computing
Storage
ApplicationsApplicationsProcessingProcessing
Operating System
DataData
I/O
Desired World for Desired World for Customers: Customers:
Customers need access to information from anyplace, anytimeCustomers need access to information from anyplace, anytime
Based upon Policy-managed quality of service (QoS)Based upon Policy-managed quality of service (QoS)Which will drive an automatic data placementWhich will drive an automatic data placement
Federation of data stores to utilize existing resourcesFederation of data stores to utilize existing resourcesAccess to all relevant resources – structured, semi-structured, unstructuredAccess to all relevant resources – structured, semi-structured, unstructured
App. App. App. App.
VirtualizedInformation
Request & QoS
Response & Delivered QoS
PlacementManager
DataPolicy
Virtualization of data on Virtualization of data on the Gridthe Grid
EMC IBM XXX TAPE
SAN Volume Controller (SVC)
MicrosoftWindowsVMWare/MSCS
IBM AIXHACMP
SunSolaris
HP/UXLinux(Intel)VMWare
IBMBladeCenter
VMWare/Windows/LinuxOPM/FCS
SAN
OGSA-DAI (GRID data access middleware)
DB2MS Oracle MySQLXML
DataSource accessOver SOA
File system access(SAN FS)
Disk Access
IBM Websphereinformation integrator
OGSA DQP?(open source distributed query processor)Schema integration
Federation
Data Virtualization
Global policy
WS J2EESQL API
GRID registry
WS OGSA-DAIOQL APIOGSA-DAI
Global Mngt
OGSA-DAIOGSA-DAI
DAIS – The GGF Database Access and Integration DAIS – The GGF Database Access and Integration Working GroupWorking Group http://http://cs.man.ac.ukcs.man.ac.uk/grid-db/grid-db
OGSA-DAI – The UK e-Science funded reference OGSA-DAI – The UK e-Science funded reference implementation of DAISimplementation of DAIS http://ogsadai.org.uk/http://ogsadai.org.uk/
IBM Emerging Technology Services : IBM Emerging Technology Services : http://http://tes.hursley.ibm.comtes.hursley.ibm.com//
OGSA-DAI - ObjectiveOGSA-DAI - Objective Define open standards and open source Define open standards and open source
based uniform service interfaces for based uniform service interfaces for accessing heterogeneous data sources within accessing heterogeneous data sources within the Open Grid Services Architecture (OGSA) the Open Grid Services Architecture (OGSA)
Since WS-RF, This is now moving to a full Since WS-RF, This is now moving to a full web services architecture. web services architecture.
Grid
Web
The definition of WSRF means that Grid and Web communities can move forward on a common base
WSRF
Started far apart in apps & tech
OGSI
GT2
GT1
HTTPWSDL,
WS-*
WSDL 2,
WSDM
Have beenconverging
Grid and Web Services: Grid and Web Services: ConvergenceConvergence
(an aside)(an aside)
Some IBM Grid Data Some IBM Grid Data ActivitiesActivitiesGGF - OGSA
OGSA Data Architecture
UK eScience - OGSA DAI
GGF - DAIS Working GroupOGSA Data Services
GGF – OGSI Working GroupOASIS WSRF
2002 2003 2004Today
2005
OGSA-DAI Project OGSA-DAI Project PartnersPartners
GDS InternalsGDS Internals
connection role
data
datadata
query
perform document
response document
elementelement element
credentials
QueryActivity
TransformActivity
DeliveryActivity
Data ResourceImplementation
Role Mapper
TheEngine
connection
OGSA-DAI Supported OGSA-DAI Supported Data ResourcesData Resources
RelationalRelational XMLXML OtherOther
MySQLMySQL XindiceXindice FilesFiles DB2DB2 OracleOracle PostgreSPostgreSQLQL
SQLServeSQLServerr
Some OGSA-DAI Some OGSA-DAI Customers…Customers…
more here more here http://www.ogsadai.org.uk/projects/http://www.ogsadai.org.uk/projects/
OGSA-DAI(http://www.ogsadai.org.uk)
AstroGrid(http://www.astrogrid.org/)
BioSimGrid(http://www.biosimgrid.org/)
BioGrid(http://www.biogrid.jp/)
Bridges(http://www.brc.dcs.gla.ac.uk/projects/bridges/)
eDiaMoND (http://www.ediamond.ox.ac.uk/)
FirstDig(http://www.epcc.ed.ac.uk/~firstdig/)
GeneGrid(http://www.qub.ac.uk/escience/projects.php#genegrid)
GEON(http://www.geongrid.org/)
IU RGRBench(http://www.cs.indiana.edu/~plale/projects/RGR/OGSA-DAI.html)
myGrid(http://www.mygrid.org.uk/)
N2Grid(http://www.cs.univie.ac.at/institute/index.html?project-80=80)
ODD-Genes(http://www.epcc.ed.ac.uk/oddgenes/)
OGSA-WebDB(http://www.gtrc.aist.go.jp/dbgrid/)
Websphere Information Websphere Information Integrator Integrator
WebSphere Information Integrator
Web Services Requests
Event ConsumptionSQL, SQL/XML,
XPath,
XQuery (soon)
Event PublishingWeb Services
JDBC, ODBC, XQJ, Web Services, JMS
Heterogeneous
Data Sources
Enterprise Service Bus
Enterprise Service Bus
Information Virtualization
Can mix and match
Can mix and match
Applications Applications
Applications
BiologicalData and
AlgorithmsText
Sybase
Informix
SQL Server
Oracle
WebSphere II
SQL, SQL/XML
Federation Engine
Wrappers and functions
Teradata
WebSphere MQ
ODBC
IBM Extended
Search
Excel
…
WWW, email,…XML
DB2 UDBon z/OS
Software AGAdabas
VSAM
CA-IDMS
IMS DB2 II ClassicFederation
ODBC
DB2 UDB
Integrated SQL or XML View
Heterogeneous Data Heterogeneous Data SourcesSources
Web services
PackagedApps
CA-Datacom
ContentManagement
systems
WebSphere II Web WebSphere II Web Services OverviewServices Overview
WebBrowser
Soap Client
HTTP
/SOA
P
HTTP
/GET
Tables
Stored Procedures
XML Extender
WebSphere
SQL
HTTP/SOAP
Web
ServiceProviders
WebSphere II applications consume Web Services data
Web Service Wrapperor UDFs
WebSphere II provides Web Services data
Dynamic Query
Service
DADX
SQL ApplicationsSoap Client
DB2 Web Service Provider
(WORF)
WebSphere II
Accessing Web Services from Accessing Web Services from DB2 – Through User Defined DB2 – Through User Defined
FunctionsFunctionsCan integrate SQL Can integrate SQL statements and Web statements and Web Service invocationsService invocations
Support for Support for generating SQL generating SQL scalar and table scalar and table UDFs (User Defined UDFs (User Defined Functions) based on Functions) based on WSDL (web service WSDL (web service description)description)Command line versionCommand line versionTool support Tool support integrated into Web integrated into Web Sphere StudioSphere Studio
Web
Airline FareLanguage Translate
Currency Rate
Stock Quote
Temperature
DB2DB2
Virtual WebData
ServiceProviders
SELECT city, GetTemperature(city)
FROM location
DB2Tables
DB2 Tables
XML
XML
1. Publish
2. Transform & Manipulate
3. Store
1. Publish:•SQL/XML Functions•XML Extender Composition•DB2 Web Services2. Transform & Manipulate:•Validate (schema and DTD)•XSL•Extract (fragments or values through XPath)•Update3. Store:•Intact as CLOB or as XML Extender Type•Shredded into relational through XML Extender Decomposition•DB2 Web Services
Today: XML Support in DB2
For references and more information see http://www.ibm.com/developerworks/db2/library/techarticle/0212malaika/0212malaika.html
Integration of XML & Integration of XML & Relational Capabilities Relational Capabilities
soon, beta in 2005soon, beta in 2005
SERVER
CLIENT SQL/XML
XQueryDB2 Engine
XMLInterface
RelationalInterface Relational
XML
DB2 Storage:
DB2 Client /Customer Client Application
Native XML data type (server & client side) Native XML data type (server & client side) (not Varchar, not CLOB, not object-relational !)(not Varchar, not CLOB, not object-relational !)
XML Capabilities in all DB2 componentsXML Capabilities in all DB2 componentsApplications combine XML & relational dataApplications combine XML & relational data
Websphere Information Integration : Wrapping it up!
Websphere Information Integrator
Wrapper WrapperPlan
QueryPlan
Query Planning
QueryExecution
Generic/ODBC, Oracle, SQL Server, Sybase, Informix, Teradata, SQL Anywhere, Flat file, Excel, XML, Documentum, BLAST,…
Encapsulate data source information inside a wrapperModels data as tables
Expose unique functionWrapper participates in query planning and executionEasy to add for unusual data sources
Planrequest
ExecutionPlan
A Typical Environment, A Typical Environment, TodayToday
HTTP Server
The goal: to achieve an Integrated, Service-Based Architecture
XML
Web Clients Business Partners Enterprise PortalsInternal Applications
XML or Web Service
Data Feeds
Windows
Purchase
Orders
Reporting &
Analysis
UnixzOS Custome
rsInvento
ry System
DB2/MVS IMS Oracle DB2 Excel
Content Manage
r
DB2Digital media
Web Server
Web Service
Web Services
EJBs Servletsand JSPs
Websphere AS
JMS
Worflows
An application produces a An application produces a Report of the SalesReport of the Sales
Purchase
Orders
grouping sales, per region, grouping sales, per region, and showing current and showing current percentage of quota percentage of quota attained.attained.
The information comes The information comes from 3 systems: the EJB from 3 systems: the EJB needs to needs to federatefederate this this data.data.
Oracle
Monthly Sales
Report
EJBs Servletsand JSPs
Application
The Application needs connectivity to each system; load and merge the data; and aggregate it for the report.
Customers
DB2/MVS
Sales Quotas
Excel
Monthly Sales per region
San Jose S.Shanghai Orlando
... 100.5 218.0 88.7
// Merge the data:for each customer in custs: for each order in ords if order.custid == customer.custid region.sales += (order.qty * order.price) i=i+1; . . . etc ...
SELECT custid, qty, priceFROM ORDERSORDER BY custid into ords;SELECT custid, regionFROM CUSTOMESORDER BY custid into custs;
...the application requires ...the application requires complex logic to implement complex logic to implement
federationfederation
Monthly ReportService
Federate
10 000 orders/month 2 000 000 customers
10 000 rows 2 000 000 rows
100 rows
Thousands of lines of
Java code!
2 000 records
2000 salespersonsThe EJB needs a connection to each back-end!JDBC helps accessing Oracle/DB2 data, but what about Excel?Millions of rows loaded into the application, stressing the JVM!Joining large sets of data complex and slow!
JDBC Java file i/oJDBC
MBs of memory required! Monthly Sales per
regionSan Jose S.Shanghai Orlando
... 100.5 218.0 88.7
Only 100 regions
with sales
Websphere AS
DB2 CUSTOMERS
Oracle ORDERS
ExcelSALES
Quotas
Our Environment, with Our Environment, with Information IntegrationInformation Integration
HTTP Server
Purchase
Orders
Reporting &
Analysis
UnixzOS Custome
rsInvento
ry System
DB2/MVS IMS Oracle DB2 Excel
Content Manage
r
DB2Digital media
XML
Web Clients Business Partners Enterprise PortalsInternal Applications
XML or Web Service
Data Feeds
Information Integrator meta-data
Web Services
EJBs Servletsand JSPs
Unified Data Model
Federation
Web Server
Web Service
application
Windows
All Information Sources appear as if from the same database!
Our EJB using Our EJB using Information IntegrationInformation Integration
Purchase
Orders
Unix
Oracle
Monthly Sales
Report
EJBs Servletsand JSPs
Application
Each data source appears as a local table: via a nicknameInformation Integration efficiently merges and aggregate data from all sources, transparently!
Information Integrator
Unified Data Model
Federation
Customers
DB2/MVS
Sales Quotas
Excel
Use SQL to query the virtual database, into which the Use SQL to query the virtual database, into which the Excel Spreadsheet appears as Excel Spreadsheet appears as just another tablejust another table!!
JDBC
ORDERSCUSTOMERS
SALESQUOTA
Native APIs
nicknames:
Federation hides the Federation hides the Location and Source of the Location and Source of the
data behind a nicknamedata behind a nickname
A nickname, for most cases, is just like any other local table!
ORDERS table
Oracle
CUSTOMERS table
DB2/MVS
QUOTAS spreadsheet
Excel
Unified Data Model:
CUSTOMERS nickname ORDERS nickname QUOTAS nickname
ORDSTATS view
Application: SELECT c.region, SUM(o.price * o.qty)FROM CUSTOMERS c, ORDSTATS oWHERE c.custid = o.custidGROUP BY c.region;
DBA:CREATE NICKNAME ORDERSFOR ORACLE8.bourbon.ORDERS;CREATE VIEW ORDSTATS ...
or, if table does not exist:
CREATE TABLE ORDERS(id primary key......OPTIONS(REMOTE_SERVER 'ORACLE8' REMOTE_SCHEMA 'bourbon')
Information Integrator
With Information Integration the With Information Integration the federation chore moves out of the federation chore moves out of the
applicationapplication
DB2 CUSTOMERS
Oracle ORDERS
100 rows
ExcelSALES
Quotas
SELECT region, SUM(o.price * o.qty)FROM SALES WHERE month(sdate) = CURRENT MONTHGROUP BY c.region;
A single simple
database query Single View
Monthly ReportService
Federation
The application still uses JDBC, but only worries about 1 data modelNicknames allow automatic type mappings and transformations
ORDERS nickname
CUSTOMERS nickname
SALESQUOTA nickname
SALES view
100 rows Application
JDBC
A few 100 bytes of memory required!
Monthly Sales per region
San Jose S.Shanghai Orlando
... 100.5 218.0 88.7
Variations of the same Report Variations of the same Report are queried repeatedly: WII are queried repeatedly: WII
can cache it!can cache it!
DB2 will cache query results for Materialized Query Table (MQT)DB2 optimizer also picks MQT for other similar queries!
SELECT region, SUM(o.price * o.qty)FROM SALES WHERE month(sdate) = CURRENT MONTHGROUP BY c.region;
Monthly ReportService
Application
Information Integrator
DB2 CUSTOMERS
Oracle ORDERS
ExcelSALES
Quotas
FederationORDERS nickname
CUSTOMERS nickname
SALESQUOTA nickname
SALES view
JDBC
MQTCache
A single simple
database query
No need to re-fetch the data!
Monthly Sales per region
San Jose S.Shanghai Orlando
... 100.5 218.0 88.7
WebSphere II: ReplicationWebSphere II: Replication
Highlights:Highlights: Release independence (install “new” replication over “old” UDB)Release independence (install “new” replication over “old” UDB) Add Oracle log-based capture for Q ReplicationAdd Oracle log-based capture for Q Replication Enable Classic Enable Classic sourcessources to participate in Q Replication (VSAM, IMS, IDMS) to participate in Q Replication (VSAM, IMS, IDMS) Enable Heterogeneous targetsEnable Heterogeneous targets Replication of schema changes (starting with Alter Table/Column)Replication of schema changes (starting with Alter Table/Column) Expand peer-to-peer conflict resolution schemes (merge, user-defined)Expand peer-to-peer conflict resolution schemes (merge, user-defined)
ControlControl
FederationEngine
Admin
WebSphere MQ
Capture TargetApply
DB2 II Classic EP for z/OS
VSAM IMS IDMS
Log Based
OracleDB2
Heterogeneous
Introduction to Replication : ftp://ftp.software.ibm.com/ps/products/db2/info/vr82/pdf/en_US/db2gpe80.pdf
WS Classic EPFor z/OS
DB2 UDBfor z/OSVSAM IMS
JDBC ClientODBC Client
WebSphere II: Event WebSphere II: Event PublishingPublishing
Workflow
WebSphere Business Integration
XML message • Asynchronous communication between data and application layers to build powerful solutions
• Key capability of DB2 II for both distributed and “classic” platforms
• New – addition of IDMS sources
IDMS
WS IIEvent Publisher
DB2 UDBfor LUW
Summary: Federated Summary: Federated TechnologyTechnology
Infrastructure for extractingInfrastructure for extracting information information from from datadata Wrappers allow access to key data sourcesWrappers allow access to key data sources Query engine provides cross-source queriesQuery engine provides cross-source queries
An extension of industrial-strength DB2 technologyAn extension of industrial-strength DB2 technology Robust, high function, high performanceRobust, high function, high performance Benefit from R & D in relational DBMSBenefit from R & D in relational DBMS Benefit from experience in content managementBenefit from experience in content management
Eases application developmentEases application development Transparency, heterogeneity, high functionTransparency, heterogeneity, high function APIs for modern environments (XML, J2EE, Web Services)APIs for modern environments (XML, J2EE, Web Services)
Application AutonomyApplication Autonomy Does not disrupt existing applicationsDoes not disrupt existing applications
The Grid wrapper for The Grid wrapper for Websphere Websphere
Information Integrator Information Integrator V8V8
What is the Grid What is the Grid Wrapper ?Wrapper ?
New technology developed here in New technology developed here in HursleyHursley
Bridging Websphere II v8 with the Bridging Websphere II v8 with the standard based Grid world to hide data standard based Grid world to hide data heterogeneity on a datagrid as well as heterogeneity on a datagrid as well as introducing new access patterns.introducing new access patterns.
Released as an IBM Alphaworks Released as an IBM Alphaworks technology in August 2005technology in August 2005 http://www.alphaworks.ibm.com/http://www.alphaworks.ibm.com/
OGSA-DAI - todayOGSA-DAI - today
External resources
External World
OGSA/OGSI Grid
External resources represented as grid services
= Grid services
Note: The application has to “integrate/federate” all the data from the different GRID services.
Grid enabled application
!!!! Schema integration and federation of Data grid services were features required by OGSA-DAI users in the 2004 user group meeting in Edinburgh.!!!!!
OGSA-DAI + Grid WrapperOGSA-DAI + Grid Wrapper
External resources
External World
= Grid/WS-RF services
External resources represented as grid services
OGSA-DAI + Websphere II
OGSI/WS RF Grid
Standard DB2 enabled application
registry
Data and GRID/Web Services access Data and GRID/Web Services access patternspatterns
The Diamond graph.The Diamond graph.
Data
Registry
Service
Access,Performance
Orchestration, data format, federation
Dynamic,Discovery
Registration,Discovery
Virtualisation
Arrow indicates interaction
Other Services
1
2
3
Websphere IIClient
OGSA-DAI Wrapper for instance
OGSADAI
DBMS
Boundary (Firewall or adminstrative, Technology)Grid
Websphere II
OGSA-DAI
Reuse DBMS specific wrapper where possible
WAS Application Application
SQL SQLSQL
Schema integration
WS/Grid Service Discovery/Creation
Leveraging Both Worlds Leveraging Both Worlds SQL
Grid/OGSADAI and Grid/OGSADAI and Websphere II Websphere II
Wrapper = registry which is pretty static Wrapper = registry which is pretty static less likely to change less likely to change
Server = type of Resource knows how to Server = type of Resource knows how to map information from this type of resource, map information from this type of resource, not tied to a specific resource but is tied to not tied to a specific resource but is tied to the virtualisation of a particular resourcethe virtualisation of a particular resource
Nickname tied to particular table schema or Nickname tied to particular table schema or set of data which can be returned from a set of data which can be returned from a service, ties it to a domain and data set but service, ties it to a domain and data set but not specific set of data in a specific location.not specific set of data in a specific location.
Some Customer Scenarios, Some Customer Scenarios, leveraging OGSA-DAI & leveraging OGSA-DAI &
Websphere IIWebsphere II SELECT:SELECT: Get data from first available or Preferred Get data from first available or Preferred
GRID Data Resource (GRID-DR) GRID Data Resource (GRID-DR) OR or PREFERRED QoSOR or PREFERRED QoS
UNION ALL:UNION ALL: Select and consolidate all result sets from all GRID-DR Select and consolidate all result sets from all GRID-DR AND QoSAND QoS
UNION ALL WITH PARTIAL RESULT SETSUNION ALL WITH PARTIAL RESULT SETS Select and consolidate all result sets from GRID-DR Data Select and consolidate all result sets from GRID-DR Data
Resource, and it may not matter if one or many GRID-DR is Resource, and it may not matter if one or many GRID-DR is down, but get a warning back that not all data were fetched. down, but get a warning back that not all data were fetched. (see EDiaMoND type scenario)(see EDiaMoND type scenario)
AND and PARTIAL AND QoSAND and PARTIAL AND QoS Third Party delivery:Third Party delivery:
results sets will be delivered to another place (FTP site, WEB results sets will be delivered to another place (FTP site, WEB Server) and result set holds URL of where to fetch the data Server) and result set holds URL of where to fetch the data from. from.
Quality of Service: ANDQuality of Service: AND
Registry
9.20.6.100
9.20.6.101
9.20.6.102
?
9.20.6.1009.20.6.1019.20.6.102
select * from table
Result:select * from table
select * from table
Quality of Service: PARTIAL Quality of Service: PARTIAL ANDAND
Registry
9.20.6.100
9.20.6.101
9.20.6.102
?
9.20.6.1009.20.6.1019.20.6.102
select * from table
Result:select * from table
select * from table
An example for Partial And:
EDiaMoND Scenario (Part I) As it is today.
CoreServices
OGSADAI
DataLoad
Core API
DB2 ContentManager
CHU
CoreServices
OGSADAI
Core API
DB2 ContentManager
UCL
DataLoad
CoreServices
OGSADAI
Core API
DB2 ContentManager
UED
DataLoad
CoreServices
OGSADAI
Core API
DB2 ContentManager
KCL
DataLoad
???
Core API
Training Application
Training API
LocalFiles
Database
TrainingServices
OGSADAI
OGSADAI
Core & Training API Core & Training API Core & Training API Core & Training API
TrainingAppl
TrainingAppl
TrainingAppl
TrainingAppl
Websphere II Data Federation (static union all view)
An example for Partial And:
EDiaMoND Scenario (Part II) with the WebSphere II wrapper.
CoreServices
OGSADAI
DataLoad
Core API
DB2 ContentManager
CHU
CoreServices
OGSADAI
Core API
DB2 ContentManager
UCL
DataLoad
CoreServices
OGSADAI
Core API
DB2 ContentManager
UED
DataLoad
CoreServices
OGSADAI
Core API
DB2 ContentManager
KCL
DataLoad
???
Core API
Training Application
Training API
LocalFiles
Database
TrainingServices
OGSADAI
OGSADAI
Core & Training API Core & Training API Core & Training API Core & Training API
TrainingAppl
TrainingAppl
TrainingAppl
TrainingAppl
Websphere II Data Federation (using OGSA-DAI wrapper: fully dynamic : no union all view!! )
Registry
Quality of Service: Quality of Service: PREFERREDPREFERRED
Registry
9.20.6.100
9.20.6.101
9.20.6.102
?
9.20.6.1009.20.6.1019.20.6.102
Result:
select * from table
Quality of Service: ORQuality of Service: OR
Registry
9.20.6.100
9.20.6.101
9.20.6.102
?
9.20.6.1009.20.6.1019.20.6.102
Result:
select * from table
select * from table
Quality of Service: Third Quality of Service: Third Party DeliveryParty Delivery
Registry
9.20.6.100
9.20.6.101
9.20.6.102
?
9.20.6.1009.20.6.1019.20.6.102Result:
URLsselect * from table
select * from table
FTP/GRIDFTP/HTTP Server holding result sets
URLs
URLs
Client
Wrapper Security – Wrapper Security – ComponentsComponents
Makes use of:Makes use of: Grid Security Infrastructure (GSI) Grid Security Infrastructure (GSI)
http://www-unix.globus.org/toolkit/docs/3.2/security.hthttp://www-unix.globus.org/toolkit/docs/3.2/security.htmlml
Grid Certificate Service (GCS)Grid Certificate Service (GCS)http://gcs.globus.org:8080/gcs/index.htmlhttp://gcs.globus.org:8080/gcs/index.html
Java Commodity Grid Kit (Java CoG Kit)Java Commodity Grid Kit (Java CoG Kit)http://www.cogkit.orghttp://www.cogkit.org
Create Wrapper Create Wrapper statementstatement
CREATE WRAPPER ogsadai LIBRARY ‘db2qgjava.dll’CREATE WRAPPER ogsadai LIBRARY ‘db2qgjava.dll’ States the wrapper name and the wrapper library to be used States the wrapper name and the wrapper library to be used
(part of Websphere II). In this case a Java non-relational wrapper.(part of Websphere II). In this case a Java non-relational wrapper. UNFENCED_WRAPPER_CLASS and UNFENCED_WRAPPER_CLASS and
FENCED_WRAPPER_CLASSFENCED_WRAPPER_CLASS Directs DB2 to the Java classes implementing the necessary logic.Directs DB2 to the Java classes implementing the necessary logic.
REGISTRY_TYPE and FILENAMEREGISTRY_TYPE and FILENAME Points to the XML registry in this case. Points to the XML registry in this case. Can also point to a relational registryCan also point to a relational registry
CREATE WRAPPER “ogsadai" LIBRARY 'db2qgjava.dll‘
OPTIONS( DB2_FENCED 'Y', FENCED_WRAPPER_CLASS 'com.ibm.db2.wrapper.onDemand.OnDemandFencedWrapper',
FILENAME 'C:\gridwrapper\bin\registry.xml',
REGISTRY_TYPE 'XMLFILE‘,
UNFENCED_WRAPPER_CLASS 'com.ibm.db2.wrapper.onDemand.OnDemandUnfencedWrapper‘
);
Create ServerCreate Server
Simple statement isn’t it?Simple statement isn’t it?
CREATE SERVER ogsadai_server WRAPPER ogsadaiCREATE SERVER ogsadai_server WRAPPER ogsadai States name of server and which wrapper it belongs to.States name of server and which wrapper it belongs to.
Options about service quality is possible here but we Options about service quality is possible here but we prefer to locate them in the registry.prefer to locate them in the registry.
CREATE SERVER “ogsadai_server" WRAPPER "THEGRIDWRAPPERBASIC“;
Create Nickname Create Nickname statementstatement
CREATE NICKNAME WRAP.basetbl (pkey INT NOT NULL, chr CHAR(1), chr20 CHAR(20), vchr20 VARCHAR(20), intgr INT, flt DOUBLE, mny DECIMAL(19,4), sdt TIMESTAMP, bt SMALLINT) FOR SERVER ogsadai_server
OPTIONS (TABLE_NAME ‘WRAP.basetbl')
CREATE NICKNAME WRAP.basetbl CREATE NICKNAME WRAP.basetbl Nickname identifier , here in schema WRAPNickname identifier , here in schema WRAP
pkey INT NOT NULL, chr CHAR(1),chr20 CHAR(20)…pkey INT NOT NULL, chr CHAR(1),chr20 CHAR(20)… Defines the nickname columns, this can be done dynamically.Defines the nickname columns, this can be done dynamically.
FOR SERVER ogsadai_server OPTIONS (TABLE_NAME FOR SERVER ogsadai_server OPTIONS (TABLE_NAME 'basetbl')'basetbl') Indicates which server will be used as well as the logical Indicates which server will be used as well as the logical
table name, the one to be matched with information stored table name, the one to be matched with information stored in the registry.in the registry.
DEMO NOW!DEMO NOW!
A basic grid wrapper XML A basic grid wrapper XML registryregistry<?xml version="1.0" encoding="UTF-8"?><?xml version="1.0" encoding="UTF-8"?>
<registry version="0.0.0" expireafter="0"><registry version="0.0.0" expireafter="0"> <class id="DB2" vendor="IBM" product="DB2" version="-"/><class id="DB2" vendor="IBM" product="DB2" version="-"/> <class id="MySQL" vendor="MySQL" product="MySQL" version="-"/><class id="MySQL" vendor="MySQL" product="MySQL" version="-"/>
<transform target="DB2" fname=“/gridwrapper/TransformPatterns/db2.transform“ <transform target="DB2" fname=“/gridwrapper/TransformPatterns/db2.transform“ kind="patterns"/>kind="patterns"/>
<transform target="MySQL" fname=“/gridwrapper/TransformPatterns/mysql.transform" <transform target="MySQL" fname=“/gridwrapper/TransformPatterns/mysql.transform" kind="patterns"/> kind="patterns"/>
<connectivity id="DB2 Test DB JDBC" type="JDBC" <connectivity id="DB2 Test DB JDBC" type="JDBC"
implementation="com.ibm.db2.jcc.DB2Driver“ implementation="com.ibm.db2.jcc.DB2Driver“ url="jdbc:db2://myhost:50000/MYDB“ userid="db2localuser“ url="jdbc:db2://myhost:50000/MYDB“ userid="db2localuser“
password="db2localpass“ />password="db2localpass“ />
<connectivity id="MySQL Test DB OGSADAI“ type=" <connectivity id="MySQL Test DB OGSADAI“ type=" OGSADAIOGSADAI " " url="url="http://localhost:8080/ogsa/services/ogsadai/DBSGDSF http://localhost:8080/ogsa/services/ogsadai/DBSGDSF “ />“ />
<table logicalname=“WRAP.demotest" assemblage="all-ns"><table logicalname=“WRAP.demotest" assemblage="all-ns"> <table assemblage="direct" using="MySQL Test DB OGSADAI " <table assemblage="direct" using="MySQL Test DB OGSADAI "
physicalname=“DB1.demotest“ class="MySQL“ />physicalname=“DB1.demotest“ class="MySQL“ /> <table assemblage="direct“ using="DB2 Test DB JDBC“ <table assemblage="direct“ using="DB2 Test DB JDBC“
physicalname=“MyDB.demotest“ class="DB2“ />physicalname=“MyDB.demotest“ class="DB2“ /> </table></table>
<table logicalname=“WRAP.basetbl" assemblage="direct" <table logicalname=“WRAP.basetbl" assemblage="direct" using="MySQL Test DB JDBC" physicalname=“DB1.basetbl“ class="MySQL“ />using="MySQL Test DB JDBC" physicalname=“DB1.basetbl“ class="MySQL“ />
</registry> </registry>
Possibilities of this Possibilities of this wrapperwrapper
Using GRID registry as policy enforcementUsing GRID registry as policy enforcement Move DB from one QoS to another by moving them in and Move DB from one QoS to another by moving them in and
out of registries.(domain specific registries)out of registries.(domain specific registries) Change data space topology by just interacting with Change data space topology by just interacting with
registries. No impact on the applications/SQL logic. No need registries. No impact on the applications/SQL logic. No need to recreate nicknames if DB move from one system onto to recreate nicknames if DB move from one system onto another. another.
DBA liked this idea very much. (see demo for GUI)DBA liked this idea very much. (see demo for GUI) Switch between local access (JDBC..) to SOA depending of Switch between local access (JDBC..) to SOA depending of
application requirement/infrastructure needs. see diamond application requirement/infrastructure needs. see diamond graph in a previous slide.graph in a previous slide.
Leverage distribution of data by fetching from Leverage distribution of data by fetching from multiple GRID data sources in parallel.multiple GRID data sources in parallel.
Many more scenario where loose coupling is Many more scenario where loose coupling is involved…involved…
ConclusionConclusion Websphere II can really be GRID enabled and be the Websphere II can really be GRID enabled and be the
unique simple front end for all your data on the GRID unique simple front end for all your data on the GRID
It is available on IBM AlphaWorks technology for freeIt is available on IBM AlphaWorks technology for free http://http://alphaworks.ibm.com/tech/gridwrapperalphaworks.ibm.com/tech/gridwrapper A tutorial is provided for easy set-up and trainingA tutorial is provided for easy set-up and training
Read the article “Read the article “Bridging the integration gap, Part Bridging the integration gap, Part 1: Federating grid data1: Federating grid data“ on IBM DeveloperWorks “ on IBM DeveloperWorks web site:web site: It’s about why we built it and the vision behind it.It’s about why we built it and the vision behind it. http://www-128.ibm.com/developerworks/grid/library/gr-feddata/http://www-128.ibm.com/developerworks/grid/library/gr-feddata/
We will add more features as customers ask for them.We will add more features as customers ask for them.
Questions/ CommentsQuestions/ Comments
Patrick DantressanglePatrick Dantressangle [email protected]@uk.ibm.com