mygrid
DESCRIPTION
myGrid. Architectural issues in a bioinformatics Grid http://www.mygrid.org.uk Luc Moreau, University of Southampton, UK. Overview. Bioinformatics background myGrid facts Service oriented architecture Architectural issues Notification service Grid component model Service directory - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/1.jpg)
Architectural issues in a bioinformatics Gridhttp://www.mygrid.org.uk
Luc Moreau, University of Southampton, UK
myGrid
![Page 2: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/2.jpg)
Overview
Bioinformatics background myGrid facts Service oriented architecture Architectural issues
Notification service Grid component model Service directory
Conclusions
![Page 3: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/3.jpg)
Bioinformatics & Genomics
Large amounts of data
Highly heterogeneous Data types Data forms Community
Highly complex and inter-related
Volatile
![Page 4: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/4.jpg)
Bioinformatics Data
Descriptive as well as numeric
Literature Analogy/
knowledge-based
Text Extractio
n
![Page 5: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/5.jpg)
Bioinformatics Analysis
Different algorithms BLAST, FASTA, pSW
Different implementations WU-BLAST,
NCBI-BLAST Different service
providers NCBI, EBI, DDBJ
![Page 6: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/6.jpg)
The Human Genome Project
The HGP will make available potentially thousands of targets for Understanding
biology & genetics Drug discovery Diagnostics
Many genes will be linked with diseases
Cancer HIV Parkinson’s Asthma Malaria Autoimmune (arthritis) Cardiovascular Antibacterial & antifungal
![Page 7: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/7.jpg)
Drug Discovery
![Page 8: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/8.jpg)
In silico experimentation
Discovery of resources and tools, staging of operations, sharing of results
Process is as important as outcome
Science is dynamic – change happens
Scientific discovery is personal & global
Provenance and history
![Page 9: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/9.jpg)
Overview
Bioinformatics background myGrid facts Service oriented architecture Architectural issues
Notification service Grid component model Service directory
Conclusion
![Page 10: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/10.jpg)
myGrid
EPSRC funded pilot project Generic middleware within application
setting 36 month in 42 month performance period Start 1st October
16 full-time post docs altogether 6 DTA studenships 1 technical project manager 1 system manager 1 secretarial post
![Page 11: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/11.jpg)
myGrid consortium
Scientific Team Biologists and Bioinformaticians GSK, AZ, Merck KGaA, Manchester, EBI
Technical Team Manchester, Southampton, Newcastle,
Sheffield, EBI, Nottingham IBM, SUN GeneticXchange Network Inference, Epistemics Ltd
![Page 12: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/12.jpg)
myGrid outcomes
e-Scientists Bioinformatics demonstrator (on cold carp)
Developers myGrid-in-a-Box developers kit
Integrating some existing bioinformatics tools with myGrid
![Page 13: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/13.jpg)
Overview
Bioinformatics background myGrid facts Service oriented architecture Architectural issues
Notification service Grid component model Service directory
Conclusions
![Page 14: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/14.jpg)
![Page 15: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/15.jpg)
![Page 16: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/16.jpg)
![Page 17: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/17.jpg)
![Page 18: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/18.jpg)
![Page 19: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/19.jpg)
![Page 20: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/20.jpg)
![Page 21: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/21.jpg)
![Page 22: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/22.jpg)
Overview
Bioinformatics background myGrid facts Service oriented architecture Architectural issues
Notification service Grid component model Service directory
Conclusions
![Page 23: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/23.jpg)
Architectural Issues
![Page 24: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/24.jpg)
Architectural Issues
Notification service
![Page 25: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/25.jpg)
Vision
Asynchronous delivery and persistence of messages Topics can be created and discovered on the fly Subscribers can subscribe to topics, publishers can
publish messages on a given topic Peer to peer network of notification services Topology can be re-organized to enhance reliability Subscribers and publishers can negotiate over QoS
![Page 26: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/26.jpg)
A notification service instance
Sub
scrib
er
Subscriberdelegator
publisherdelegator
QoS
notifications
Sub
scrib
er s
tub
Pub
lishe
r
Pub
lishe
r st
ub
![Page 27: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/27.jpg)
Federated notification services
Strong communication links between hubs
Efficient data replication Simple notification routing
Hub-1
NS-1-1
S-1-1-1
P-1-1-1 P-1-3-2
P-1-1-2 NS-1-2
NS-1-3
Hub-2
P-2-2-1
NS-2-1
NS-2-2P-2-2-2
S-2-1-1
Hub-3
P-3-2-1
NS-3-1
NS-3-2
P-3-1-1
S-3-1-1
P-1-3-1
![Page 28: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/28.jpg)
QoS Negotiation Protocol
![Page 29: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/29.jpg)
Current status
Push and pull messaging Topic,message and publisher filter WSDL interface Workflow interaction Integration with mySQL, openJMS, tomcat and
Axis
Federated service (undergraduate project) QoS negotiation (PhD work underway) OGSA compliance
![Page 30: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/30.jpg)
Experimentation
Windows and Unix platforms with Tomcat 4.0.5, Axis beta 3.0, OpenJMS 0.7.2 and mySQL 3.23.51
Aggregation test with 500 topics, 2,000 subscribers, 2,000 publishers and 10,000 registered subscriptions, 10,000 notifications
72 hours non-stop subscribing/publishing with the above populations
![Page 31: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/31.jpg)
Architectural Issues
Notification service
![Page 32: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/32.jpg)
Architectural Issues
Notification service Grid component model
![Page 33: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/33.jpg)
Grid Component Model
The myGrid framework is a component model for flexible, simple and future-proof deployment and use of services on the Grid.
![Page 34: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/34.jpg)
Problems Addressed
For service developers and deployers: Ease of development of sophisticated
services by separation of concerns and re-use of third party functionality.
Consistent distribution of functionality over a set of services, e.g. access control, support for fault-tolerance.
Application of solutions to the above to services deployed using technologies such as OGSA Grid Services, Web Services and Enterprise JavaBeans.
![Page 35: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/35.jpg)
Problems Addressed
For service clients: Development of service clients that are not
limited by the range of standards known at deployment time.
Control over how service operations are invoked, so that they can make use of the most suitable protocols supported by a service.
Provision of a standard client interface hiding the differences in deployment philosophy that each middleware technology brings.
Application of solutions to the above to services deployed using technologies such as OGSA Grid Services, Web Services and Enterprise JavaBeans.
![Page 36: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/36.jpg)
Nested Component Model
![Page 37: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/37.jpg)
Framework
![Page 38: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/38.jpg)
Current Status
Startpoints for Web Services Deployment within nested
containers Facades for exposing EJBs as Web
Services Performance tests
![Page 39: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/39.jpg)
Current Implementation
![Page 40: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/40.jpg)
Current Work
Automated deployment in nested containers
Definition of containers for deployment-time configuration
Using containers to provide minimal functionality of OGSA Grid Services
Startpoints for EJBs, Grid Services
![Page 41: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/41.jpg)
Experimentation
Our experiments have shown that nesting in our containers is not costly compared to method invocation and nested inner classes
The cost of calling EJBs via the Web Service façade comes mostly from the use of SOAP, and the consequential requirement for conversion to/from objects
![Page 42: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/42.jpg)
![Page 43: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/43.jpg)
![Page 44: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/44.jpg)
Architectural Issues
Notification service Grid component model
![Page 45: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/45.jpg)
Architectural Issues
Notification service Grid component model Service directory
![Page 46: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/46.jpg)
Service Directory Views
Multiple service directories will co-exist (IBM, Microsoft, EBI, local institutions)
Need to attach metadata to service directory entries
Metadata is personal to the scientist: trust, perceived QoS, ontological description
Need for a mechanism to allow scientists to add their metadata and to make it available to other users as a “regular service directory”.
![Page 47: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/47.jpg)
Views: status
Currently in design phase Use cases in the process of being
finalized Preliminary specification of interfaces More work is needed on policy languages
Design to be finalized by end of January
First prototype of core functionality 4 months later
![Page 48: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/48.jpg)
Overview
Bioinformatics background myGrid facts Service oriented architecture Architectural issues
Notification service Grid component model Service directory
Conclusions
![Page 49: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/49.jpg)
Conclusions
More architectural issues being addressed Security (GSI, RBAC), but where is the
community going? Fault tolerance
![Page 50: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/50.jpg)
Workflow enactment WSFL compatible enactment engine Support for fault tolerance,
checkpointing, migration Editor
![Page 51: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/51.jpg)
Conclusions
4 months development cycle with “integration fest”
Our roadmap is based on a layered organisation of functionality
![Page 52: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/52.jpg)
myGrid in Southampton
Luc Moreau, Michael Luck, David DeRoure
Terry Payne, Keith Decker Simon Miles, Juri Papay, John
Dickman, Xiaojian Liu, Claudia di Napoli, Vijay Dialani, Richard Lawley
![Page 53: myGrid](https://reader036.vdocument.in/reader036/viewer/2022081519/56814069550346895dabe2da/html5/thumbnails/53.jpg)
www.mygrid.org.uk
m