pacific rim application and grid middleware assembly - pragma: an overview of past and future...
TRANSCRIPT
Pacific Rim Application and Grid Middleware Assembly - PRAGMA:
An Overview of Past and FutureCatalyzing Global Team Science
PRIME, PRIUS, GLEON
Peter ArzbergerPhilip Papadopoulos, Mason Katz
Gabriele Wienhausen, Linda FeldmanTim Kratz, Fang-Pang Lin
And many more
The Minority-Serving Institutions Cyberinfrastructure Institute MSICI226 June 2006
Why Bother with International Activities? • In the 21st century advances in science and engineering
(S&E) will, to a large measure, determine economic growth, quality of life, and the health of our planet. The conduct of science, intrinsically global, has become increasingly important to addressing critical global issues…Our participation in international S&E collaborations and partnerships is increasingly important as a means of keeping abreast of important new insights and discoveries in science and engineering [NSB 2000]
• What nations don’t know can hurt them. The stakes involved in study abroad are that simple, that straightforward, and that important. For their own future and that of the nation, college graduates today must be internationally competent. [Lincoln Report 2005]
• Peace and prosperity around the world depend on increasing the capacity of people to think and work on a global and intercultural basis. As technology opens borders, educational and professional exchange opens minds. [Annual Report IIE 2005, and http://www.iie.org/ “About”)
e-science’s New Frontier: Merging of Science and Information
Technology – PRAGMA’s Activities
PreviouslyUnobtainable
Observations andUnderstanding Enabling
Technology
• Advance science
Science Drivers
• Focus development
PersistentInfrastructure
• Broaden impact
Education & Capacity
Building
• Develop human resources
Sustained Collaboration
• Build teams and trust
PRAGMA’s Founding Motivations
• The grid is transforming e-science: computing, data, and collaboration
• The problem remains that the grid is too hard to use on a routine basis
• Middleware software and people need to interoperate
• Science is an intrinsically global activity
http://www.pragma-grid.net
Establish sustained collaborations
and
Advance the use of the grid technologies for applications
among a community of investigators working with leading institutions around the Pacific
Rim
Overarching GoalsPRAGMA
Working closely with established activities that promote grid activities or the underlying infrastructure,
both in the Pacific Rim and globally.
PRAGMA’s Role in the Community
• Constructed conduit of technology, ideas, information, people– Multi-directional flow among institutions and
communities of software, approaches
• Developed framework for collaboration– Catalytic in forming teams– Foundation for continued and greater work
• Built trust among members– Interoperability essential among people– Sharing of resources, ideas, and people resulted
Result: many unplanned successes
Collaborative Frameworkfor Success
• SARS: Grid Community Pulls together to Battle SARS
• GLEON: Launching new community effort– Building on EcoGrid in Taiwan
• Expanding Reach of Projects– Optiputer; NEESit; iGEON; IVOA
• PRIME: Creating opportunities for undergraduate students– Reciprocal Projects from Jilin, Osaka
• PRIUS: Osaka University, international internships and “PRAGMA Classes”
• KRocks: Localization of Rocks in Korea
Successes that happened because PRAGMA exists
Conduit for Success• Telescience: Sharing and Contributing Technology• Multi-way Dissemination, Integration, and Synthesis
of Software: – Rocks Rolls of SCE, Gfarm, Ninf-g– Ninf-G into NMI 8.0 (and soon Naregi CA) – iGAP/Gfarm, GAMESS/Nimrod
• Creation of Laboratory for Routine Use Experiments– Grew from 8 sites to 19 sites; Multiple, persistent experiments– Use and co-development of SCMSWeb and other software in
PRAGMA (e.g. CICESE porting SCMSWeb to solaris system; CNIC porting it to Itanium) and use of MOGAS (Grid Acct System – from PRAGMA 8)
– Joint papers Issues and Methods for Building a Multi-Application International Grid Resource (6 institutions on author list); Applications
• Assisted in RPC standards at GGF, and launching of International Grid Trust Federation (Lead by AIST)
Outcomes through on-going meetings
Overview and ApproachProcess to Promote Routine Use Team Science
Application-Driven CollaborationsApplications Middleware
Routine Use Lab/TestbedTesting Applications
Building Grid and GOC
Multiway DisseminationKey Middleware
Workshops and Organization
Information Exchange
Planning and Review
New Collaborations
New Members
Expand Users
Expand Impact
Products
Improved middlewareBroader Use
New CollaborationsTransfer Tech.
StandardsPublications
New KnowledgeData AccessEducation
PRAGMA Grid TestbedPRAGMA Grid Testbed
AIST, JapanCNIC, China
KISTI, Korea
ASCC, Taiwan
NCHC, TaiwanUoHyd, India
MU, Australia
BII, Singapore
KU, Thailand
USM, Malaysia
NCSA, USA
SDSC, USA
CICESE, Mexico
UNAM, Mexico
UChile, Chile
TITECH, Japan
QUT, Australia
UZurich, Switzerland
JLU, China
NGO, Singapore
MIMOS, Malaysia
OSAKAU, Japan
IOIT-HCM, Vietnam
http://goc.pragma-grid.net
Source: Cindy Zheng
PRAGMA Grid resourceshttp://goc.pragma-grid.net/pragma-doc/resources.html
Applications http://goc.pragma-
grid.net• Real science, multiple applications
– Resource sharing• Mpich-g2• Reservation and meta-scheduling
– TDDFT: quantum-chemistry, AIST, Japan– Savannah: climate Model, MU, Australia– QM-MD: quantum-mechanic, AIST, Japan– iGAP: bioinformatic, UCSD, USA– Gamess-APBS: organic chemistry, UZurich,
Switzerland– Siesta: molecular simulation, UZurich,
Switzerland– Amber: molecular simulation, USM, Malaysia– FMO: quantum-mechanics, AIST, Japan– HPM: Genomics, IOIT-HCM, Vietnam– (GEON, Sensor, … <data, sensor>)
SCMSWebhttp://www.opensce.org/components/SCMSWeb
• Web-based monitoring system for clusters and grid– System usage– Performance metrics
• Reliability– Grid service monitoring– Spot problems at a glance
Software Layers
• Globus 2, 3, 4• GT4 pre-WS, 9 sites
• GT4 WS, 1• Moving requirements
iGAPhttp://pragma-goc.rocksclusters.org/applications/igap/igap.html
• Genome annotation pipeline
• Use CSF/Gfarm
• Driver: Wilfred Li (SDSC), Osamu Tatebe
(AIST), Xiaohui Wei (JLU)
• Ready on 7 sites:– AIST: Yoshio Tanaka, Yusuke Tanimura– ASCC: Hurng-Chun Lee, Mike Chiang– KISTI: Jysoo Lee, Jae-Hyuck Kwak – NCHC: Weicheng Huang, Chien-Lin Huang– NCSA: Radha Nandkumar, Tom Roney– SDSC: Mason Katz, Cindy Zheng– TITECH: Satoshi Matsuoka, Hitoshi Aoki
Ninf-G/NMI/Rocks Integration• Ninf-G-2.4/NMI
– Lead integrators: • Yoshio Tanaka (AIST)• Mats Rynge (NMI)
• Ninf-G 2.4 Rocks Roll – Lead integrators:
• Mason Katz (SDSC)• Yoshio Tanaka (AIST)
Ninf-G v2.4.0 released in NMI v8first experience for NMI to include non-U.S. software.
Evaluate and improve Ninf-G2 through the PRAGMA routine-basis experiments
GRID TECHNOLOGIES
PortalsMiddlewareGraphics
Computational Chemistry EngineData Analysis Tools
Hardware
ENABLING NEW SCIENCE
Exploiting grid technology & hybrid computational methods
PARAMETER SEARCH
4 variables15,876 points
Refineable hypersurface
ICCS’03 PRAGMA 4
Monash, Australia HPCC, Japan CRAY, Japan SDSC, USA UCSD, USA CPE, Thailand KISTI, Korea
GEOGRAPHIC DISTRIBUTION OF JOBS DURING EXECUTION
Source: Wibke Sudholt, Kim Baldridge, David Abramson, Colin Enticott, Slavisa Garic
GAMESS and Nimrod/G
The PRAGMA Steering Committee
http://www.pragma-grid.net/steering_committee.htm
Working Groups: Organize PRAGMA Efforts
• Resources– Mason Katz, SDSC– Yoshio Tanaka, AIST
• Biological Sciences– Karpjoo Jeong, Konkuk
U/KISTI
• Telescience– Shinji Shimojo, Osaka– Fang-Pang Lin, NCHC
• Data Computing– Osamu Tatebe, AIST
Some Members and Participants
http://pragma-goc.rocksclusters.org/pragma-doc/org.html; Resource Group Album: Total 85 Pictures – and another album
PRAGMA Future Meetings• PRAGMA 10
– QPSF, JCU, APAC, Townsville, Australia 26 – 28 March 2006
– Bringing the Grid to coastal zones; Held in conjunction with GLEON and Coral Reef
• PRAGMA 11– Osaka University, Japan, approx. 15 – 17 October
2006– Preparing Future Generations; in conjunction with
PRIUS program• PRAGMA 12
– NECTEC, Kasetsart University, Thailand, Spring 2007– Advancing Collaborations with ThaiGrid
• PRAGMA 13– NCSA, Illinois, USA, Fall 2007– PRAGMA Engagements in Cyberenvironments
• PRAGMA 14– NCHC, Taiwan, Spring 2008– Living Grids; Held in conjunction with Taiwan Grid
Activities
Annual Reports
•Prepare for SCxy•Highlight Accomplishments•Promote PRIME and PRIUS•Describe Working Group Progress•Summarize Individual Institution Contributions•Give Additional Information, e.g. Publications, Sponsors
Articles On• Infrastructure in
Australia
• India’s Grid Initiative
• Cyber Science Initiative in Japan
• CI in Korea
• Taiwan CI for Knowledge Innovation
• Brazil
• South Africa
• PRAGMA
http://www.ctwatch.org/quarterly/articles/2006/02
Guest Editor: Radha Nandkumar
CCGrid - Singapore16 – 19 May 2006
• Abramson D, Lynch A, Takemiya H, Tanimura Y, Date S, Nakamura H, Jeong K, Hwang S, Zhu J, Lu ZH, Amoreira C, Baldridge K, Lee H, Wang C, Shih HL, Molina T, Li, W, Arzberger P. Deploying Scientific Application on the PRAGMA Grid Testbed: Ways, Means and Lessons. CCgrid 2006
• Lee B-S, Tang M, Zhang J, Soon OY, Zheng C, Arzberger P. Analysis of Jobs on a Multi-Organizational Grid Test-bed. CCGrid 2006.
• Huang W, Huang C-L, Wu, C-H., The Development of a Computational Grid Portal. Accepted CCGrid 2006.
• Zheng C, Abramson D, Arzberger P, Ayuub S, Enticott C, Garic S, Katz M, Kwak J, Papadopoulos P, Phatanapherom S, Sriprayoonsakul S, Tanaka Y, Tanimura Y, Tatebe O, Uthayopas P. The PRAGMA Testbed: Building a Multi-Application International Grid CCGrid 2006.
Preparing the Global Workforce
PRIME: Providing Students International Interdisciplinary Research Internships and Cultural Experiences
preparing the global workplace of the 21st century•Computer Network Information Center (CNIC), Chinese Academy of Sciences•Cybermedia Center (CMC), Osaka University, Japan•Monash University, Australia•National Center for High-performance Computing (NCHC), Taiwan
PRIME 2005 – Presentations at iGRID 2005• Phylogeny Determined by Incomplete Protein Domain
Content, I.Lee, CNIC• A visualization of network measurements, J.Lee, CNIC• Deployment and Extension of JuxtaView for the Scalable
Adaptive Graphics Environment, C.Cheung, NCHC• Developing the Interface between PDA and Sensors, O
Langman, NCHC* (from U Wisconsin)• Visualizing internet connectivity using Cytoscape, S.Lee,
NCHC• Extending EcoGrid Capability, D.Leu, NCHC*• BOINC as a Nimrod Resource for Quantum Chemistry,
J.Hwang, Monash• Computational Grid Tools for Protein-Ligand Docking
Studies, L Berstis, Monash• Modeling Cardiac Rhythm Alternation, J.Nevo, Monash• SNPs, Protein Structure and Disease, D.Bitton, Monash• Computational Cardiac Modeling, D.Dederko, Monash• The Development of A Querying System for Structured
Metadata in a Datagrid Environment, J.Chen, Osaka• Visualization Tools for Bio-molecular Simulation, C.Liang,
Osaka• A Bio-molecular Simulation Portal, E.Wang, Osaka
prime.ucsd.edu/presentations SC’05
Background and Motivation of PRIUS Pacific Rim International UniverSity
• Success of PRAGMA community– Accumulated Expertise and practice of building Highly advanced
Grid applications and middleware– Formation of Human network of researchers and scientists in
pacific rim
• Activation of Educational activities– PRIME: Providing UCSD students with an opportunity of experiencing,
studying, and learning highly sophisticated technology and cultural background through practical R&D with PRAGMA partners.
Maturity of PRAGMA R&D networkMaturity of PRAGMA R&D network
Stimulating the movement toward Stimulating the movement toward the establishment of PRAGMA Educational networkthe establishment of PRAGMA Educational network
PRIUS: Pacific Rim International UniverSities12 Oct 05
•
• Exchange among PRAGMA Sites• Lectures from PRAMGA members
PRAGMA 11Oct 2006 –
to expand PRIUS
Rough Approach of PRIUS
• Establish a consistent educational program for graduate students.
• On-the-Job-Training Education on PRAGMA
Lecture course:Stimulate students’ ambitious
Short Abroad Internship:Offer trial opportunities
Long Abroad Internship:Provide skill building chances
M1
M2
Doc
P.D Exchange prgm: produces liquidity in human resource
P.D
Educational Part
Research Part
Sou
rce
: S
usum
u D
ate
, O
saka
Uni
vers
ity
http://prius.ics.es.osaka-u.ac.jp/en/index.html
Taiwan’s Natural Beauty
Lake Metabolism Website
http://lakemetabolism.org
10
12
14
16
18
20
22
20-Aug-04
21-Aug-04
22-Aug-04
23-Aug-04
24-Aug-04
25-Aug-04
26-Aug-04
27-Aug-04
28-Aug-04
29-Aug-04
30-Aug-04
31-Aug-04
Wat
er T
emp
erat
ure
(C
)
0
5
10
15
20
25
30
35
40
Win
d S
pee
d (
m/s
)
Surface Temp (C)
0.5m Temp (C)
1m Temp (C)
1.5m Temp (C)
2m Temp (C)
2.5m Temp (C)
3m Temp (C)
Wind Speed (m/s)
RAIN_FALL
Wind Speed
Precipitation (mm/5 minutes)
Typhoon causes water column mixing
Mixing event
Source: Tim Kratz
From Cole, J. J., N. F. Caraco, G. W. Kling, and T. K. Kratz. 1994. Carbon dioxide supersaturation in the surface waters of lakes. Science 265:1568-1570
Of 4665 samples from 1835 lakes worldwide,87% were supersaturated
Why?
Source: Tim Kratz
Programs-Australia-Canada-China-Finland-Florida-New Zealand-Israel-South Korea-Taiwan-United Kingdom-Wisconsin
First meeting:San DiegoMarch 7-9, 2005
Source: T. Kratz
Steering Committee-Peter Arzberger, UCSD, USA-David Hamiltion, University of Waikato, New Zealand-Tim Kratz, University of Wisconsin, USA-Fang-Pang Lin, NCHC, Taiwan
Scalable instrumentation and cyberfrastructure is critical
We can do this scale nowhttp://lakemetabolism.org Source: Tim Kratz
Problematic, but possible with today’s cyberinfrastructureSo
urc
e: T
im K
ratz
Scale needed to answer regional/continental questions
Not currently possible
Source: Tim Kratz
New Paradigm: Global Team Science
U.WaikatoD.Hamilton
Models
NCHCF.P.Lin
Maintain YYLParallelize Codes
U.WisconsinT.Kratz
Maintain Trout BogLake MetabolismUCSD
F.Vernon, S.Peltier,T.Fountain P.ArzbergerROADNet, TelescienceMoore Fnd, PRAGMANIGLAS
B.Q QinMaintain Taihu
Physical Limnology
Kangwon UB.Kim
Maintain SoyangPublic Policy
e-science’s New Frontier: Merging of Science and Information Technology – PRAGMA’s and
GLEON’s Activities
PreviouslyUnobtainable
Observations andUnderstanding Enabling
Technology
• Advance science
Science Drivers
• Focus development
PersistentInfrastructure
• Broaden impact
Education & Capacity
Building
• Develop human resources
Sustained Collaboration
• Build teams and trust
•Bioscience•Chemistry•Envir .Sci.•Geosciences•Lakes (GLEON)•Others
•Ninf-G•Gfarm•Nimrod•SCMCWeb•MOGAS•KRocks•Others (Web service)•Testbed
•Links to resources
•PRAGMA &GLEON Framework
•PRIME•PRIUS•Others
International Education Matters
• Globalization is driving demand for an internationally competent workforce. Our nation's trade with Asia, now concentrated in California, has exceeded Europe since 1979 and is expected to approach $1 trillion a year this decade. The majority of future growth for industries of all sizes is in overseas markets; today one in six new jobs created flows from international trade.
• Access to good jobs will require new skills and competencies. Future careers in business, government, health care, law enforcement, and other good jobs will all require global knowledge and skills. Unfortunately, minorities are underrepresented in international careers and must be exposed to global content earlier in their education.
• Solving new national and human security challenges, including terrorism and HIV/AIDS, and the ongoing problems of poverty and environmental degradation, will require increased knowledge of other world regions, cultures and languages.
• Increased diversity in our nation's classrooms, workplaces, and communities, including new immigrants from many different parts of Asia and Latin America, requires greater understanding of the myriad cultures and histories students bring to school.
•http://www.internationaled.org/why.htm
Acknowledgements
• PRAGMA– Philip Papadopoulos (UCSD)– Mason Katz, Wilfred Li, Kim
Baldridge, Tomas Molina, Cindy Zheng
– Fang-Pang Lin (NCHC)– And many others at all 28
institutions, in particular the Steering Committee
• GLEON– Tim Kratz (U WI)– David Hamilton (U Waikato)– Fang-Pang Lin (NCHC)– And others at 10 other sites
• CREON– Sally Holbrook (UCSB)– Stuart Kininmonth (AIMS)
• PRIME– Gabriele Wienhausen– Linda Feldman– All Host sites and students
• PRIUS– Shinji Shimojo (Osaka)– Susumu Date (Osaka)
• CAMERA– Larry Smarr
• NSF– Bill Chang– Many others
• Gordon and Betty Moore Foundation
• National Institutes of Health
Additional Slides
e-science’s New Frontier: New Directions and Activities for PRAGMA
PreviouslyUnobtainable
Observations andUnderstanding Enabling
Technology
• Advance science
Science Drivers
• Focus development
PersistentInfrastructure
• Broaden impact
Education & Capacity
Building
• Develop human resources
Sustained Collaboration
• Build teams and trust
•Geosciences•Metaproteomics driver•Real-time sensing•Access to microscope•Coupling chem codes
•OptIPuter•Collab. Env. •GOC•Gateways•Globus•Planet Lab
•Links to production resources (TeraGrid, others) •Middleware interactions UK, Netherlands (Switzerland)
•Grids in China•SE Asia, NZ•Central/S. Am.
•Training
e-science’s New Frontier: Merging of Science and Information
Technology – PRAGMA’s Activities
PreviouslyUnobtainable
Observations andUnderstanding Enabling
Technology
• Advance science
Science Drivers
• Focus development
PersistentInfrastructure
• Broaden impact
Education & Capacity
Building
• Develop human resources
Sustained Collaboration
• Build teams and trust
•Bioscience•Chemistry•Envir .Sci.•Geosciences•Others
•Ninf-G•Gfarm•Nimrod•SCMCWeb•MOGAS•KRocks•Others (Web service)•Testbed, AMP
•Links to resources
•PRAGMA Framework
•PRIME•PRIUS•Others
e-science’s New Frontier: Merging of Science and Information
Technology – PRAGMA’s Activities
PreviouslyUnobtainable
Observations andUnderstanding Enabling
Technology
• Advance science
Science Drivers
• Focus development
PersistentInfrastructure
• Broaden impact
Education & Capacity
Building
• Develop human resources
Sustained Collaboration
• Build teams and trust
e-science’s Team Science: Merging of Science and Information
Technology
PreviouslyUnobtainable
Observations andUnderstanding
EnablingTechnology
• Advance science• Parameter Sweep• Dist.Data Files• Web Services• Cross-site query• Collaborative Tools• Many more
Science Drivers
• Focus development• Chemistry• Biology• Lakes• Geosciences• Many more
PersistentInfrastructure
• Broaden impact• Lambda Grids• Wireless sensor network
Education & Capacity
Building
• Develop human resources• Students and postdocs
Sustained Collaboration
• Build teams and trust• Many meetings