astrogrid status soft the vo the future astrogrid presentation to gsc andy lawrence july 2003
TRANSCRIPT
AstroGrid AstroGrid
• status
• SOFT
• the VO
• the future
AstroGrid presentation to GSC Andy Lawrence July 2003
AstroGrid presentation to GSC Andy Lawrence July 2003
status status
• six months into Phase B • team of 26 people (23.4 FTEs)• spent 1.5M (40% of budget) • 10,000 lines of code • working s/w released • working on infrastructure, standards
– not tools
Delivered in Phase-A Delivered in Phase-A
• requirements analysis• technology evaluation• comprehensive Phase A report• demonstration s/w (with AVO)• initial VO standards (with AVO and NVO)• respected position in UK e-sci programme• world ranking position in VO work
Delivered in Itn-01,02 Delivered in Itn-01,02 • basic engine room components
– registry schema, implementation, and query service– data access and job control methods– MySpace manager, explorer, and registry
• preliminary interface and contents– simple portal, no tools– populated with a handful of DCs and datasets
• further international/esci visibility– use of OGSI, OGSA-DAI, formed GGF-RG– defined IVOA standards process
• OGSI experiments– intercontinental visualisation with grid services
Planned for Itn03,04 Planned for Itn03,04
• engine room continued– consolidate Registry, Data Access, MySpace – add Auth/Auth, workflow tool (poss Triana)– Registry harvesting
• interface and contents– astronomer useable interface– meaningful contents– data federation services
• international programme– standards for Registry, VOQL
demonstrations demonstrations
• Jan 2003 : with AVO– modest technical advance – new tools, well integrated, excellent PR
• July 2003 : AGOC and Sydney– large technical advance– no new tools, minimal interface
• next steps– feedback from AGSAG beta testers crucial– wrap tools for next demo
TargetTarget
CurrentCurrent
successes successes
• well motivated hardworking team
• rigorous s/w eng processes
• code cut and released on schedule
• basic engine room components in place
• status in international VO
• status in astronomical community
opportunities opportunities
• our infrastructure could become standard
• additional tool access via AVO collaboration
failures failures
• portal work failed : GUI hacked fixed
• process take-up slow fixed
threats threats
• US diverging on registry standard• OGSA-DAI vs standard Grid• too little Data Centres effort
– incorporating datasets– writing services
Progress on AstroGrid goalsProgress on AstroGrid goals(1) develop standards for data, metadata, data exchange and provenance GOOD(2) develop a software infrastructure for data services GOOD(3) establish a physical grid of resources GOOD but DC effort worrying(4) construct and maintain a Service and Resource Registry GOOD(5) implement a working VO system of real scientific use to astronomers TBD NEXT SIX MONTHS(6) provide a user interface to that VO system BEHIND(7) provide or adapt a set of science user tools to work with the VO LACK OF FUNDING(8) establish a leading position for the UK in VO work GOOD
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system
• give data centres a standard framework for publishing and delivering services using their data.
AstroGrid Phase A ReportOctober 2002
multi- views of a Supernova Remnant
Shocks seen in the X-ray
Heavy elementsseen in the optical
Dust seen in the IR
Relativistic electrons seen in the radio
What happens to the Earth's magnetosphere during a coronal mass ejection ?
Event imaged by space-based solar observatory
Effect detected later bysatellites and ground radar
needles in a haystackneedles in a haystack Hambly et al 2001
- faint moving object is a cool white dwarf- may be solution to the dark matter problem- but hard to find : one in a million- even harder across multiple archives
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system,
• give data centres a standard framework for publishing and delivering services using their data.
VO-1
VO-2
VO-3
The VO is a system that will ..The VO is a system that will ..• allows users to interrogate multiple data centres in a
seamless and transparent way
• provide new powerful analysis and visualisation tools within that system,
• give data centres a standard framework for publishing and delivering services using their data.
VO-1
VO-2
VO-3
looking good
need effort for DCs to "publish and deliver"
almost untouched
the future : data servicesthe future : data services
• VO : geometry– not a warehouse; not a hierarchy; not P2P – small set of service centres + large popn of end-users
• VO : content– not software monolith : framework + standards – needs content : data services and user tools
• implies work for Data Centres– writing services; framework uptake; resource management– upcoming opportunities internationally competitive – establish a few centres as professionally competitive
yesterdayyesterday
browserfrontend
CGIrequest
html
web page
DBengine
SQL
data
todaytoday
appl
icat
ion
webservice
SOAP/XML request
SOAP/XML data
DBengine
SQL
nativedata
anyt
hin
g
standard formats
tomorrowtomorrow
appl
icat
ion
webservice
job
results
anyt
hin
g
webservice
webservice
webservice
webservice
webservice
Registry Workflow
GLUE AstroPass MySpace
standard semantics
publ
ish W
SDL
day after tomorrowday after tomorrow
appl
icat
ion
gridservice
job
results
anyt
hin
g
gridservice
gridservice
gridservice
gridservice
gridservice
Registry
Workflow GLUE AstroPass MySpace
poo
led
res
ourc
e
standard semantics
agents
ontology
work neededwork needed
appl
icat
ion
gridservice
job
results
anyt
hin
g
gridservice
gridservice
gridservice
gridservice
gridservice
Registry
Workflow GLUE AstroPass MySpace
poo
led
res
ourc
e
standard semantics
agents
TOOLS
STANDARDS
INFRASTRUCTURE
TECHNOLOGYRESEARCH
DATA SERVICES(access and analysis)
GRID UPTAKE
DA
TA
PIP
EL
INE
S
ontology
expected status 2004expected status 2004
• tools : ok but minimal
• standards : good but evolving
• glue : good but could be better
• data access services : ok but under-resourced
• data analysis services : almost untouched
• grid uptake : ready to deploy but resource worry
• technology research : treading water
AstroGrid-2 AstroGrid-2
• priority areas– core : evolve and improve, build tools
– establish content : data services, resource grid
– analysis services : visualisation, datamining : research and deploy
– next technology wave : ontology, agents
• expertise needed– existing AstroGrid developer team
– alliance of Data Centres
– add Computer Science expertise
CS expertiseCS expertise
• Leeds : visualisation experts
• Portsmouth : kd-tree algorithms
• Exeter : Intelligent Agents for astronomy
• commercial experience : Linde, Noddle
• local links : Bristol, Edinburgh, Cambridge
• external links : OGSA-DAI, GGF, MS, IBM
InfrastructureInfrastructure
• framework largely in place but..
• needs to be robust and reliable
• needs continuing work on standards– IVOA and Grid
• needs suite of tools
• key idea is personalisation– tool requests from competitive calls
delivers VO-1
R&DR&D
• really mostly D– assess, customise, integrate
• keep on the next wave– ontology, agents
• keep promise that everybody can be a power user– large volume and multi-D visualisation and datamining services
delivers VO-2and re-newed VO-1
Data CentresData Centres
• Data Centres are the heart of the VOthere is no UK VO without well resourced DCs
• top priority is support for VO/uptake
• second priority is core support for DCs– not mission specific resources
– not pipeline or archive development
– basic physical resource management and data curation
delivers VO-1
delivers VO-3
Costed Model OptionsCosted Model Options
• M1 : full programme £9.25M• M2 : full research £6.68M
– lose DCA core support
– world beating VO infrastructure but UK content compromised
• M3 : partial research £5.28M– lose some lines and trim others; minimise new algorithms
– good working VO but loses opportunity for UK lead in new technology
– approximately flat continuation of AG1-PhaseB
• M4 : minimum viable £3.75M– no research; unlikely to lead any third wave of VO development
– working VO but relies on external components
FIN