nvo summer school, september 20061 desktop integration with vo voclient, dalclient nvo summer...
TRANSCRIPT
NVO Summer School, September 2006 1
Desktop Integration with VOVOClient, DALClient
NVO Summer School, Aspen Sept 2006
Doug Tody (NRAO), Mike Fitzpatrick (NOAO)
US National Virtual Observatory
NVO Summer School, September 2006 2
Application Development in the VO
• Types of VO-Aware Applications– Webapps (e.g., datascope)
• interface is a web browser– Application services (wesix, wcsfixer, etc.)
• interface is a web service– Application environments (IRAF, IDL, Python, PHP, etc.)
• interface is a full user applications environment– Grid workflows (e.g., astrogrid)
• interface is a user-generated workflow script (batch job)
• At the back-end, all of these can share the same client-side interface to the VO.
NVO Summer School, September 2006 3
VO Client-Side Development: Requirements
• Support for multiple languages/environments– Every language or environment has its application domain– Let the user decide which is best for their application
• High level functionality– Applications generally prefer a high level environment– Integrate VO with existing powerful data analysis tools
• Isolate application from details of VO infrastructure– VO interface is relatively low level– Functionality can be complex
• multiple versions of interfaces• multiple protocols, grid and web service technologies• storage managers (VOSpace), asychronous data transfers, etc.
– Java is the standard language for the VO infrastructure• but is not favored for most science applications development
NVO Summer School, September 2006 4
Option 1: Custom Integration for each Language
• Approach– Thin layer implemented natively in each target environment
• Advantages– Simplest approach so long the as middleware is thin– Easier to customize interface for each target language– Avoids need for multi-language integration
• Issues– Doesn't scale well as middleware becomes more complex
• have to reimplement functionality for every target environment
• ok, until it gets beyond a certain size– Interface and capability matrix will tend to be different for
each target environment.
NVO Summer School, September 2006 5
Option 2: Common Facility with Multiple Bindings
• Approach– A common client-side middleware facility with a thin binding for each
target language or environment– MySQL (and many other such facilities) use this architecture
• Advantages– Portable– Uniform interface and capability matrix for all target environments– Scales well
• Only one implementation required for all target languages• Also scales well in the sense of parallel computing• New capabilities appear immediately in all target languages
– Can use Java• Most VO and web/grid system-level software is in Java
• Issues– More complex as architecture is multi-language and distributed– Distributed computing performance can be a concern if not careful
NVO Summer School, September 2006 6
The VOClient Middleware
• Objectives– Provide a high level interface to the VO for user development– Support all major languages and environments– Provide reference code within NVO for the DAL services
• Origin– Subsystem in the Opticon/NVO scalable framework architecture
• stands on its own due to open systems approach
– History• began in 2005 with the dalclient Java library• expanded this year to add multi-language support
NVO Summer School, September 2006 7
Architecture
• Applications layer– Python, Java, IRAF, IDL, PHP, CASA, etc.
• Message bus– RPC and async progress events
• VOClient daemon (voclientd)– dalclient Java library– Registry interface– Sesame name resolver– VOSpace, security, async operations– Data access objects
• table, image, spectrum, etc.
PythonIRAF
IDL etc. Java
voclientd
dalclientregistry
NVO Summer School, September 2006 8
VOClient Functionality
• Capabilities– Sesame name resolver
• convert astronomical object names to coordinates– Registry query
• required for data collection and service discovery– Client side of DAL interfaces
• cone search (catalogs), SIAP (images)• soon: SSAP (spectra) and SLAP (line lists)
• Basic interface– Multi-threaded, multiple clients and connections– Stateful client-server connection
• connection-oriented, like a DBMS or other object interface• state is maintained where the data is managed (voclientd)• concurrent access to a shared object is possible
– DAL service interface is RPC to dalclient running in voclientd
NVO Summer School, September 2006 9
Sesame Name Resolver
• Start voclientd if not already running
• Open a connection voc_initVOClient (config_opts)
• Name resolver API sr = voc_nameResolver (target) pos_str = voc_resolverPos (sr) radeg = voc_resolverRA (sr)decdeg = voc_resolverDEC (sr)
• Close connection voc_closeVOClient (shutdown_flag)
NVO Summer School, September 2006 10
DALClient Interface
• DAL class hierarchy– Generic dataset (any dataset)
• Cone catalogs• SIA images• SSA spectra (soon)• SLAP spectral line lists (soon)• SNAP theoretical models (future)
(etc.)
• Basic API– Similar for each class of data
• Form query, execute query, walk result set(Similar to a JDBC database interface)
– Multi-level outputs• stream, votable, data model
NVO Summer School, September 2006 11
DAL Query (Base class)
dal = new DALConnection () dal = new DALConnection (service-url) dal.addService (service-url) dal.getServiceCount () dal.getServiceURL (index)
query = dal.getDALQuery () query.addParameter (name, value) query.getQueryString (serviceIndex) query.executeCSV () query.executeCSV (fname) vot = query.executeVOTable () is = query.executeRAW ()
NVO Summer School, September 2006 12
CONE Query (extends DAL Query)
cone = new ConeConnection ()cone = new ConeConnection (service-url)
query = cone.getConeQuery ()query = cone.getConeQuery (ra, dec, sr) query.addParameter (name, value)
qr = query.execute () query.executeCSV () query.executeCSV (fname)vot = query.executeVOTable () is = query.executeRAW ()
NVO Summer School, September 2006 13
SIAP Query (extends DAL Query)
siap = new SiapConnection () cone = new SiapConnection (service-url)
query = siap.getSiapQuery () query = siap.getSiapQuery (ra, dec, size) query = siap.getSiapQuery (ra, dec, size, format) query = siap.getSiapQuery (ra, dec, ra_size, dec_size) query = siap.getSiapQuery (ra, dec, ra_size, dec_size,
format) query.addParameter (name, value)
qr = query.execute () query.executeCSV () query.executeCSV (fname)vot = query.executeVOTable () is = query.executeRAW ()
NVO Summer School, September 2006 14
Query Response (data model view)
• Format– Result set: a sequence of records
• Interface qr.getRecordCount () rec = qr.getRecord (i)
rec.setAttribute ("<attrname>", "<value>") v = rec.getAttribute ("<attrname>")
v.intValue () v.floatValue () (etc.)
rec.getDataset ("<path>") path = rec.getDataset ()
NVO Summer School, September 2006 15
Registry Interface
• Approach– Very similar to DAL query– Form query, execute query, step through result set
• Interface res = voc_regSearch (sql, keywords, orValues)res =voc_regSearchByService (svc, term, orValues)
query = voc_regQuery (term, orValues) voc_regAddSearchTerm (query, term, orValue) res = voc_regExecute (query)
count = voc_resGetCount (res) str = voc_resGetStr (res, attribute, index) dval = voc_resGetFloat (res, attribute, index)
NVO Summer School, September 2006 16
Current status
• Language Support– C/C++, Java, IRAF, FORTRAN (so far)– Partial support for Python, PHP, etc.
• Messaging– Message bus currently uses a simple brute-force approach
• VOClient Daemon– Connection context currently limited to single service
• Usage– Java and C interfaces are usable– IRAF interface is a good example of where we are headed
with this
NVO Summer School, September 2006 17