October 7, 2005VOQL-Madrid 1

IVOA – VOQL WG session

ESACVillafranca del Castillo Madrid, Spain

Friday, October 7th, 9:30 - 13:00

Yuji Shirasaki Maria Nieto-Santisteban

October 7, 2005VOQL-Madrid 2

Agenda

1. Road Map (10’)– Where are we? – Meeting goals

2.Review ADQL and SkyNode specifications WD (10’)

3.Discuss schedule to upgrade SkyNodes from 0.7 to 1.0

4. Metadata. How enforce quality?--- BREAK ----5.Short presentations

Brian Thomas - A VO Catalog model proposal André Schaaff - VizieR in the context of ADQL-SkyNode

October 7, 2005VOQL-Madrid 3

Agenda

6. Future - Brief discussions (As time permits)– The '!' operator problem. Need for proper footprints. – VOQL needs diversity: equivalent implementations of Open

SkyQuery portals accepting distributed queries– XMATCH function(s) algorithms (How do we include errors?) – How do we deal with big query results? The path to VOStore– XMatch works only between primary table from SkyNode

• What needs to be done to allow XMatch with others?– Table data model (standard column names) Where does

Catalog Data Model WG stand? – Query by UCD and Utype– Theorists – How is theory data published and queried?

7. Update Road Map

October 7, 2005VOQL-Madrid 4

Roadmap 05-06 (May – 20th 05)

• May 16-20 2005 Interop Kyoto • June Prepare PR documents• Jul (-- Aug ?) RFC• Aug Vote by the Exec → Rec ADQL & SkyNode------------ future version -------• Sept 2005 First WD ADQL1.x

– Integration of ADQL and DAL (with DAL)– X-Matcher (with DM/DAL)– Inclusion of simulated data access ? (with theory IG)

• Oct 2005 ADASS + IVOA InterOp Madrid, Spain– Discussion on the draft, and revise the WD

• Jan 2006 NVO demos ?• Spring 2006 reference implementation ?• May 2006 InterOp WS in Canada• Aug 2006 IAU Prague : VO Special Session• Autumn 2006 InterOp WS in Russia

– To PR ????

October 7, 2005VOQL-Madrid 5

Where are we? New since KyotoProtocols:• ADQL & SkyNodes WD 1.0? have been edited but NOT

moved into PR• Two new WD docs have been produced aiming simplification

and minimize the impact of upgradesADQL & DAL Integration:• JVO plans to make a second release of the JVO SkyNode

toolkit for building a Skynode with DAL integration and cross match functionality.(To be released after ADASS)

• NVO preliminary discussion about how to use ADQL to query DAL services

• http://dev.openskyquery.net has a prototype version of using Open SkyQuery to query SIAP services and display images

October 7, 2005VOQL-Madrid 6

Where are we? (New since Kyoto)

ADQL <-> SQL Translation:• ESAVO: ESA/ESAC has published a style sheet to

translate between a subset of ADQL and SQL• VO includes a Java translator for ADQL/s <-->

ADQL/x (To be released after ADASS)

SkyNodes implementation:• FULL SkyNode Java implementation was released for

the second NVO Summer School in September 2005• Work has been started to implement a parallel

system that will allow for Large-scale catalog and cross-match queries in Open SkyQuery

October 7, 2005VOQL-Madrid 7

Roadmap 05-06 (May – 20th 05)

• May 16-20 2005 Interop Kyoto • June Prepare PR documents • Jul (-- Aug ?) RFC• Aug Vote by the Exec → Rec ADQL & SkyNode------------ future version -------• Sept 2005 First WD ADQL1.x

– Integration of ADQL and DAL (with DAL)– X-Matcher (with DM/DAL)– Inclusion of simulated data access ? (with theory IG)

• Oct 2005 ADASS + IVOA InterOp Madrid, Spain– Discussion on the draft, and revise the WD

• Jan 2006 NVO demos ?• Spring 2006 reference implementation ?• May 2006 InterOp WS in Canada• Aug 2006 IAU Prague : VO Special Session• Autumn 2006 InterOp WS in Russia

– To PR ????

October 7, 2005VOQL-Madrid 8

ADQL Specification

• Draft still needs some work• Reduced the scope of SQL to the SELECT

statement• Defined a CORE and EXTENSIONs • Defined the BNF independent from language (Post C# as an external reference)• Included 2 “new/different” XMATCH

function(s) – XMATCH -> XMATCH_chi2 (a, b, sigma) (keeps their parameters)– XMATCH_distance(c1a, c2a, c1b, c2b, ‘frame’)

October 7, 2005VOQL-Madrid 9

SkyNode Specification

• Draft still needs some work• Added more metadata information

– ADQLExtension() -> supported extensions– Tables() -> added XMatchSupp and RegionSupp att.– Relation() -> TN1, CT1, TN2, CT2– Column() -> (Indexed ?)

• Needed to add– Decouple cascade and parallel in the ExecPlan– Remove the XMatch algorithm out of the spec

How do we do references– Add a mechanism to transmit "messages" in addition to

data– Content standardization in returned VOTable

• Needed to modify – WSDL

October 7, 2005VOQL-Madrid 10

Metadata

• How do we enforce it?

SELECT t.TableName, t.Name, t.Unit, t.UCD, t.enum, t.descriptionFROM     SHORTNAME:DBColumns t

October 7, 2005VOQL-Madrid 11

Metadata

October 7, 2005VOQL-Madrid 12

Schedule to upgrade SkyNodes

• First, Get WD -> PR -> Rec• Finish WDs:

– Nov 15– Open for comments within WD: 2 weeks (end of

Nov)– Promote to PR (needs at least a month)– Promote to Rec (needs approval from Exec)

• Implement changes & Upgrade by– After AAS, other critical times?– March 1st?

October 7, 2005VOQL-Madrid 13

Brief discussions

• The ! operator problem. Need for footprints

SELECT o.objId, o.ra,     o.dec, t.ra, t.dec,     t.objId, o.type

FROM     SDSS:PhotoPrimary o, TWOMASS:PhotoPrimary t,     USNOB:PhotoPrimary p

WHERE XMATCH(o, t, !p) < 3.5 AND    Region('CIRCLE J2000 182.5 -0.89 8') AND    o.type = 3

It is not the same nothing was found from nothing was there! => Either resolve the footprint or at least give a warning!!

October 7, 2005VOQL-Madrid 14

Brief discussions

• VOQL needs diversity

– Portal supporting distributed queries!

October 7, 2005VOQL-Madrid 15

Brief discussions

• XMatch functions– How do we include error?– It works only between primary table

• What needs to be done to allow XMatch between others tables?

October 7, 2005VOQL-Madrid 16

Brief discussion

• How do we deal with big queries?

October 7, 2005VOQL-Madrid 17

Brief discussion

• Table data model? standard column names• Querying by UCD / UType• Where does Catalog Data Model WG stand?

October 7, 2005VOQL-Madrid 18

Roadmap 05-06 (Oct – 7th 05)• JVO will release a new SkyNode toolkit in October

------------on the tap for future -------• The '!' operator problem. Need for proper footprints• More flexibility for XMATCH

– How do we include errors?– new algorithms

• SkyNode -> High level data access service? • ADQL/x <–> ADQL/s what do we really need?• Include VOStore – Async mech.• Inclusion of simulated data access (with theory IG)

• Jan 2006 NVO demos • Spring 2006 reference implementation ?• May 2006 InterOp WS in Canada• Aug 2006 IAU Prague : VO Special Session• Autumn 2006 InterOp WS in Russia