cwic data partner’s guide (opensearch)ceos.org/.../cwic_opensearch_data-partner-guide.pdf ·...

CEOS CWIC Project

CWIC Data Partner’s Guide (OpenSearch) ApprovalDate:2017-05-09

PublicationDate:2017-05-10

ReferencenumberofthisDocument:CWIC-DOC-14-001r010

Documentversion:V1.0

Category:CWICTechnicalDocument

Editors: Eugene G. Yu, Lingjun Kang, Archie Warnock, Li Lin

CWIC Data Partner’s Guide (OpenSearch) Version: 1.0

Page I

CWICImplementationTeam

EugeneG.Yu,GeorgeMasonUniversity([email protected])ArchieWarnock,A/WWWEnterprises([email protected])LiLin,GeorgeMasonUniversity([email protected])

Approvals ApprovedBy Signature Date

Yonsook K. Enloe

DocumentControl

Name CWICDataPartner’sGuide(OpenSearch)Doc. Ref. No. CWIC-DOC-14-001r010Document status Under reviewing Date of release 2017-04-26

RevisionHistory

Date Version Brief Description Author

2014-04-26 0.9 CWIC Data Partner’s Guide (OpenSearch) drafting

Lingjun Kang Archie Warnock

2017-04-26 1.0 Revised release Eugene G. Yu, Archie

Warnock, Lingjun Kang, Li Lin


Page 2

TableofContentsCWICImplementationTeam........................................................................................................I

DocumentControl........................................................................................................................I

RevisionHistory............................................................................................................................I

ExecutiveSummary.........................................................................................................................4

RecommendationsforCWICDataPartners................................................................................4

1.BeforeYouBegin.........................................................................................................................5

1.1CWICBackground..................................................................................................................5

1.2CWICConceptandDesign.....................................................................................................5

1.3CWICArchitecture.................................................................................................................7

1.4CWICTermsandDefinitions..................................................................................................8

1.5CWICSystems........................................................................................................................9

1.6.ContactInformation.............................................................................................................9

2.CWICOpenSearchQueryInterface...........................................................................................10

2.1Introduction.........................................................................................................................10

2.2ObtainingOpenSearchDescriptionDocument(OSDD).......................................................10

2.3Searchrequest.....................................................................................................................11

3.CWICOpenSearchresponse......................................................................................................12

4.PartnerGuidelines.....................................................................................................................13

4.1Metadata&SemanticMapping..........................................................................................13

4.1.1 HTTPaccess...............................................................................................................13

4.1.2 Spatialsearch............................................................................................................13

4.1.3 Temporalsearch.......................................................................................................13

4.1.4 RequestfordatasetbyID.........................................................................................14

4.1.5 RequestforgranulebyID.........................................................................................14


Page 3

4.1.6 UniquegranuleIDs...................................................................................................14

4.1.7 Recordcounts...........................................................................................................14

4.1.8 Pagination.................................................................................................................14

4.1.9 Othersearchparameters..........................................................................................14

4.1.10 Searchstatus&Errorresponses..........................................................................15

4.2Interaction&ServicesModel..............................................................................................15

4.2.1 UniquegranuleID.....................................................................................................15

4.2.2 Contactinformation..................................................................................................15

4.2.3 BrowseURL...............................................................................................................15

4.2.4 OrderURL.................................................................................................................15

4.2.5 Paginationsupport....................................................................................................16

4.2.6 Spatialsearch............................................................................................................16

4.2.7 Temporalsearch.......................................................................................................16

4.3ErrorHandling.....................................................................................................................16


Page 4

ExecutiveSummary

RecommendationsforCWICDataPartnersWhileCWICcanserveasanOpenSearchproxytosearchalmostanyInternet-accessibleinventorysystemforDataPartners,thereareasmallnumberofrecommendationsforDataPartnersthatwillmakethejobvastlyeasier.

1. Registereachdistinct,searchabledatasetintheIDN2. ProvideasearchinterfaceaccessibleviaasimpleURL(i.e.HTTPGET),ideallyincluding

parametersforstartingrecordnumberandnumberofrecordsdesiredintheresponse3. Supportsearchingonspatialboundingbox4. Supportsearchingontemporalextent,atleastobservationstartandenddates5. Providesearchresponsesinwell-structuredtext(XML,JSON,etc.)returningmatching

datagranules6. Identifyeachreturneddatagranulebyanidentifierthatisuniquewithintheinventory

system7. Provideacapabilityforusingthegranuleidentifiertoretrievemetadataaboutthe

granule8. ReturnURLsforbrowsedataanddirectaccesstogranule-leveldata(ortoadata

orderingsystem)inthesearchresponse

Ingeneral,thesearecommonandwidelyimplementedcapabilitiesinalmostanygranulesearchsystemandshouldnotrepresentanimpedimenttojoiningCWICasaDataPartner.Ifanyofthesecapabilitiesarenotimplemented,itisstillpossibletobecomeaCWICDataPartner–contactanyoftheCWICteamfordetails.


Page 5

1.BeforeYouBegin

1.1CWICBackgroundForscientistswhoconductmulti-disciplinaryresearch,theremaybeaneedtosearchmultiplecatalogsinordertofindthedatatheyneed.Suchworkcanbeverytime-consumingandtedious,especiallywhendifferentcatalogsmayusedifferentmetadatamodelsandcataloginterfaceprotocols.Itwouldbedesirable,therefore,forthosecatalogstobeintegratedintoacatalogfederationwhichwillpresentawell-knownanddocumentedmetadatamodelandinterfaceprotocoltousersandhidethecomplexityanddiversityoftheaffiliatedcatalogsbehindtheinterface.Withsuchafederation,usersonlyneedtoworkwiththefederatedcatalogthroughthepublicinterfaceorAPItofindthedatatheyneedinsteadofworkingwithvariouscatalogsindividually.

CommitteeonEarthObservationSatellite(CEOS)addressescoordinationofthesatelliteEarthObservation(EO)programsoftheworld'sgovernmentagencies,alongwithagenciesthatreceiveandprocessdataacquiredremotelyfromspace.WorkingGrouponInformationSystemsandServices(WGISS)isasubgroupofCEOS,whichaimstopromotecollaborationinthedevelopmentofsystemsandservicesthatmanageandsupplyEOdatatousersworld-wide.TorealizeafederatedcataloguefordatadiscoveryfrommultipleEOdatacenters,theCEOSWGISSIntegratedCatalog(CWIC)systemwasimplemented.CWICwasexpectedtoprovideinventorysearchtoWGISSagencycatalogsystemsforEOdata.

1.2CWICConceptandDesignCWICusesamediator-wrapperarchitecturethathasbeenwidelyadoptedtorealizetheintegratedaccesstoheterogeneous,autonomousdatasources.AsdepictedinFig.1,thedatasourcearchivesdataanddisseminatesitthroughtheInternet.Thewrapperontopofthedatasourceprovidesauniversalqueryinterfacebyencapsulatingheterogeneousdatamodels,queryprotocols,andaccessmethods.Themediatorinteractswiththewrapperandprovidestheuserwithanintegratedaccessthroughtheglobalinformationschema.

Wrappersofferqueryinterfaceshidingtheunderlyingdatamodel,accesspath,andinterfacetechnologyofthepartnercatalogsystems.Wrappersareaccessedbyamediator,whichoffersusersafront-endintegratedaccessthroughitsglobalschema.Theuserposesqueriesagainsttheglobalschemaofthemediator;themediatorthendistributesthequerytotheindividualsystemsusingtheappropriatewrappers.Thewrapperstransformthequeriessotheyareunderstandableandexecutablebythepartnercatalogsystemstheywrap,collecttheresults,transformthemappropriatelyandreturnthemtothemediator.Finally,themediatorintegratestheresultsasauserresponse.


Page 6

Fig. 1 The Mediator-Wrapper Architecture

Basedonthemediator-wrapperarchitecture,currentversionofCWIChasbeendevelopedandoperationalwithfollowingdatapartnercatalogsystems:theCommonMetadataRepository(CMR)ofNASA,theNationalCentersforEnvironmentalInformation(NCEI)ofNOAA,theGroupforHighResolutionSeaSurfaceTemperature(GHRSST)ofNOAA,theUSGSLandsatSurfaceImaging(LSI)Explorer,theNationalInstituteforSpaceResearch(INPE)CatalogSystemofBrazil,theEuropeanOrganisationfortheExploitationofMeteorologicalSatellites(EUMETSAT),theCanadaCentreforMappingandEarthObservation(CCMEO),theMeteorologicalandOceanographicSatelliteDataArchivalCentre(MOSDAC)oftheIndianSpaceResearchOrganisation(ISRO),andtheNationalRemoteSensingCenter(NRSC)ofISRO.

Differentqueryinterfaceswereusedtoaccessthedatapartnercatalogsystems:

Data partner OpenSearch OGC CSW Native query interfaceNASACMR Yes No YesNOAAGHRSST Yes Yes NoNOAANCEI Yes Yes NoUSGSLSI Yes No NoBrazilINPE No No YesCanadaCCMEO Yes Yes NoISROMOSDAC No Yes YesISRONRSC Yes No YesEUMETSAT Yes No No

Inordertoimplementaone-stopfederatedcatalogsystem,wrappershavebeendevelopedtoimplementCWICOpenSearchforindividualmembercatalogsthatdonotcurrentlyofferthatcapability.


Page 7

Fig. 2 The System Architecture of CWIC

Fig.2illustratesthesystemarchitectureofCWIC.Wrapperswereimplementedfordifferentdatapartnercatalogsystems(i.e.,NASACMR,NOAAGHRSST,NOAANCEI,USGSLSI,INPE,CCMEO,ISROMOSDAC,andISRONRSC).Thewrapperisresponsiblefortranslatinganddispatchingrequesttodifferentdatainventories.Themediatorisinchargeofdispatchingthequeryrequesttothedatapartnerinventorysystemandreturnstheresponsetodatauser.

1.3CWICArchitectureAtitscore,CWICpresentstoEndUsersandClientsanOpenSearchserver.ToDataPartners,itappearstobeaweb-basedclient.Itconnectsthetwo(EndUsersandDataPartners)throughtheMediatoronthefrontendservingastheOpenSearchservertoendusersandOpenSearchclienttoConnectors.TheConnectorsarecustom-writtenproxiesforthedatagranuleinventorysearchsystemsattheindividualDataPartners,acceptingOpenSearchsearchrequestsfromtheMediator,translatingthemintovalidsearchrequestsforthetargetdataset,thenparsingtheresultsfromtheinventorysearchsystemandtranslatingthoseintoOpenSearchsearchresponseswhicharepassedbacktotheMediator.

Inthisway,outsideclientsand,forthemostpart,theMediatoritselfneedtohavenospecificknowledgeoftheparticularpartnerdatasystemsandcommunicateonlyviaOpenSearch.EachDataPartnerwillgenerallybeaccessedbyadedicatedConnectorcalledbytheMediator.TheConnectorhandlesallofthedetailsuniquetoindividualdatapartnerinventorysystemandallof


Page 8

thecommunicationswiththepartner’sinventorysystemismanagedexclusivelybytheconnector.

1.4CWICTermsandDefinitionsForthepurposesofthisdocument,thefollowingtermsanddefinitionsapply:

(1) Client

Asoftwarecomponentthatcaninvokeanoperationfromaserver

(2) DataClearinghouse

Thecollectionofinstitutionsprovidingdigitaldata,whichcanbesearchedthroughasingleinterfaceusingacommonmetadatastandard

(3) Identifier

Acharacterstringthatmaybecomposedofnumbersandcharactersthatisexchangedbetweentheclientandtheserverwithrespecttoaspecificidentityofaresource

(4) IDNDatasetID

UniquedatasetidentifierinIDN,returnedfromtheIDNinresponsetotheOSDDrequest.ThisidentifierisassignedbytheIDNCMRdatabase.

(5) NativeID

DatasetidentifierusedbyCWICtoretrievegranulemetadatathroughdataproviderAPI.Thisidentifierisassignedbythedataprovider.

(6) CatalogID

Identifiersofdataprovidercatalogsorconnectionsservinggranulemetadata.

(7) Operation

Thespecificationofatransformationorquerythatanobjectmaybecalledtoexecute

(8) Profile

Asetofoneormorebasestandardsand-whereapplicable-theidentificationofchosenclauses,classes,subsets,optionsandparametersofthosebasestandardsthatarenecessaryforaccomplishingaparticularfunction

(9) Request

TheinvocationofanoperationbyaCWICclient

(10) Response

Theresultofanoperation,returnedfromCWICservertoCWICclient

(11) Collection


Page 9

Agroupingofgranulesthatallcomefromthesamesource,suchasamodelinggrouporinstitution.Collectionshaveinformationthatiscommonacrossallthegranulesthey"own"andatemplatefordescribingadditionalattributesnotalreadypartofthemetadatamodel.

(12) Dataset

Samemeaningascollection,see(8)

(13) Granule

Thesmallestaggregationofdatathatcanbeindependentlymanaged(described,inventoried,andretrieved).Granuleshavetheirownmetadatamodelandsupportvaluesassociatedwiththeadditionalattributesdefinedbytheowningcollection.

(14) IDN

TheCEOSInternationalDirectoryNetwork(IDN)isaGatewaytotheworldofEarthSciencedata.

1.5CWICSystemsTherearetwooperationalCWICsystemstowhichend-usershaveaccess.

§ CWICPROD–thisisCWICproductioninstanceandisavailabletoallusers.

Location:http://cwic.wgiss.ceos.org/

§ CWIC TEST – this is CWIC testing instance used by data partners and CWIC clients toperformtestingbeforechangesaremadetotheCWICproductioninstance.

Location:http://cwictest.wgiss.ceos.org/

TheproductioninstancewillprovideaccesstoonlydatasetswhichhavebeenregisteredwiththeIDN.Thetestinginstancemayprovideaccesstoadditionaldatasets(e.g.,newdatasetsundergoingtestingandnotyetregisteredintheIDN),andcapabilitieswhichhavenotyetbeentestedsufficientlytomovetotheproductionsystem.

1.6.ContactInformationAllthedocumentsandinformationaboutCWICareavailableatWGISSCWICpageat

http://wgiss.ceos.org/cwic

AnyquestionsregardingtoCWIC,pleasesendtheemailto

[email protected]


Page 10

2.CWICOpenSearchQueryInterface

2.1IntroductionOpenSearchisalight-weightsearchspecification1usedbyCWICtosearchandreturnmetadatarelatedtogranule-levelinventorydata.CWICOpenSearchisnotdesigned,norisitusedforreturningobservationaldatafromtheinventorysystems,althoughthemetadatareturnedshouldincludelinksdirectlytodatagranulesortoadataorderingsystem.CWICOpenSearchisintendedtotaketheenduserasclosetoactualdataaspossiblewithintheconstraintsofthedatapartnerinventorysystemsandthelimitsoftheOpenSearchprotocolitself.

TheCWICConnectorshavethetaskofreturningvalidresponsesinAtom2formatasperCWICOpenSearchrequeststotheMediator.Thesearegeneratedon-the-flybysubmittingsearchrequeststotheDataPartner’sinventorysystemfortherequesteddataset,retrievingtheresultsandtranslatingthemintosyntacticallyvalidandsemanticallymeaningfulCWICOpenSearchresponses.TheConnectorimplementerwillworkwiththeDataPartner’ssupportteamtodefinethemappingsbetweenquantitiescontainedintheinventorysystemresponseandtheassociatedelementsintheCWICOpenSearchresponses.

2.2ObtainingOpenSearchDescriptionDocument(OSDD)OpenSearchDescriptionDocuments(OSDDs)providenecessaryinformationforclientstoprogrammaticallyconstructvalidsearchrequests.Specifically,clientsareabletoacquirebothcardinalityanddomainofrequestparametersbasedonquerytemplate.Datasetvalids(i.e.spatialfootprintandtemporalextent)arealsoprovidedthroughtheOSDDinbothmachineparsableandhumanreadableformats.Withdatasetvalids,clientsareabletoconstructrequestsyieldingmoreaccurateresults.

CWICprovidesbothgenericanddatasetspecificOSDDs.ClientscanfetchagenericOSDDthroughtheCWICOSDDendpoint.TheOSDDrequestmustalsoincludeaclientidentifierstring,asrecommendedbytheCWICOpenSearchBestPractices.Clientscanalsoretrieveadataset-specificOSDDthroughtheOSDDendpointbysendingbothclientIDanddatasetidentifier(i.e.DIFEntryID).Inadataset-specificOSDD,domainisalsoprovidedforsomeparameters(i.e.timeStartandtimeEnd)inadditiontotherequestparametersyntax.

GenericOSDDrequestURLexample:

http://cwic.wgiss.ceos.org/opensearch/datasets/osdd.xml?clientId=foo

1OpenSearchspecificationverison1.1draft5(http://www.opensearch.org/Specifications/OpenSearch/1.1)

2Atomsyndicationformat(http://tools.ietf.org/search/rfc4287)


Page 11

DatasetspecificOSDDrequestURLexample:

http://cwic.wgiss.ceos.org/opensearch/datasets/C1235542031-USGS_LTA/osdd.xml?clientId=foo

2.3SearchrequestTheCWICOpenSearchrequestoperationcanbeusedtoconductgranule-levelsearchesonthetargetsystemforarangeofparameters.Italsodefinesthestartpageoftheresultsetandmaximumnumberofresultsperpagetobereturned.CWICOpenSearchcurrentlysupportsfollowingrequestparameters:uniqueidentifier(dataset/granuleID),spatial(geo:box)andtemporalsearch(time:start/time:end)


Page 12

3.CWICOpenSearchresponseAnOpenSearchresponseisanATOMfeedwithzeroormoreATOMentries.Eachentryrepresentsthemetadataofasinglegranulepertainingtothequerysubmittedwithinasetofresultsalsodefinedbythequery.

AnATOMresponseisasingleATOMfeedelementcontainingthefollowing,1. Informationaboutthesearchconductedintermsoftitle,authorandID.2. Informationaboutthenatureoftheresultsetintermsoftotalnumberofresults,numberof

resultsreturnedandhowmanytheclientaskedfor.3. Navigationinformationfortraversingthatresultsetincludinglinkstotheprevious,next,

firstandlastresultsintheset.4. Zeroormoreentriespertainingtogranulemetadatamatchingtheclientquery

AnentryisanATOMentryelementcontainingthefollowing,1. Basicinformationaboutreturnedgranuleintermsoftitle,author,summaryandID.2. Informationaboutthespatialfootprintofreturnedgranule.3. Informationaboutthetemporalextentofreturnedgranule.4. CollectionofATOMlinkelementscontainingendpointsofgranulebrowse,orderand

metadataforthereturnedgranule.


Page 13

4.PartnerGuidelines

4.1Metadata&SemanticMapping

4.1.1 HTTPaccessAlthoughCWICwillattempttouseanymechanismavailableforconnectingtoDataPartners’datamanagementsystemsinordertoaccesstheavailableinventorysearch,thereareafewspecificswhichmaketheprocesssimplerandmorerobust.

TheuseofHTTPforaccessingtheinventorysearchengineisstronglypreferred.Thisiswidelyusedalready,aswebbrowsersarenearlyuniversalandprovideaneffectiveuserinterfaceforbothhumanandautomatedaccess.

Similarlyforresults,CWICwillattempttoextracttherelevantresultsfromanyresponsesthepartnerdatasystemreturns.However,structuredtextofsomesort–XML,forexample–isstronglypreferred.Theabilitytoeasilyanddefinitivelyparseresultsmakestheprocessofmappingthemetadatareturnedinthesearchresponsesimplerandlesserror-prone.Otherstructuredformatslikecomma-delimitedtablesorJSONareacceptable.

4.1.2 SpatialsearchAllCWICdatapartnersareexpectedtosupportsomelevelofspatialsearchsincealloftheinventorydataareanticipatedtohaveaspatialcomponent.Foreachgranule,asimpleboundingbox,withtheboundingcoordinatesindividuallyidentified,istheminimumrequired,althoughmorecomplexspatialfootprintgeometriesarepossibleinthefuture.Thenumberofspatialfootprintsindatapartner’sresponseisnotlimitedtoone.

ItisdesirabletohavetheAPIalsosupportadynamiccalltoreturnthelimitsofthespatialsearch,althoughnotnecessary.ThepresenceofsuchaservicecanhelpCWICavoidinvalidorinappropriatesearchrequests,suchasthoseoutsidethespatialboundariesforspecificdatacollections.

4.1.3 TemporalsearchSimilartospatialsearchdescribedintheprevioussection,allCWICdatapartnersareexpectedtosupportsomeleveloftemporalsearchsincealloftheinventorydataisanticipatedtohaveatemporalcomponent.Simpletemporalextent,withthestartandendtimesindividuallyidentifiedistheminimumrequired,althoughmorecomplextemporalrelationsareanticipatedinthefuture.ItisbesttosupportsomeminimalsubsetoftheISO8601timespecificationforsyntax–YYYY-MM-DD,atleast.

ItisdesirabletohavetheAPIalsosupportadynamiccalltoreturnthelimitsofthetemporalextentsearch,althoughnotnecessary.ThepresenceofsuchaservicecanhelpCWICavoid


Page 14

invalidorinappropriatesearchrequests,suchasthoseoutsidetheexistingtemporalextentforspecificdatacollections.

4.1.4 RequestfordatasetbyIDDatasetID(cwic:datasetId)servesasamandatoryparameterintheCWICOpenSearchqueryinterface.CWICwillqueryagainstdatapartner’ssearchsystem,usingthedatasetID,toretrievethematchinggranulelevelmetadata.Fromthispoint,alldatapartnersareexpectedtosupportthequerybasedondatasetidentifier.Inparticular,agloballyuniquedatasetidentifier(i.e.DIFEntryID)isrecommendedtobeimplementedasqueryparameterondatapartnerside.However,otheruniquedatasetidentifierswithindatapartnersystemisalsoacceptable.

4.1.5 RequestforgranulebyIDCWICOpenSearchsupportssearchingforasinglegranulebygranuleID.CWICwillqueryagainstdatapartner’ssearchsystemwiththegranuleIDinordertoretrievethematchinggranulelevelmetadata.Fromthispoint,alldatapartnersarerecommendedtosupportthequerybasedongranuleidentifier.

4.1.6 UniquegranuleIDsEachdatagranulereturnedinasearchresponseshouldhaveanidentifierassociatedwithitwhichisuniquewithinthedataset.Itisimportantthatthesearchresponseincludetheuniqueidentifierforeachgranulesothatthefulldataonindividualgranulesmayberetrievedwithoutre-executinga(potentiallytime-consuming)search.

4.1.7 RecordcountsAspartofthesearchresponsefromtheinventorysystem,itishighlydesirabletohavethetotalcountofmatchinggranulesreturned,evenifthemetadataforthegranulesisnotcontainedinthesearchresponse.Thisparameter,coupledwiththeabilitytospecifythestartingrecordnumberandnumberofdesiredrecordsfromtheinventorysystem,willallowclientstoimplementresultspagingandreducingtheloadonboththeCWICsystemandonthedatapartners.

4.1.8 PaginationPaginationissupported(i.e.searchbyspecifyingstartpageandcountsperpage)intheCWICOpenSearchqueryinterface.Thesearchbasedonpaginationparameter(e.g.startpageanditemperpage)orcursorparameter(e.g.startindexanditemperpage)isexpectedtobeimplementedbytheDataPartner.

4.1.9 OthersearchparametersTheDataPartnerinventorysystemsmaysupportsearchonadditionalparameterslikeLANDSATpath/row.ThesearenotsupportedthroughCWICOpenSearchatthecurrenttimebutposenoparticularissuesfortheCWICconnectors.

CWICisalsoconsideringsupportforresultssorting–bystarttime,byduration,bygranuleID,etc.–anditwouldbeuseful,althoughnotmandatory,tohaveasortoptionavailablefromthePartnerinventorysystem.


Page 15

4.1.10 Searchstatus&ErrorresponsesUsefulstatusanderrormessageshelptheConnectormanageclientsessionseffectively.Anylimitationsonsubmittedsearchrequeststotheinventorysystemsshouldbenotedintheresponse(e.g.,“toomanyrecordsrequested”,“searchtimedout”)sothatpredictableerror-handlingcanbemanagedbytheConnector.

4.2Interaction&ServicesModel

4.2.1 UniquegranuleIDAsdescribedabove,eachdatagranuleshouldhaveauniqueidentifierwhicha)ispassedbacktotheclientaspartofthesearchresponseandb)canbeusedasakeywithwhichtoretrievethatspecificgranule.TheCWICcomponentswillmanagethetaskofassociatingtheidentifierwiththecorrectdatasetanddatacenter.

4.2.2 ContactinformationTheCWICOpenSearchresponsesincludecontactinformationforeachgranule.Theseareusuallythesameforalldatagranules,andfrequentlythesameforalldatasetswithinasingledatacenter.

Thereisnoneedforthisinformationtobereturnedwitheachsearchresponseoreachdatagranule,althoughitmightbe.TheCWICConnectorcancachethisinformationintheCWICruntimeenvironment,butthismechanismisnotrecommendedbecauseitremovescontroloverthecontactinformationfromtheDataPartnerandchangesmayrequiremodificationstotheConnectorsourcecode.Thiscanleadtodelaysinincludingthecorrectcontactinformationwhenchangesareimplementedbythedatacenter.

4.2.3 BrowseURLIfbrowseimagesofthedatagranuleareavailable,avalidURLtodisplaythebrowseimageshouldbeincludedinthesearchresponseforeachgranulesothattheclientcandisplayitasalink.WhileitispossiblefortheCWICconnectortobuildtheURLbasedonsomepre-defined,fixedpattern,thismechanismisnotrecommendedbecauseitremovescontrolovertheformoftheURLfromtheDataPartnerandchangesmayrequiremodificationstotheConnectorsourcecode.ThiscanleadtodelaysinthedeploymentofthecorrectURLwhenchangesareimplementedbythedatacenter.

4.2.4 OrderURLTheCWICteamrecommendsthat,whenthegranuledatacanbedownloadedfromthedatacenterdirectly,avalidURLtoretrievethedatabeincludedinthesearchresponseforthegranule.

Alternatively,thesearchresponsemaycontainaURLdirectingtheusertoawebsitefororderingthedataifthisistheonlyoptionpermittedbythedatacenter.Thisisoftennecessaryevenforfreelyavailabledataif,forexample,datacenterpoliciesrequireuserregistrationbeforedownloadsaremadeavailable.Insuchcases,theCWICteamstronglyrecommendsthatthegranuleIDrequestedbecachedattheorderingsystemsothatwheneverthedatacenter


Page 16

requirementsfordownloadingdataaremet,theuserwillbeabletoretrievethedatawithoutre-enteringthegranuleID.

4.2.5 PaginationsupportAsstatedabove,thereisnostrictrequirementintermsofpaginationmechanismorpaginationparametersimposedbyCWIC.Variouspaginationmechanismandheterogeneousparametersareexpectedacrossdatapartners.Toprovideuniversalpagination,theCWICconnectortranslatesstartPageandcount(stipulatedbyCWICOpenSearchBestPractices)tonativepaginationparametersforeachDataPartner.

4.2.6 SpatialsearchAsstatedabove,aDataPartnerisexpectedtosupportspatialsearchandincludeproperspatialcomponentineachgranuleresponse.Nativespatialcomponents(oneormany)fromdatapartnerareconvertedsyntacticallyandsemanticallytoproperspatialcomponents(e.g.georss:boxandgeorss:polygon)whicharestipulatedbyCWICOpenSearchBestPractices.Besidesthat,aminimumboundingrectangle(MBR)isalsocalculatedbasedonnativespatialcomponentsforeachgranuleifitisnotreturnedexplicitlyintheresponseandincludedinCWICOpenSearchresponse.

WiththeknowledgeofDataPartnerspecificlimitationsonspatialsearch,CWICwillimposevalidationonthespatialrequestparameterandreturnboththeHTTPstatuscodeandanerrormessageifnecessary.

4.2.7 TemporalsearchAsstatedabove,aDataPartnerisexpectedtosupporttemporalsearchandincludestandard(ISO8601compliant)temporalelementsineachgranuleresponse.Temporalelementswithnativeformatfromdatapartnerareconvertedsyntacticallyandsemanticallytosingletemporalelement(i.e.dc:date)whicharestipulatedbyCWICOpenSearchBestPractices.

WiththeknowledgeofDataPartnerspecificlimitationsontemporalsearch,CWICwillimposevalidationontemporalrequestparameterandreturnboththeHTTPstatuscodeandanerrormessageifnecessary.

4.3ErrorHandlingErrorhandlinginCWICOpenSearchisbasedonstandardHTTPstatuscodes.TheCWICdevelopmentteamisinvestigatingwaystoenhancetheerrorlogginganddocumenting.EfforthasalsobeendedicatedtoprovidesensibleerrormessagesinadditiontothegenericHTTPstatuscodetotheenduserorclient.Inordertosupportthiseventuality,itwouldbeusefulfortheinventorysearchsystemtoattempttoreturnsensibleandrelevantHTTPstatuscodes(whereapplicable)ifsomethinggoeswrongwiththesearchor,perhapsevenbetter,asmall,descriptiveresponsedocument(inXMLorJSONorwhateverthedefaultformatmightbe)providingerrorcodesanderrortext.Inthisway,theCWICconnectorscandistinguishthetypeoferrorarisingattheinventorysystemfromthosearisingelsewhereandtakeappropriate


Page 17

action.TherearenospecificrecommendationsatthistimebutthisshouldbepartofongoingdiscussionsbetweentheConnectordevelopersandtheDataPartner’ssupportstaff.