space weather serviceswe.ssa.esa.int/.../ral/wp432_report_v11.pdfa key distinction is made in the...

ESTEC/ContractNo. 14069/99/NL/SBESASpaceWeatherStudy(ESWS)WP432.SpaceWeatherService

Space Weather Service

ESWS-RAL-TN-0003Issue 1.1, November 6, 2001

Author: Richard Stamper , CLRC Rutherf ord Appleton Laborator y

ESA technical Officer: A Hilg ers D/TOS, TOS-EMA

ESWS Doc No:Issue:1.1

ESWS-RAL-TN-0003Date:November6, 2001

SpaceWeatherService Page2

©CCLRC2001Thisdocumentis approvedfor wider releaseby ESA underthetermsof ESAContract14069/99/NL/SB.




Contents

Contents 3

List of Tables 4

List of Figures 4

1 Intr oduction 81.1 Purpose..... . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 81.2 Scope..... . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 8

1.2.1 ServicevsDataprovision. .... . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 81.2.2 Targetaudience. .... . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 8

1.3 Structureof thedocument..... . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 9

2 Ar chitecture overview 102.1 Philosophy. .... . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 102.2 UserRequirements. ... . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 10

2.2.1 General.... . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 102.2.2 Serviceproviders. ... .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 112.2.3 Wider publicaudience...... . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 11

2.3 GenericArchitecture. .... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 12

3 Data Handling 123.1 Datainput ..... . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 14

3.1.1 Datasources. .... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 143.1.2 DataRetrieval ... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 15

3.2 DataStorageandProcessing..... . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 173.2.1 Local versusRemotedata ..... . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 173.2.2 DataHistory..... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 183.2.3 Scheduling. .... .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 193.2.4 Databasesystems. ..... . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 193.2.5 Dataformats..... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 213.2.6 Metadata. .... . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 21

3.3 DataOutput .... . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 233.3.1 Formats.... . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 233.3.2 Methods..... . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 233.3.3 Bandwidth. ... . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 24

4 EnhancedData Products 254.1 Datapipelining. .... . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 254.2 Models. ... . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 26

4.2.1 Requiredmodels.... .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 264.2.2 Model interconnection...... . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 284.2.3 Modelexecution. .... .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 30

4.3 Graphics..... .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 30

5 User Interface 315.1 On-linedataservices. .... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 315.2 Staticmaterialsfor distribution ..... . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 335.3 Humancontact..... . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 34




6 Outr each 356.1 Generalpublic. ... .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 356.2 Schools.... . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 356.3 Universities. ... . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 366.4 Industry. ... . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 36

7 Operationsand Management 367.1 Management. ... . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 36

7.1.1 Resourcerequirements. ... .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 377.1.2 Realizationrequirements.... . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 377.1.3 Remedialrequirements..... . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 37

7.2 Operations. .... . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 377.2.1 ComputingInfrastructure. .... . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 387.2.2 Personnel.... . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 387.2.3 Datainputs .... .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 397.2.4 Dataoutputs..... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 407.2.5 Dataprocessing..... .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 40

7.3 Costing..... . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 417.3.1 Start-upcosts.... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 417.3.2 Operatingcosts. ... . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 41

8 Futur eDevelopment 428.1 Scientificdevelopment. ... . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 428.2 Technicaldevelopment.... . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 448.3 Organisationaldevelopment. ... . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 45

9 Summary 46

References 47

A Data StorageRequirements 50

B Service Implementation Proposals 52

List of Tables

2 ServiceUserRequirementsandServiceFunctionalElements.... . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 133 Starter/Determinerclassificationof dataretrieval ... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 154 Request/Delivery classificationof dataretrieval .... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 165 StructuredFile Formats:Advantagesanddisadvantages..... .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 226 Initial modelsetof theSWS...... . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 277 Set-upcosts. ... . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 428 Runningcosts.... .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 429 Futuremodelsfor theSWSto provide. .... . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 4310 Measurementdatarate. ... . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 5111 ConsolidatedServiceImplementationProposals. ... . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 52

List of Figures

1 High-level designfor theSpaceWeatherService..... . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 14




2 Modesof executionof two models.... . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 293 Exampledataplots. ... . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . .. . . . . .. . . . . .. . . . . . 32




List of Abbreviations

AE AuroralElectrojetAMPTE ActiveMagnetosphericParticleTracerExplorerAO AuroralOvalAp PlanetaryA index (geomagneticactivity index)ASCII AmericanStandardCodefor InformationInterchangeCARI Civil AeromedicalResearchInstituteCDF CommonDataFormatCDPP Le CentredeDonneesdela PhysiquedesPlasmasCERN ConseilEuropeenpourla RechercheNucleaireCME CoronalMassEjectionCORBA CommonObjectRequestBroker ArchitectureCREME CosmicRayEffectsonMicro-ElectronicsCSMR ConsolidatedSystemMeasurementRequirementCTIP CoupledThermosphereIonospherePlasmaspheremodelDoD Departmentof DefenseDst Disturbancestormtime(geomagneticactivity index)DTD DocumentTypeDefinitionECMWF EuropeanCentrefor Medium-rangeWeatherForecastingEPO EducationandPublicOutreachESA EuropeanSpaceAgencyESTEC EuropeanSpaceResearchandTEchnologyCentreESWP EuropeanSpaceWeatherProgrammeEU EuropeanUnionEUV ExtremeUltraVioletFTP File TransferProtocolGCR GalacticCosmicRaysGIC GroundInducedCurrentGIM GlobalIonosphericMapsGNSS GlobalNavigationSatelliteSystemGNU Gnu’s Not UnixGUMICS GrandUnified Ionosphere-Magnetosphere CouplingSimulationHF High FrequencyHLA High Level ArchitectureHTML HyperText MarkupLanguageHTTP Hypertext TransferProtocolIACG Inter-Agency Consultative GroupIDL Interactive DataLanguage

InterfaceDescriptionLanguageIEEE Instituteof ElectricalandElectronicsEngineersIMF InterplanetaryMagneticFieldIPS InterPlanetaryScintillationISO InternationalStandardsOrganisationISTP InternationalSolar-TerrestrialPhysicsKp PlanetaryK index (geomagneticactivity index)LEO Low EarthOrbitLET LinearEnergy TransferLHC LargeHadronColliderMSFM MagnetosphericSpecificationandForecastModelNASA NationalAeronauticsandSpaceAdministration




NGDC NationalGeophysicalDataCenterNOAA NationalOceanicandAtmosphericAdministrationPHP PHP:Hypertext ProcessorPIM ParameterizedIonosphericModelPOES PolarOperationalEnvironmentalSatellitePR PublicRelationsPRISM ParameterizedReal-timeIonosphericSpecificationModelRAID Redundany Array of IdenticalDiskss/c SpacecraftSCR SolarCosmicRaysSEC SpaceEnvironmentCenterSFE ServiceFunctionalElementSFR ServiceFunctionalRequirementSIP ServiceImplementationProposalSLAC StanfordLinearAcceleratorCenterSPENVIS SPaceENVironmenetInformationSystemSUR ServiceUserRequirementSIDC SolarInfluencesDataanalysisCenterSMS ShortMessageServiceSMTP SimpleMessageTransferProtocolSQL StructuredQueryLanguageSRD SystemRequirementsDefinitionSSN SunSpotNumberSTP Solar-TerrestrialPhysicsSWS SpaceWeatherServiceTEC TotalElectronContentUCL UniversityCollegeLondonUK UnitedKingdomUS UnitedStates(of America)UV UltraVioletWBMOD WideBandMODelWDC World DataCentreXDF eXtensibleDataFormatXML eXtensibleMarkupLanguageXSIL eXtensibleScientificInterchangeLanguage




1 Intr oduction

1.1 Purpose

This documentpresentsa proposalfor implementationof the serviceelementof an operationalEuropeanSpaceWeatherService(SWS). Specifically, it describesa genericarchitecturefor providing sucha ser-vice, considershow the serviceshouldacquire,processanddisseminatedata,andoutlinesthe operatingproceduresrequiredto manageit.

Useis madeof findingspresentedin otherreportsthathavebeenwrittenaspartof awide-rangingstudy[1]of the prospectsfor a future EuropeanSpaceWeatherProgramme(ESWP). The principal componentsof the study that have provided input to this documentare the Market Analysis report [2], the SystemRequirementsDefinitionDocument[3] andtheprototypingactivity carriedout in Work PackageWP434[4].TheMarket Analysispresentsa broadassessmentof the interestin andpotentialmarket for spaceweatherservicesandis usedto provide generalinput to thedesiredfeaturesof theSWS.TheSystemRequirementsDefinition documentprovidesa wealthof specificdetailsof the dataandmodellingneedsthat the SWSshouldmeet,andtheprototypingwork givessomeinsightsinto thepracticalobstaclesthattheSWSwouldneedto overcomeduringdevelopment.

1.2 Scope

1.2.1 Servicevs Data provision

A key distinctionis madein theMarket Analysisreport[2] betweenSpaceWeatherserviceprovidersanddataproviders. Themain functionof a dataprovider is to distribute raw, or nearlyraw, datato end-users.Serviceproviders,on theotherhand,processdatawith a targetedcustomerbasein mind in orderto providea tailoredandvalue-addedfacility focusedon theneedsof eachcustomeror classof customers.

The Market Analysisshowed that thedataprovider market is dominatedby, andhaslargely beendefinedby, NOAA/SEC. Any futureEuropeanSpaceWeatherProgrammewouldcertainlyprovidedatastreamsthatboth complementedandduplicatedthoseprovided by existing dataproviderssuchasNOAA. A notablefeatureof thedataprovider market is thatthereis a historyof freedataprovision, sothattheexpectationoftheusermarket is thatraw or nearlyraw datashouldbefree.A significantfindingof themarketanalysiswasthat‘it is doubtfulthatanotherbroad-scaledataprovider is neededor wantedin themarketplace’[2, section8.1]. Therefore,althoughanESASpaceWeatherprogrammewouldundoubtedlycontainanelementof rawdataprovision, theSWSshouldoffer more.

Serviceprovidersaremorenumerousthandataprovidersandhave hithertoconcentratedon specificclassesof end-users.Becauseof the extra value they bring to usersin sifting, organisingand presentingspaceweatherdata,customersarehappy to pay for theservicesofferedevenwhenthey cananddo retrieve rawdatafreeof charge.

1.2.2 Targetaudience

In thelight of theseobservationsandothers,theassumptionin this reportis thattheSpaceWeatherServiceshouldaim to bea coordinator, supporterandadvocateof serviceproviders,andaneducatorandsourceofinformationfor a wider public audience.It shouldassistserviceproviders in providing servicesfor end-users,ratherthanproviding themdirectlybecauseof theamountof sector-specificknowledgerequiredto do




soeffectively. This is not to saythatend-userswouldnotbeableto usetheSWS,but thatto makeuseof thedetaileddataor modellingservicesacertainlevel of backgroundknowledgewouldberequired,sothatif anend-userorganisationhadthisexpertisein-houseit couldberegardedasactingasits own serviceprovider.

This approachis similar to thatadoptedby nationalandinternationalmeteorologicalofficesin relationtoterrestrialweather. Meteorologicaloffices issueforecastsof variousparametersof the troposphericenvi-ronment;theselectionof parametersto forecastis madewith a wholerangeof potentialend-usersin mind,but the focusof a meteorologicaloffice is the environmentnot its specificeffectson users.For example,thelikelihoodof futurewarmdry weatheris importantto asupermarket company in decidingthevolumeofsaladvegetablesit needsto purchase,but ameteorologicalofficewouldnotpresumeto instructthecompanyon its purchasingpolicy.

Thesamefocuson describingandforecastingtheenvironmentshouldapplyto theSWS,only moresobe-causeof thefactthattheimpactof spaceweatheris mediatedby relatively complex technologicalsystems.Without detailedknowledgeof theoperatingcharacteristicsof thesystemsinvolved,a SpaceWeatherSer-vice could only give the broadesttype of ‘at risk’ warningwhich is unlikely to provide a sufficient basisfor taking potentially costly operationaldecisions. The vulnerability of a power distribution network toproblemsarisingfrom GeomagneticallyInducedCurrents,for example,will dependon factorssuchastheorientationof the network, the characteristicsof the transformersandthe positionandorientationof theelectrojets.Thebusinessof theSWSshouldbeto provide reliableandaccuratedescriptionsandforecastsof the environment,in this casethe electrojets,ratherthanattemptto provide a specificdecision-makingsystemfor power companies.

NotethattheSpaceWeatherServiceis not targetedat ‘scientists’in general– its focusis spaceweathernotspacephysics. It is assumedthatdatarelevantto scientificwork wouldbebetterplacedin long-termarchivesof scientificdata,for exampletheWorld DataCentresystemor theCDPPat Toulouse.Whereindividualscientistsor scientificteamsareconcernedwith spaceweatherper se thentheir requirementsof theSWSarelikely to matchthoseof companiesor institutionsproviding servicesto end-users.

The two setsof target users– spaceweatherserviceproviders and the broaderinterestedcommunity–are very different. Thereis a greatdistancebetweenthem, both in termsof the sophisticationof theirunderstandingof thephysicsof spaceweatherandof theirdataneeds.TheSWSmustthereforebedesignedwith carefulattentionpaid to the differentneedsof thesetwo target usergroups. It is on thebasisof thisanalysisthatstructureof theproposedSpaceWeatherServicepresentedin thisdocumentis developed.

1.3 Structure of the document

Section2 describesthegenericsystemobjectivesin moredetailandoutlinestheoverall systemarchitectureneededto meettheseobjectives. TheServiceUserRequirements(SURs)areidentifiedandenumeratedasaretheServiceFunctionalElements(SFEs)neededto meetthem.

The issuesraisedduring this overview arethenexploredin moredetail in subsequentsections:Section3dealswith all data-handlingissues– thedataformatsandthedataflows into, throughandoutof thesystem;Section4 considershow simpledataproductsshouldbe enhancedto provide a morevaluableservicetousers;Section5 dealswith theissuesof theuserinterfaceanduser-support;Section6 considerstheoutreachroleof theSWSin promotingawiderunderstandingof spaceweather;Section7 coversthemanagementandoperationof theserviceandidentifiestheresourcesrequiredto implementthesystem,in termsof equipmentandpersonnel.Throughoutthesesectionsfunctionalpropertiesthatarerequiredof theSFEsareidentifiedandtabulatedat theendof eachsectionasServiceFunctionalRequirements(SFRs)numberedwithin therelevantSFE;wherespecificproposalsaremadeasto how thesefunctionsshouldbeprovided,theseService




ImplementationProposals(SIPs)arealsolisted.

Section8 considerspossiblefuturedevelopmentsof theservice,whetherscientific,technicalor organisa-tional level. Finally therearetwo appendices– AppendixA presentsthedetailsof anestimateof thedailydataflows into theSWS,andAppendixB collatestheSIPsfrom theearliersectionsof thedocumentinto asingletablewith cross-referencesbackto theearlierdiscussion.

2 Ar chitecture overview

2.1 Philosophy

On thebasisof theanalysissummarisedin theprevioussection,it is clearthat theSpaceWeatherServiceshouldaimto satisfytwo, largelydistinct,setsof targetusers:spaceweatherserviceprovidersandabroaderinterestedpublic. The mostproductive useof resourceswill be to develop a systemthat is well suitedtomeetthespecificneedsof thesetwo verydifferentgroups,withoutdeliberatelytrying to developauniversalsystemthatseamlesslydeliverswhatbothgroupsrequire.

Section2.2 considersthe needsof the two setsof target users.Theseareusedto identify a collectionofservicefunctionalelementsthat theSWSshouldprovide. Theseserviceelementsarethendrawn togetherin Section2.3 into a singleschematicillustratingtheproposedsystemarchitecture.

2.2 UserRequirements

We briefly enumerateheretheprincipalrequirementsof thetargetusers,togetherwith functionalelementsof theSpaceWeatherServicethatmeettheseneeds;thesearesummarisedin Table2,whichassignsnumbersto theServiceUserRequirements(SURs)andtheir associatedServiceFunctionalElements(SFEs).

2.2.1 General

Althoughthe intentionis to caterspecificallyfor two very differenttargetgroupsof users,someneedsarecommonacrossall classesof user. Theseinclude:

• Datafrom multiple sourcesworldwideavailablein a reliableandtimely manner.• Sufficientdocumentationto supportunderstandingof how to usetheservice,andhow to interpretthe

dataproducts.• A servicethatcontinuesto develop.

Serviceelementsto meettheseneedsinclude:

• Datasupplymustbevia thenetwork from high-availability computingsystems.• A flexible schedulerto fetchdatawhenappropriate.• A comprehensive setof documentation,includingdatasetmetadata.• Accessto somelevel of humansupportwhenpre-prepareddocumentationis notenough.• Monitoringof predictionperformance.• Proceduresfor userfeedback.




2.2.2 Serviceproviders

Serviceprovidersareassumedto beknowledgeableaboutthephysicsof spaceweatherandcanbeexpectedto be runningoperationalmodelsof specificphenomenaof their own, driven by spaceweatherdata. Thisusergroupthereforehasthemoredemandingsetof needs,sincethey arelikely to needawiderangeof dataregularly andpromptly, andwill have well-definedinterestsin modelsandpredictionsof specificpartsofthespaceweathersystem.Their needsinclude:

• A straightforwardanduniform interfaceto multiple datasets.• Easeof identifyingrelevantdatasetsby classof application,typeof measurement,temporalresolution

of observation,promptnessof dataacquisition.• Accessto sophisticatedenhancedproductse.g. altitude-varying modelsof energetic particlefluxes,

spatiallyresolvedforecastsof GICs.• Historical informationneedsto beaccessiblefor post-incidentanalysisandfor modeldevelopment.• Ability to tailor apersonalisedlist of regulardataretrievals• Accessto informedadviceandscientifictechnicalsupport.

Furtherserviceelementsarerequiredto meettheseneeds:

• A genericbut comprehensive andaccessibledataformat.• A Data Dictionary to enabledatasearchesto be madeby classof application,region of interest,

featureof interest,promptnessof dataacquisition,predictionperiodetc.• Theservicemustprovideacoreof enhancedproductse.g.dataaggregation,modeloutputs,forecasts,

appropriatevisualrepresentation.• A dataarchive coveringanumberof monthsor years.• Personalprofilesfor FTPor e-mailingof data.• Humansupportthatcommandstherespectof usersby beingdemonstrablycompetentto handlede-

tailedqueriesaboutdataproductsatboththescientificandtechnicallevels.

2.2.3 Wider public audience

Theotherclassof userthat theSWSshouldtarget is thebroaderinterestedcommunity. This classcoversmany groups:informedcommentatorssuchastechnicaljournalists,decision-makersin politicsandindustry,potentialindustrialusers,the educationsectorrangingfrom schoolsthroughto universitiesandtechnicalcolleges,andthegeneralpublic. Theneedsof this rangeof groupsareratherdifferentfrom thoseof serviceproviders,sincethey arelikely to needconsiderablebackgroundinformationonthescienceof spaceweatherandareunlikely to have anongoinginterestin detailedmodellingwork. Their principalneedsare:

• Extensive andaccessiblebackgroundinformationon thescienceandapplicationsof spaceweather.• Attractive presentationof selecteddataproducts.

Theserviceelementsrequiredfor theseneedsare:

• On-linematerialintroducingthescienceof spaceweatherandtheimpactsit hasonvarioustechnolog-ical sectors:airlines,electricpower, geological/offshore,HF radio, insuranceandfinancialservices,military, satellites,tourism.

• Outreachmaterialse.g.posters,stickers,CD-ROMs,curriculumpacksfor schools,museumexhibits,newsletters.

• A graphicsengineto generateattractive plotsfrom data.




Thelastof theseelementswouldhavegeneralvaluefor all users,but thefirst two areverydifferentin naturefrom theotherscoveredearlier. Moreover, muchof the materialwill alreadyexist so thedevelopmentofthesesystemelementscouldbelargely independentfrom therestof theSWSanddevelopedseparatelyorin parallel. Thechallengewill beto bring thescienceandspaceenvironmentdatacloserto thepublic in areadily appreciatedaway (e.g. theAuroraWatchUK web-site[5]), without slipping into the role of beingjust ‘anotherbroad-scaledataprovider’.

2.3 Generic Ar chitecture

A high-level designof thecomputeranddatabasesystemsneededto meettheuserneedsoutlinedabove isshown in Figure1. Therearefive largecomponents:

Retriever Thoseelementsthatfetchrelevantdatafrom dataproviders.

Database Storagefor parameters,enhancedproductsandassociatedmetadata.

Modeller Thecollectionof modellingandforecastingprocessesthatwork ondatato generatemoresophis-ticateddataproducts.

Interface Themodulesconcernedwith handlinginteractionswith users.

Scheduler A genericmodulethat schedulesthe many systemprocessesthat needto happenat specifictimes.

Thedetailedfunctionalityof thesecomponentsisdealtwith in moredetailin Sections3,4and5. Throughoutthesedetailedconsiderationsthreeunderlyingassumptionsaboutthe appropriatecomputingtechnologiesapply:

• For easeof accessandeaseof use,by dataproviders,serviceprovidersandthe broaderinterestedcommunity, all systemsshouldbe basedon the public internetusing openstandardprotocolsandprogramminglanguageswhereverpossible.Thisfacilitatesinteropabilityandmaintenanceof systemsandsoftware.

• Any websitesshouldbe databasedriven,usingtechnologiessuchasPHP, Perl or Java Servlets;allthreehave public standardsandinterfaceswith all major relationaldatabases.Staticweb pagesareharderto maintain.

• To save on developmenttime andcosttheuseof off-the-shelfsoftwareshouldbeencouraged.Muchof thismaybeopensourcesoftware,whichshouldnotbeneglectedprovidedit is well-supportedandhasa robust pedigree.ProminentexamplesaretheApacheweb server, Perl for scripting,theGNUwget tool for retrieval or mirroringof datafrom datasuppliers.

3 Data Handling

Thecoreof theSWSis thedatadescribingthespaceweatherenvironment,plusancillarydatadescribingtheimpactof spaceweatherphenomenaontheimmediateenvironmentof ground-basedsystems.Therearethreerolesthatdataplaysin theSWS:asaninput to thesystem,asit is usedandstoredwithin thesystem,andasanoutputfrom thesystem;eachrole is consideredin turn.




SUR UserRequirement SFE ServiceFunctionalElement Comments1 Timely andreliabledata

from multiple sources1 Networkedandreliable

dataaccessDelivery by magneticmediais tooslow.

2 Retrieval scheduler Must beableto carryoutretrievalsautomatically.

2 Gooddocumentationofsystemanddata

3 On-linehelp

4 Comprehensive metadata A prerequisiteforsophisticateddataprovisionto users.

5 Humansupport For whenall elsefails.3 Consistentinterfaceto

multiple datasets6 Generic,comprehensive

andaccessibledataoutputformat

4 Easyto identify relevantdatasets

7 Datadictionary Having metadatais notenough– it mustbequery-able.

8 Yellow pagessystem For locatingdatasetsatremotedataproviders.

5 Accessto enhancedproducts

9 Dataaggregation.

10 Modelsandforecasts.6 Accessto pastdata 11 A local archive of relevant

dataImportantfor post-incidentanalysisandmonitoringofqualityof warnings.

7 Personalisedregulardataretrieval

12 Useraccountswithpersonalprofiles

8 Accessto informedadviceandscientifictechnicalsupport

13 Technicallyandscientificallycompetentpersonnel

A consultancy role

9 Backgroundinformationonscienceandimpactof spaceweather

14 On-lineintroductiontospaceweather.

Much pre-existing materialexists.

15 Outreachmaterials Posters,stickers,curriculummaterials,CD-ROMs,displays,museumexhibits; seeSection6.

10 Graphicalpresentationofselecteddataproducts

16 Graphicsengine

11 Continuousservicedevelopment.

17 Regularservicemonitoring

18 Userfeedbackfacilities On-lineandface-to-face.19 Mediumandlong-term

strategic planning

Table2: ServiceUserRequirementsandServiceFunctionalElements




Fetcher

databasescratch

databaseproduct

modeller process

modeller process

modeller process modeller process

modeller process

modeller process

accountsUser

StaticDocuments

databaseparameter

request

data

Scheduler

Modeller Interface

Help desk

databasemetadata

Database

Retriever

Users

��

Figure1: High-level designfor theSpaceWeatherService

3.1 Data input

Theessentialinput to theSWSis theflow of raw datafrom thespaceenvironment. For theSWSto serveits purposeit needsto acquirethedatanecessaryfor modellingandprediction,andit needsto do so in atimely mannerto meettheoperationalneedsof its users.Thissectionis concernedwith thedatato acquire;the matterof its timelinessis dealtwith in section3.2.3. The datafeed to the SWSwould needto sat-isfy theConsolidatedSystemMeasurementRequirements(CSMRs)identifiedin theSystemRequirementsDefinition (SRD)[3] andsummarisedin Table10,AppendixA.

3.1.1 Data sources

Oneimportantgroupof datainputsto theSWSarethosefrom ESWPspaceweatherinstruments.As partofa full programmeit is expectedthattherewill beanumberof instrumentsflown onESWPmissions,whosepurposeis primarily to provide datafor theSpaceWeatherprogramme.Theinterfacebetweentheseinstru-mentsandthegroundsegmentis coveredby anotherdocument[6], whichdescribesaconventionalfront-endservicethatgeneratescalibratedphysicalparametersfrom theraw instrumentdata.It is thecalibratedphys-ical parametersthatareof directinterestto theusersof theSWS,andwith whichweareconcernedhere.

It is assumedthateachinstrumentfront-endservicewill provide its own archive of theraw instrumentdataandashort-termcollectionof thederivedcalibratedphysicalparametersthatresult.A genericinterfacehasbeenspecified[7] for thedeliveryof theseparametersto theSWS.Theresponsibilityfor long-termarchivesof thecalibratedphysicalparametersfrom dedicatedESWPspaceweatherinstrumentscouldlie eitherwiththe instrumentfront-endserviceor with the SWS. Which option is chosenmay dependon the natureofthemissionsonwhich theinstrumentsareflown; adedicatedmissionis morelikely to have anautonomousarchive thana ‘piggyback’ instrumentonanothermission.

Theotherclassof datainputsto theSWSarethosefrom otherdataproviders,principally by thedelivery




of dataover the internet. Becauseof the variety of ways in which other remotesitesprovide datatherearea variety of dataretrieval mechanismspossible,several of which were implementedin the prototypeservice[4] developedaspart of this study. The dataretrieval capabilitiesthat the SWSshouldoffer areconsideredin thefollowing section.

3.1.2 Data Retrieval

Datadelivery (andrequest)mechanismscanbeclassifiedusefullyaccordingto two differentschemes.Bothschemesregardthedelivery of dataasa dialoguebetweena local anda remotesite,andbothanalysethisdialogueusingapairof classifiers.Thefirst schemeclassifiesmethodsby whostartsthedialogueandwhatdeterminesthe datacontent;the secondschemeclassifiesthe methodsaccordingto how the two partiescommunicatetherequestandthedata.

Thefirst classificationis by Starter/Determiner. In thiswe consider

• Who startsthedialogue– eithertheLocalsite(needingdata)or theRemotesite(providing data).• What determinesthedatathat is sent– this canbe oneof threepossibilities:theTime at which the

dialoguewasbegun,someinstructionfrom theLocal site,or at thediscretionof theRemotesite

Thesix possiblecombinationsof thesetwo classifiersareshown in Table3, whereeachis paraphrasedasaquestionandanexampleis givenof how thatmodehasbeenusedin practice.

DeterminerTime Local Remote

Starter

Local

“Tell me about now”Themethodusedwhenretrieving afile thatisupdatedregularly – thetime youaccessthefiledeterminesthedatayouget.(ACE datafrom SEC)

“Tell meabout this”This is theusualapproachwherethesiteneedingdataspecifies,usinganHTML form or byconstructinga filename,whatdatato fetch.(Geophysicalparametersfrom WDC for STP,Chilton)

“Tell me what youknow”A retrieval of opportunitywhere,for example,afileis updatedintermittently.A retrieval will getthelatestdata,but ‘latest’needbearnofixedrelationto thetimeofretrieval.(ClusterScienceDataSystem)

Remote

“This is what I knowabout now”This is themodeofregulare-maildistributions.(SIDC Monthly mailingof sunspotnumbers)

“What do you want toknow?”Therequestercouldputarequeston their localFTPsite,whichwouldberetrievedby thedataprovider at theirdiscretionto determinewhatdatato transferbyFTPPUT, for example.(No known examples)

“This is what I know”This is themodeof AlertsandWarningsdistributedby e-mail.(SIDCPRESTO alerts)

Table3: Starter/Determinerclassificationof dataretrieval




Thesecondclassificationis by Request/Delivery. Thisdividesdataretrieval methodsaccordingto theproto-colsusedto make theinitial contact(datarequest)andto deliver thedata.This classificationis open-endedin thesensethatnew messaginganddatatransferprotocolsaredevelopedover time andexisting protocolschangein popularity. Table4 showsthecombinationsof requestanddelivery for thethreemostwidely-usedprotocolsat thepresent– FTP, HTTP (web)andSMTP(e-mail). Notethatthetwo dimensionsof thetable

DeliveryFTP HTTP SMTP

Request

FTP StandardFTPretrieval Not used Not Used

HTTP Typically a largedatarequestis madeby HTTPthenretrievedfrom anFTPsite

Standardwebretrieval Datae-mailedfollowingawebrequest

SMTP E-mail requestthenFTPretrieval e.g.NDADS-ARMSatNSSDC

E-mail triggersHTTPretrieval

Mail respondere.g.GeomagneticInformationNodes

Table4: Request/Delivery classificationof dataretrieval

areonly theprotocolsusedfor thestartandendof thedialoguebetweenlocal andremotesites.For exam-ple, theHTTP-FTPcombinationwould typically bea two-stageprocessin which therequestercompletesawebform to requestdataandis returneda webpagegiving the locationof a newly createdfile on anFTPsite.Parsingthiswebpageto extracttheFTPlocationwouldenabletherequesterto startanFTPsessiontoretrieve thedesireddata.

Two of the combinationsareshown as‘Not used’,not becausethey areimpossiblebut becausetherearebetterwaysto achieve thesameends.For example,onecould imagineanFTP-SMTPschemein which afile indicatingthedatarequiredwasdepositedonaremoteFTPsite,andsubsequentlytheremotesitewoulde-mailthedatato therequester. This is entirelyfeasiblebut thesameeffect is morelikely to beimplementedin practiceusingtheconventionalSMTP-SMTPcombination.

TheSWSwill needto beableto retrieve datausingmany of themethodsdescribedin Tables3 and4, andconsequentlytheSWSmustincludeseveralcapabilities:

• Useof anFTPclient to retrieve data.• Useof anHTTP client to retrieve data.• An e-mailsystemfor sendingdatarequestsandparsingreceiveddatae-mails,

All of thesemustbeableto functionwithoutoperatorinterventionandbecapableof beinginvokedautomat-ically on a predefinedor adaptive schedule.Thee-mail requirementscanbemetby commonlyusedUnixtoolssuchassendmailor procmail in conjunctionwith thestandardmail client mail. It is worth emphasis-ing thatthewidespreaduseof e-mailasadataexchangemechanismis not alwaysappreciatedby corporatemanagements.Thisbringsthedangerof corporatee-mailpoliciesor systemsthatassumethate-mailis usedonly for person-to-personcommunicationandpreventtheuseof e-mailfor automaticdataexchange.

The FTP andHTTP retrieval requirementsalso includeautomaticinvocationandprocessingof retrievedresults. Thesecan be met by varioustools in the Unix environment; a prominentexample is the opensourcesoftwaretool wget thatprovidesrecursive FTPandwebsitedownloadandmirroring facilities. The




schedulingdemandscanbe met by standardoperatingsystemschedulingtools suchasUnix cron or theWindows NT scheduleservice.For thehighesttime resolutionretrievalsoperatingat minuteor sub-minutefrequencies,a continuousmonitoror server processwouldbemoresuitable.

SFR SFRDescription SIP SIP Description1.1 Allow dataretrieval usingtheFTP, HTTP

andSMTPprotocols1.1.1 UsetheFTP/HTTPagentGNU wget for

FTPandHTTP retrieval andmirroring1.1.2 On Unix, therequirementsaremetby

standardMail Delivery Agentssuchassendmailin conjunctionwith procmailandthemail client mail.

1.2 Dataretrieval mechanismsshouldbesufficiently flexible to copewith thesystemsof many differentdatasuppliers.

1.2.1 Provide for dataacquisitionusingallmethodsfrom Table3 excepttheRemote/Localcombination.

2.1 Dataretrieval mustbeableto proceedon apre-determinedschedulewithouthumanintervention.

2.1.1 TheUnix cron or Windows schedulefacilitiesshouldbeusedto startretrievaljobsasrequired.

3.2 Data Storageand Processing

Therearea numberof key datastorageandprocessingissuesthattheSWSmustaddress:

• Local versusRemotedata• Datahistory• Scheduling• Databasesystems• Dataformats• Metadata

3.2.1 Local versusRemotedata

TheSWSis not primarily intendedto beanarchive of spaceweatherdata.Thereareseveralgoodreasons,however, for keepingcopiesof retrieveddatalocally for anextendedperiod:

• Necessity– For any dedicatedESWPspaceweatherinstrumentsit is assumedthattheSWSwill actasthedefinitive archive of thecalibratedphysicalparametersthat they generate.Notethat thepresenceof theselocal archives is not materialto the modellingandforecastingfunctionsof theSWS,sincethedataretrieval interfacesshouldnotdiscriminatebetweenlocal andremotedata.

• Buffering– To guardagainstnetwork delaysimpactingthepredictionfunctionsof theSWS,it wouldbe useful to fetch dataassoonas it is available ratherthanon a just-in-timebasis. This could beregardedasa cachingratherthanarchiving function, but given the rangeof modelsandpredictorsthat theservicewill run on differenttimescales,thecacheddatamight needto bekept for weeksormonths.

• Review – Someuser requirementsare for post-event analysisup to several monthsafter a spaceweatherevent. Examplesincludethe evaluationof insuranceclaimson satellitefailuresandairlinemonitoringof radiationdosesto crew membersfollowing extremeevents.




• Development– Models and predictiontools needdatafor their development,and for them to beeffective underoperationalconditionsthey needto be developedandtestedon datathat resemblestheenvironmentin which they will berun in earnest.Having accessto realisticsnapshotsof thedataenvironmentovera rangeof timesis thereforeof greatvalueimportantwhendevelopingpredictors.

For all thesereasonsit would be prudentfor the SWS to maintainsubstantiallocal databasesof physi-cal parametersretrieved from remotesources.AppendixA presentsan estimateof the daily volume ofdatarequiredto satisfythesystemmeasurementrequirementsidentifiedin theSystemRequirementsDoc-ument[3]. The conclusionis that the daily datarate is of the order200-1000Mb,equivalent to roughly70-400Gbperyearof archived data.Thesedataratesandvolumesaremanageablewith relatively modestmodernhardware;for reliableandquick on-lineaccessa RAID (RedundantArray of IdenticalDisks)sys-temcanbeemployed,with a larger volumeof near-on-line datastoredon a modularandexpandabletapestoragesystem.

TheSWSwould provide accessto routinerunsof variousmodelsandpredictors,executedeitherlocally orat co-operatinginstitutions.Theoutputsof thesemodelrunsmight alsobeworthpreservingin many cases,particularlyif their productionwascomputationallyexpensive. Theissuesregardingmodelsareconsideredin depthin Section4.2.

SFR SFRDescription SIP SIP Description11.1 Maintainlocaldatabasesof retrieveddata,

estimatedto accumulateat200-1000Mb/day.

11.1.1 UseaRAID systemwith capacityof atleastseveralmonthsof datafor immediateon-lineaccess,with near-on-linetapestorageholdingtheremainingdata.

3.2.2 Data History

Relatedto theextentof localarchiving of parametersandproductsis theability to provideadatahistory. Inorderto assesstheeffectivenessof predictionalgorithmsandoperationaldecisiontools,serviceprovidersandtheir userswill needto beableto recreatethedataenvironmentat the time thatpredictionsandopera-tional decisionsweremade.

For example,neuralnetworks frequentlyrequirecompletesetsof desiredinputson which to operate,andalsothattheinputsbeof goodquality. Cleanandcomprehensive historicaldatais oftenavailable,but if theoperationaldataenvironmentin which a neuralnetwork predictorhasto executehasmany datadropoutsor poorquality valuesthenthepredictormayperformmuchlesswell. Indeed,anotherpredictordevelopedwith explicit allowancemadefor thelikely qualityof operationalinputdatamayprove superiorin practice,althoughinferior underidealconditions.

Anotherexamplewould bea post-mortemon anoperationaldecisionmadeby anend-userin thelight of aspaceweatherprediction. If thatprediction,or otherrelatedpredictions,wereto be re-runafter theevent,thenif therelevant input datahadbecomemorecompleteor hadbeencorrectedthenthepredictionresultswould not matchthoseavailablewhenthe original operationaldecisionsweremade. This would clearlyinvalidateany post-morteminvestigation.

Beingableto recreatethedataenvironmentis a non-trivial requirementsincetheavailability andqualityofdataarelikely to changeastime passes.Thereis a tendency for dataproviders in the spacedatafield todivide into two camps– thosewhich concentrateon recentdata(e.g.SEC)andthosewhich provide long-termarchives(e.g.NGDC). Whena long-termarchive is constructedit is usuallythecasethatonly thebestqualityandmostcompletedatasetis stored.This will notbethecasefor aSpaceWeatherarchive.




Tobeableto recreatetheoperationaldataenvironmentimposestwo significantconditionsonthecapabilitiesof theSWSdatastoragesystems:

• Dataupdatesmustbenon-destructive. In particular, if a parameterhasits valuecorrectedat a laterdate,or anew versionof adatasetis issued,theuncorrectedor olderversionsmustbepreserved.

• All dataitemsretrievedandarchivedby theSWSmusthave a timestampof retrieval associatedwiththem,in additionto thetimestampof observationthatthey customarilyhave.

Providedtheseconditionsaremet,it shouldbepossibleto recreatethedataenvironmentat any giventime.Note that this providesanotherreasonfor maintaininga local archive, sinceit is only with local datathattheseconditionscanbeguaranteed.

SFR SFRDescription SIP SIP Description11.2 Storedatasothattheoperationaldata

environmentat any point in time isrecoverableatany latertime.

11.2.1 Updatedatanon-destructively.

11.2.2 Timestampall datafiles or valueswiththeir time of retrieval.

3.2.3 Scheduling

Thetimelinessof dataretrieval, processinganddistribution is vital for a spaceweatherservice.Datamustberetrievedonavarietyof timescales,rangingideally from every few secondsfor magnetometerdataup toonly a coupleof timesa yearfor spacedebrisdistributions. Thequantityanddiversityof thedatarequired(seeAppendixA) necessitatesautomaticretrieval for the majority of datasets.Many userswill want toreceive datafrom theSWSona regularbasisaswell, soautomaticdistribution mustalsobepossible.

Besidesretrieving datafor onward distribution, spaceweatherparameterswill be usedto drive predictiontoolsandothermodels.Onecanenvisagethesebeinginvoked bothon demandandroutinely. In the lattercase,themodelor predictionprocesseswill alsoneedto bestartedandmanagedautomatically.

Theseschedulingdemandscanmostlybemetby standardoperatingsystemschedulingtoolssuchasUnixcron or theWindows NT scheduleservice.

SFR SFRDescription SIP SIP Description2.2 Dataprocessinganddistribution mustbe

ableto proceedon apre-determinedschedulewithouthumanintervention.

2.2.1 TheUnix cron or Windows schedulefacilitiesshouldbeusedto startdataprocessingor distribution jobsasrequired.

3.2.4 Databasesystems

The internalorganisationof the datastoredby the SWSshouldnot have any functionalconsequencesforusersof the system. The choiceof databasesystemwill have significant impact, however, on the easewith which dataareingested,processedanddistributed. Threebroadapproachesarepossible:the useofa relationaldatabasesystem,anobject-orienteddatabasesystem,or storinga collectionof structureddatafiles.




Relational databases Relationaldatabaseshave theadvantageof beingamaturetechnologywith awell-definedandwidely usedquery language,SQL [8, 9]. By placing all datain a set of tablesthey renderstraightforward theprocessof selectingdatafrom multiple sourcesaccordingto complex criteriabasedonthevaluesof dataitems.This capabilityhassomepotentialbenefitfor theSWSgivenits role in managingdataacquiredfrom numeroussources,althoughsuchcontent-leadqueriesaremorelikely to be a featureof a researchactivity thanan operationalservice. A drawback to the relationalapproachis that thereislittle directprovision for metadatain relationalsystems.This is not necessarilya problemfor simpledataor datathat is well understoodby all users,but this is not thecasefor spaceweather. Spaceweatherdatais relatively complex andhasextensive metadatanecessaryfor its properinterpretation,andfor this reasonrelationaldatabasesarenot thebestdatastoragemethodfor theSWS.

Object-oriented databases Onemethodfor bindingmetadataanddatamorecloselyis to useanobject-orienteddatabase,which representsdatanot astuplesin tablesbut asobjectswith associatedbehavioursandproperties.This is a muchnewer technologybut onewith severaladvantages.Thedatamodelcorre-spondsbetterto theworld thandoesthe relationaldatamodelandbecauseinformationaboutanobjectismaintainedwithin a singledataentity, dataaccesscanbe muchquicker for standardqueries. Moreover,theobject-orientedapproachto datastoragefits well with modernobject-orientedprogramminglanguages,thussimplifying dataaccesscode.This would beanadvantageto theSWSin thedevelopmentof codeformodellingandprediction. Therearesomegeneraldrawbacks,prominentbeingthe absenceof a standardquerylanguagealongthelinesof SQL. In thecaseof theSWS,however, thereareotherreasonswhy object-orienteddatabasesmight not beappropriate.TheSWSwill mainly beretrieving datasolelyon thebasisofits time of observation,andfeedingit eitherto modelandpredictorprocessesor direct to users.Themodeof dataqueryis thusonedriven by thevalueof anattribute of anobservation, ratherthanthe relationshipbetweenobjects.This is a form of querythatobject-orienteddatabasesdo not handlewell, sincethey caninvolve a trawl throughmany objects.Moreover, thematterof presentingdatato usersin a consistentandeasilycomprehendedfashionis not addressed,andthis is a critical considerationfor the usability of theservice.Althoughobject-orienteddatabasespresenta betteroption thantherelationalmodel,they arestillnot ideal.

Structur ed datafiles Thefinal optionis theuseof structureddatafiles. Althoughthis might beregardedasan old-fashionedapproach,for the purposesof the SWSit is the mostsatisfactory. The dataretrievalsnecessaryfor theoperationof theSWSwouldnotusuallybedata-driven;almostall datawouldberequestedby time of observation,sodatafiles thatcanbeidentifiedasholdingobservationsfrom aninstrumentfor aspecifictime rangearesufficient. Anotheradvantageof holding datainternally in files is that dataoutputfrom theSWSwill needto usea structuredfile format. With eithera relationalor object-orientedsystemextracteddatawould needto beappropriatelypackagedbeforedistribution, but if structuredfiles areusedinternally by the SWSthenpreparingdatafor distribution is greatlysimplified. Onekey requirement,asdiscussedpreviously, is that themetadatabeeasilyretrievedwith thedata.Thereareseveralcandidatefileformatsthat enableoneto keepmetadataanddatatogether, andtheir relative meritsarediscussedin thefollowing section.On balance,theuseof structuredfiles is themostsuitabledatastoragemethod,with anobject-orienteddatabasesystemasthesecondchoice.

SFR SFRDescription SIP SIP Description11.3 Storedatain amannerthatsimplifies

ingestion,processinganddistribution.11.3.1 Storedatain structuredfiles ratherthan

relationalor object-orienteddatabasesystems

4.1 Storedatain amannerthatfacilitatestheassociationof datawith thenecessarymetadatafor its properuse.

4.1.1 Storedatain structuredfiles withintegratedprovision for metadata




3.2.5 Data formats

If a structuredfile format is to beusedfor internaldatastoragein theSWS,thena choiceof formatmustbe made. The oldestandcurrentlymostwidely usedstructuredfile format in the spaceweatherfield isthebinaryCommonDataFormat,CDF [10]. Recentlyotherformatshave beendevelopedbasedon XML,the ExtensibleMarkup Languagewhich is becomingwidely usedfor dataexchangein many disciplines.The formatsproposedthat are relevant to sciencedatain generalandspacephysicsin particularare theExtensibleDataFormat,XDF [11] andtheExtensibleScientificInterchangeLanguage,XSIL [12]. Someof theadvantagesanddisadvantagesof theCDF, XDF andXSIL formatsaregivenin Table5.

The variousadvantagesanddisadvantageslisted in Table5 vary in importance.The fact that XSIL is alightweight formatconsequentlymakingmuchgreaterdemandson dataaccessprogramsis crucial,partic-ularly if XSIL wereto beusedasanoutputformatto users.On thatconsiderationalone,XSIL is probablynot suitable.ThedecisionbetweenCDF andXDF is muchlessclearcut. CDF hastheadvantageof beingwell-established;XDF, beingbasedonXML, canbeseenas‘coming technology’andis likely to bewidelyadoptedgivenits origin (likeCDF)with NASA. TheXML basisof XDF alsomakesdatamoreeasilyacces-sibleby non-proprietarysoftware,whichmight bea considerationfor theserviceprovidersthattheSWSisintendedto serve. For CDF, andpotentiallyfor XDF, theNASA pedigreeis significantsinceit follows thatassociatedsoftwareis freeandis likely to besupportedover many yearsoncein widespreadusefor NASAdatasets.

SFR SFRDescription SIP SIP Description11.3 Storedatain amannerthatsimplifies

ingestion,processinganddistribution.11.3.2 Storedatain CDF or XDF formatfiles. At

thetimeof writing CDF is thebetteroption,but thismaychangerapidly overtime.

3.2.6 Metadata

Theneedfor goodmetadatahasbeennotedpreviously. Metadatais necessaryfor theproperunderstandingof dataso shouldaccompany datawhenstoredin or exportedfrom theSWS. Sufficiently comprehensivemetadatais alsoneededwhenautomatingdataprocessingor displayprocedures.For example,theCDAWlibdisplayandretrieval tools thatwork with theCDF dataformatrely on themetadatafor a datasetscomply-ing with the ISTP/IACG guidelines[13]. Without thedegreeof metadatacompletenessenforcedby theseguidelinesthe CDAWlib software will not be able to handlethe dataproperly. This aspectof metadatafunctionalityhasbeencoveredby SIP4.1.1.

Metadatais alsoa valuableresourceto enableusersto locatethedatathey needby specifyingpropertiesofthe dataratherthanwhereit canbe found, for exampleselectingby a region of interest,type of particle,energy range,or by nameof parameter. To beableto usemetadatain this fashionit is notsufficient for it tobestoredwith thedatain CDF or XDF files, it mustalsobedatabasedsothatit is searchable.

TheSpaceWeatherServiceprototype[4] implementsadatabaseof metadatacalledtheSpaceEnvironmentYellow Pages.This demonstratestwo formsof metadatathatareimportantto storein a readily retrievableformat: metadatadescribingtheform andcontentof thedata,andmetadatadescribinghow to retrieve thedata– its location. In thefull SWSbeingoutlinedin this document,muchof the locationmetadatawouldpoint to a local sourcewithin the SWS itself, but this would not eliminatethe needfor it. It would bevaluablebothasaninternalresourceto theSWSwhenretrieving datafrom remotedataproviders,andasaresourcefor usersto enablethemto identify theseremotedatasources.




Format Advantages Disadvantages

CDF – Widely used.– Many toolsandlibrariesfor

manipulatingfileson manyarchitecturesandoperatingsystems.

– Supportfor Java andIDL accessto data.– Standardsfor spacesciencemetadata

defined(ISTP/IACG guidelines[13].– Developedandusedwidely by NASA.– Widely userin Europee.g.Cluster,

Equator-S, AMPTE, SPENVIS.

– Thefile structureis relativelycomplicated.

– A privatebinaryformat,sotied toproprietarydataaccesslibraries.

– Formathaschangedbetweenversions,forcing changesin installedsoftware.

XDF – Basedon theopenstandardof XMLandthemetadata(at least)is ASCII, soeasilyinspected.

– Definedby anXML DTD (DocumentTypeDefinition)sofilesareparse-ableby generalXML software.

– Softwarelibrariesfor PerlandJava toaccessdata.

– XML featureof referenceto orinclusionof theDTD allows theformatto evolve in amannervisible withinindividual datafiles,hencevisible tosoftware.

– Developedandsupportedby NASA.

– Not widely usedyet.– No standardsdefinedfor required

metadata.– Thefile structureis relatively

complicated.

XSIL – Basedon theopenstandardof XMLandthemetadata(at least)is ASCII, soeasilyinspected.

– Definedby anXML DTD sofilesparse-ableby generalXML software.

– Well coupledwith avisualisationtool.– Softwarelibrary for Java to accessdata.– XML featureof referenceto or

inclusionof theDTD allows theformatto evolve in amannervisible withinindividual datafilessovisible tosoftware.

– Lightweightformatto beextendedon aperdisciplinebasis.

– Not widely usedyet.– Lightweightformatimposesgreater

demandson processingcode.

Table5: StructuredFile Formats:Advantagesanddisadvantages




Metadataqueriesaretypically valuedriven,unlike thoseto beexpectedof thespaceenvironmentdataitself.It is thereforeappropriateto userelationaldatabasetechnologyfor storingmetadata,following the sameargumentsrehearsedearlierin Section3.2.4whendiscussingthedataitself. Using a centraliseddatabasefor thelocationmetadatais alsoadvisable,sinceit is theform of metadatathat is mostvolatile soneedstobeeasyto amendasdatasourcesaremovedor changetheir format.

SFR SFRDescription SIP SIP Description7.1 Allow thedataholdingsto besearchedby

metadataattributes7.1.1 Extractdatafilemetadataandstorein a

relationaldatabase– a ‘Data Dictionary’8.1 Allow easyconfigurationandaccessto

locationmetadata8.1.1 Storelocationmetadatain a relational

database– a ‘Yellow PagesDirectory’.

3.3 Data Output

3.3.1 Formats

The SWSwill not have control over the format of datafrom otherdataprovidersbut candeterminetheformatof internaldataanddatait distributes.Oneof thekey userneedsgivenin Table2 is for a consistentandaccessibledataformatin whichtodistributethediversedataholdingsof theSWS.TheleadingcandidateformatsareagainCDF andXDF sincethey integratedataandcontentmetadataandarewell supportedbylibrariesandtoolsfor readingandmanipulatingthedata.Theseformatsarewell suitedto professionalusersof spaceweatherdata,but theremaybedemandfrom morecasualusersfor a more‘approachable’ASCIIformat. Provision of simplified accessmight be worthwhile for a selectionof dataproducts,but shouldnot be thefirst priority. Theuseof theXML-based,andhenceASCII, XDF formatmight be sufficient tomeetthe needsof both professionalandcasualusers,particularlysincethe XDF format doesprovide thefacility for referencingasubsidiarydatafile from theformalXDF document.Thissubsidiaryfile couldthenbea plain ASCII datafile suitablefor casualretrieval or inspection.More sophisticateduserswho requiremetadatawould fetchthemorecompleteXDF file instead.

3.3.2 Methods

As with datainput to theSWS,it is necessaryto considerthemethodsby whichuserswouldneedto retrievedata. Theanalysisof dataretrieval methodspresentedin Section3.1.2appliesequallywhenviewed fromtheotherside,with theSWSasthe ‘remote’ data-providing site ratherthanthe ‘local’ dataretriever. Thesamereasoningapplies– theSWSshouldmake it possiblefor usersto retrievedataby avarietyof methods,to accommodatetheir variousneedsandoperationalconstraints.

Examiningfirst the Starter/Determinerclassificationof dataretrieval (Table3), the SWSshouldoffer allof the combinationsexceptRemote/Local.The Local startedcombinationsareall simpleto provide; thedistinctionbetweenTime or Remote(i.e. SWS)determinedretrieval will dependon thetypeof databeingretrieved,andtheLocal determinedoptionis thatcommonlyprovidedby dataaccessdrivenby webforms.TheRemotestartedcombinationsof Time andRemotedetermineddata,correspondingto regularmailingsandirregularalertsrespectively, shouldbeofferedfor appropriatedataproductssuchasdaily Kp forecastsandCME warnings.

A numberof thedatadelivery mechanismsshown in Table4 shouldalsobeoffered.Giventhedominanceof the web for datadelivery the principal methodshouldbe to useHTTP for requestanddelivery. Thusthecoreof theSWSwill bea websiteof dynamicpages,offering a a rangeof forms-driven accessto the




dataholdingsandotherservices.Themarket-leaderfor webserversis theApacheserver [14], which haswell-integratedsupportfor a rangeof scriptinglanguagessuchasPerlCGI andPHP.

Providing ane-mailinterfaceto thesamedatarequestsoftwarewouldbestraightforwardandshouldalsobedone.Thedelivery of databy e-mailcanbeaccomplishedusingstandardUnix toolssuchassendmailandmail, anduseof theseor similar toolscanbeabstractedfrom by using,for example,thePerlMail::Mailermodule.

UsinganFTPclient to deliver datato auser’s FTPsiteis conceptuallynodifferentfrom delivery by e-mail,andcanbeachievedwith variousdegreesof sophistication:simpleshellinvocationof acommand-lineFTPclient, manageduseof a command-lineFTP client with the expecttool, or programmaticuseof the FTPprotocolwith thePerlLWPor Net::FTPmodules(or similar for otherlanguages).

The provision of passive FTP accessvia an FTP site shouldbe a lesserpriority. The natureof the FTPprotocolis that it providesaccessto datavia a hierarchy. Giventhatit is proposedto storetheSWSdatainstructuredfilessuchaccesswouldbepossible,but thedatastoragehierarchydoesnotnecessarilycorrespondwith theconceptualrouteby whichusersmightwantto reachthedata.Providing morethanonerouteto thesamedatais alsomadedifficult.

Therearecertainclassesof dataproductthat might requiremorespecialiseddataretrieval options. Oneexamplefrom theMarketAnalysisreportis theapplicationof auroraltourism;users,whetherindividualsortour companies,might welcomea facility by which anSMSmessagecouldbesentto their mobilephonesgiving short-termalertsthatanauroraldisplaywaslikely.

3.3.3 Bandwidth

Estimatingthevolumeof datathatwill flow outof theSWSis moredifficult thanestimatingtheinflows,ascarriedout in AppendixA, becauseit is entirelydependenton thenumberandtypeof usersof theservice.It is possibleto make conservative estimateshowever, to arrive at an estimateof the network bandwidthrequiredto supportlikely usage.

Assumingall accessto datais via a nominal ‘hit’ on a web server, a generousestimateof the numberofhits/daywould be

��anda similarly generousestimateof the transferperhit would be 50 KBytes. This

implies � � �� KBytesperdaywhichis roughly60KBytes/sor 500Kbits/s.Allowing afactorof 10variationmultiplier betweenaverageandpeakrequirementswould imply a bandwidthof 5Mbits/s. This level ofnetwork capacitycanbemet throughcommercialleasedlines,which arewidely availableat capacitiesupto 155MBits/s. Thecostof leasedlinesvarieswidely in theEU (see[15]) but illustrative costsareincludedin Section7.3.




SFR SFRDescription SIP SIP Description1.3 Datadistribution mechanismsshouldbe

sufficiently flexible to copewith theneedsof many differentusers.

1.3.1 Provide for datadistribution usingallmethodsfrom Table3 excepttheRemote/Localcombination.

1.4 Allow datadistribution usingtheFTP,HTTP andSMTPprotocols

1.4.1 Offer active FTPdelivery of data(PUToperation)by usingappropriatesoftware,suchasexpector thePerlLWPandNet::FTPmodules.

1.4.2 Offer dynamicHTTP accessby runningaforms-drivenwebserviceon theapachewebserver [14].

1.4.3 Offer e-maildatadistribution usingastandardMail TransportAgentsuchassendmail, Mail UserAgentsuchasmail,or wrappedby higher-level softwaresuchasthePerlMail::Mailer module.

1.4.4 Considernovel datadistribution channels(e.g.SMS)for selectedproducts(e.g.alerts)

1.5 Providesufficient network capacitytoallow promptaccessto SWSdataproductsby users

1.5.1 Procurea leasedline connectionwithbandwidthof at least5Mbits/s.

6.1 Outputall datausingagenericandaccessibledataformat,for consistency ofaccess.

6.1.1 Outputdatain CDF or XDF formats.

6.1.2 FacilitatesimpleASCII outputbystructuringXDF files sothatthedataelementhassufficient internalstructureforit to beintelligible in theabsenceof themetadataelements.

4 EnhancedData Products

TheSWSasdescribedsofarwouldprovideaconsistentinterfaceto multipledatasetsproviding observationstaken betweenthe surfacesof the Sunandthe Earth. To offer a quality serviceto spaceweatherserviceprovidersandthenceto commercialusersmoreis required;it mustbepossiblefor simpleobservationalandindex datato becombinedto producemoresophisticateddataproducts.Threegenerictypeof enhancementarediscussedin thissection:datapipelining,modelling,andgraphicaldisplay.

4.1 Data pipelining

Thereare a numberof simple operationsthat can be appliedusefully to many numericaldatastreams.While it is not thepurposeof theSWSto providegenericdataprocessingfacilitiesfor its dataoutputs,someoperationsarelikely to bewantedsufficiently oftenasto bedesirableto offer. Thetwo classesof operationsthatshouldbeprovidedarefiltering andaggregation.

Filtering by time will be implementedasa matterof course;mostdatarequestswill be for a specificdataproductdescribingsomefeatureof the spaceweatherenvironmentat a specifictime or during a specific




time interval, whetherpast,presentor future. Adding thefacility to filter by datavalueswould allow usersto fashionpersonalised‘Alerts’; for example,to e-mail theAp valuesto theuserwhenthey exceeda giventhreshold.Morecomplicatedscenarioscanbeimagined,whereuserscouldconfigurethesystemsothatonedatastreamwouldberetrievedcontingentonsomethresholdbeingreachedin anotherdatastream.

Variousdataaggregationfunctionsshouldalsobe offered. Theseincludetaking themean,median,mini-mum, maximumor sumover someregular repeatinginterval of the domainof interest. This aggregationcouldbeover time for a simplescalartime series,or on a spatialgrid for observationsdistributedover anareaor volume.

SFR SFRDescription SIP SIP Description9.1 Enableusersto filter databy time 9.1.1 All datarequestsshouldspecifya time

range.9.2 Enableusersto applysimpleaggregating

operationsto datastreams.9.2.1 Implementasuiteof aggregationfunctions

to computemean,median,minimum,maximumandsumon arbitrarydatafieldsover regularrepeatingintervalsof timeorspace.

4.2 Models

Theprovisionof models,of varyingcomplexity, is themostsignificantwayby whichtheSWScanenhancethesimpledatastreamsthatareat its disposal.In thecourseof determiningthemeasurementrequirementsfor theSpaceWeatherProgramme,theSystemRequirementsDefinition (SRD)document[3, AppendixB]considersthemodelprocessesneededto generatethedesiredinformation. Thesemodelsor processesareclassifiedasMature, Immatureor Speculative; a modeldescribedas ‘Mature’ canbe regardedas beingalreadyin a positionto give informationof value. It is thereforeincumbenton theSWSto provide thesematuremodelsasaminimuminitial offering. Theothermodelsmentionedby theSRDshouldbeconsideredfor additionto thesystemwhenthey aresufficiently mature,andthereis somediscussionof thisin Section8.

4.2.1 Requiredmodels

Table6 presentsdetailsof all the maturemodelor processesidentifiedin the SRD document.The tablebreaksthesedown in to four groupings:plasmaandradiationenvironment,solid bodyenvironment,mag-neticenvironment,atmosphericenvironment.For eachmodelthetableshows:

• Timeof operation: F(orecast),N(owcast),H(indcast)• Modeof operation: S(tandard),C(ustom).Standardoutputswouldbegeneratedroutinelyandstored;

customoutputswould requireuserinput to generatethedesiredoutputsondemand.• Updateperiod• Thenumbersof theSRDUserRequirementsthatthemodelis designedto helpmeet.

SFR SFRDescription SIP SIP Description10.1 Offer awide rangeof modelsand

forecastingtools.10.1.1 Themodelsin Table6 shouldall be

provided.




Table6: Initial setof modelsfor theSWSto provide. Thecolumnsheaded‘T’, ‘M’ and ‘UR’ show the operationTime, operationModeandtheSRDUserRequirementnumbersrespectively.

Model T M Updateperiod

UR

Plasmaand Radiation EnvironmentClimatic modelsof solarprotonsfrom SolarProtonEventse.g.JPL-91

F S 6 monthly 1

Climatic modelsof GCRe.g.CREME[16] or CARI [17] H S 6 monthly 3Magneticfield rigidity modelsto mapobservedfluxesofGCR,SCR,solarprotons,energetic ionsandneutronstoarbitraryorbitsor spacecraftpositions.

NH C 30minutes 13151920

Rigidity cut-off models(e.g.Stormer[18] or SheaandSmart[19]) to mapobservedfluxesof � 10MeV ionstoaltitudes,routesandlocationsof aircrew, avionicsequipmentandspacecraftlaunches.

NH C hourly 2 322

Radiationtransportmodel(e.g.GEANT[20]) to map� 10MeV ionsto aircraftaltitudesandroutes.

NH C hourly 1315

Magneticfield model(s)(e.g.thoseof Tsyganenko [21]) tomapobservedfluxesof electrons,ionsandprotonsof allenergiesalongfield linesto arbitraryorbitsandspacecraftlocations.

NH C 1 minutetohourly

1315

Predictionof SolarProtonEventsfrom CME andflaredetection.

F S 1 minute 1 18

Plasmaenvironmentmodelling(e.g.Salammbo[22] orMSFM) to fill gapswheregoodplasmameasurementsarenotavailable.

NH C 3 hourly 1315

Solid Body EnvironmentClimatologicalmodelsof debris(e.g.MASTER-97[23] orIDES)andrandommeteoroids.

FNH S 6 months 13151819

Gravitationalmodellingof debrisandrandommeteoroidstoLow EarthOrbit.

FNH S 6 months 13151819

Orbit propagationmodellingof largedebris. F S 6 months 14Modelsof meteoroidstreams[24]. FNH S daily 1314

151819

Magnetic EnvironmentPredictionsof Kp from solarrotationout to � 27 days. F S daily 7Predictionsof Kp from CMEsor flaresout to 2 or 3 days. F S hourly 4 7Predictionsof Kp from solarwind 1 or 2 hours(Lund [25, 26]or Costello[27] models).

F S 15minutes 7

Predictionsof Dst from solarrotationout to � 27 days. F S daily 7Predictionsof Dst from CMEsor flaresout to 2 or 3 days. F S hourly 7Predictionsof local �� from CME detection F S 15minutes 4Estimatesof �� by interpolationfrom local fieldmeasurements.This is probablybetterdonedirectly by enduserswith their serviceprovidersbecauseof thehigh temporalresolutionrequired.

NH C 10seconds 5




Table6: (continued)


UR

Atmospheric EnvironmentPredicted�� profilesfrom climatic modelsdrivenby SSN,F � �"! # , EUV flux andKp (e.g.IRI[28]).

F C daily 10

Derived �� profilesfrom ionosondeobservationsandinterpolationsof globalor regionalmodelse.g.ITU-R [29].

N C 15minutes 11

Derived �� profilesby scalingstaticmodelswith dynamicTEC data.e.g.PRISM[30], PIM [31] updatedby JPLGIM [32].

N C 5 minutes 11

Ionosphericscintillationfrom theNwRA modelWBMOD [33, 34] drivenby Kp andSSNforecasts.

F S 15minutes 10

PolarCapabsorptiondrivenby CME/flaredetection. F S 5 minutes 10SWfadeoutsfrom flaredetection. F S 5 minutes 10Predictionsof decreasedS/Nratio from solarradioemissions F S 5 minutes 10Ionosphericstormforecastsfrom Kp predictionsbasedonsolarrotation.

F S 3-hourly 10

Ionosphericstormforecastsfrom Kp predictionsbasedonCME andsolarflaredetection.

F S hourly 10

Ionosphericstormforecastsfrom Kp predictionsbasedonsolarwind (Lund [25, 26] or Costello[27] models).

F S 15minutes 10

TEC from GNSS. N C 5 minutes 12Atmosphericmodel(e.g.UCL CTIP) modulatedby solaractivity to getatmosphericdragin LEO.

N S daily 16

Statisticalmodelof aurorallocation(e.g.NOAA POES[35])from geomagneticactivity usingKp forecasts.

F S 3 hourly 17

4.2.2 Model interconnection

GiventhattheSWSwill needto useawiderangeof modelstherearisesthequestionsof modellocationandmodelinteraction;whereshoulda modelresideandbeexecuted,andhow shouldmodelscommunicateifthey needto?

The modellingexpertisethat hasresultedin the list of modelsin Table6 is distributed acrossa numberof scientific institutionsin Europeand elsewhere. The choicefacing the SWS, for any given model, iswhetherto implementit locally or to accessit remotelyfrom anothersite.With sufficiently complex modelsit is generallyadvantageousto site an operationalmodelat an institution wherethe model is well under-stood,typically whereit wasdeveloped. This arrangementencouragesthe model to be well-maintained,and trouble-shootingcan be doneby staff experiencedin the model’s development. The advantagesoflocal implementationof a modelat the SWSarethat thereis full control over it andthe overheadof set-ting up communicationprotocolsbetweentwo sitesis eliminated. Many of the modelslisted in Table6aresufficiently matureandwell documentedfor local implementationto bepossible,but distributedmodelimplementationshouldstill beconsidered.

Figure2 illustratesthreewaysin which two distinctmodelscouldberun by theSWS.By combiningthesethreeforms all possibleinteractionsof multiple modelscanbe built up. The first andsecondforms aresimpleserialandparallelexecution;the third form is morecomplicatedsincethetwo modelsinteractpass




informationbetweeneachotherduringexecution. This might occurwhenthemodelscall eachotheriter-atively to converge on a mutuallysatisfactoryoutcome.Anotherexampleis wherea modelcanberun bymakingassumptionsabouttheoperatingconditions,but theseassumptionscanberemovedby usingoutputsfrom anothermodel.

databaseparameter

ModelModel

(a) Serial

Model

databaseparameter

Model

(b) Parallel

Model

databaseparameter

Model

(c) Interacting

Figure2: Modesof executionof two models

For themodelslisted in Table6 only theserialform of interactionis needed;for example,the forecastsofionosphericstormsrely on forecastsof Kp. This caseof simplechainingposesrelatively few problemsofcommunication,sinceafull modelrunis thenaturallevel of granularity. Theoutputof thefirst modelcanbepassedin its entiretyto thesecondandthisprovidesthecorrectsynchronisationbetweenthem.Thecaseofindependentbut parallelexecutionoffersno difficulty, sincethetwo modelscanbothrun to completion.Inthesecases,whetherthemodelsarelocalor remoteto theSWS,communicationbetweenmodelsis naturallyachievedby passingthestructureddatafiles (CDF or XDF) thatcontainthemodeloutputs.

Thecaseof interactingmodelsis morecomplicatedandraisesall theclassicalproblemsof synchronisationandcoordinationbetweenparallel processes.Someof theseproblems,suchas deadlockavoidance,arespecificto themodelsthatarecommunicatingsomustbedealtwith on a caseby casebasis.Thequestionof how modelsshouldcommunicateis genericand the SWSshouldadopta standardapproach.This isconsideredin detail in Section8.2.




SFR SFRDescription SIP SIP Description10.2 Allow modelsto beinterconnected 10.2.1 For themodelsin Table6, passing

standardoutputdatafiles in CDFor XDFformatis sufficient.

10.2.2 Runmodelsat themostappropriatelocation,whichmaybeata remoteinstitution.

10.2.3 Maintainawarenessof developmentsintheGrid field, andon possibleapplicationswithin spaceweathermodelling.

4.2.3 Model execution

Thedatainputsto themodelsin Table6 areupdatedatawiderangeof timeintervalsfrom everytensecondsto only twice a year. The modelsalsovary widely in thecomputerprocessinganddatastorageresourcesrequiredfor a full model run, and the frequency with which they are likely to be consulted. Thesetwofactorswill determinewhetherit is mostappropriateto runa modelroutinelyor on demand.

An exampleof routineexecutionwould beforecastsof Kp for hoursaheadto 27 daysaheadon timescalesfrom 15 minutesto daily, which would be of generalinterestandrelatively light in computingdemands.Othermodeloutputsareof interestonly to specificparties;anexamplewould bemappinghigh energy ionfluxesto specificaeroplaneflight pathsusinga radiationtransportmodel. Thesearespecificto particularoperatorsandthepossibleflight pathsaretoo numerousto generateroutinely. Instead,a userwould invokethemodelon themostcurrentsetof dataon demand,althoughtheremight bepre-processingof input datacarriedout routinelyto primethemodelfor userinvocation.

If thefrequency of updatebecomestoogreator themodelresultstoouser-specific,thenit maybebetterfortheSWSnot to offer a modeldirectly. Theclearestexamplefrom Table6 is theestimationof �$�� byinterpolationfrom local field measurementswith a tensecondupdateperiod.In suchcasestheSWSshouldactasasourceof adviceor consultancy with themodelrun locally by theuser.

SFR SFRDescription SIP SIP Description10.3 Offer modelsto berun routinelyor on

demandasappropriate10.3.1 Themodelswith S(tandard)modein

Table6 shouldgeneratestandardoutputsthatarearchivedroutinely.

4.3 Graphics

With theexceptionof someindicesof genericinterest,suchasKp or indicatorsof thelikelihoodof auroralactivity for auroraltourism,the SWSshouldnot make the graphicaldisplayof dataproductsits priority.Sinceit is anticipatedthat theSWSwould beactingprincipally in supportof serviceproviders,thegreaterpriority is providing datain a readilyaccessibleform ratherthanimagesof thatdata.

Theoptionof providing graphicaldatapresentationshouldbeincluded,however, for whichtheSWSshouldmakeuseof agenericdisplaypackagesuchasIDL from ResearchSystemsInc. (whereIDL herestandsforInteractive DataLanguage,asopposedto theCORBA InterfaceDescriptionLanguage).For theCDF dataformat thereareadditionaldatadisplaylibrariesavailable,suchasthe CDAWlib library of IDL routines.Providedthedatahasappropriatemetadatathis canplot arbitrarytime seriesdatausingappropriateline or




spectrogramplotsandspatialdataon anappropriategrid. A capabilitylike this is anecessaryfeatureof theSWS.A coupleof examplesof thetypesof plotsrequiredarepresentedin Figure3.

SFR SFRDescription SIP SIP Description16.1 Provide thecapabilityof generatingplots

from dataoutputs16.1.1 Usethegenericdatavisualisationpackage

IDL.16.1.2 If usingCDF, usetheCDAWlib plotting

routineson topof IDL.

5 User Interface

TheinterfacebetweenSWSusers,whetherserviceprovidersor thegeneralpublic,consistsof threeparts:

• On-linedataserviceson theinternet.• Off-line materialsfor distribution.• Humancontactfor supportor consultancy.

5.1 On-line data services

Themajority of interactionwith usersfor theSWSwould beusingon-linedataservicesprovidedover theinternet.Themainaccesswould bethrougha webserviceenablingusersto setup dataproductretrievals.Datadistribution would thenbecarriedout usingtheappropriatemethods,for exampleHTTP, FTP, e-mailor SMS.

A key featureof this serviceis thatusersshouldbeableto configurea personalsetof dataretrievals. Thecomponentsthey wouldneedto specifyare:

• Timing of dataretrievals.• Whichdataproductsto retrieve.• Parametersfor modelevaluationor dataselectione.g.time intervals,aircraftroutes,orbits,geograph-

ical areas.• Formatof dataproductsretrievede.g.CDF/XDF datafile, plots.• Modeof dataretrieval e.g.HTTP, FTP, e-mail.

‘Dataretrieval’ is hereassumedto encompassactionsrangingfrom fetchasimpleindex suchasKp, throughgeneratingaplot of someparameterandviewing it onawebbrowser, to initiating amodelrunandreturninga set of model results. Eachuserwould needa personalpassword-protected‘account’ that would storethedetailsof their dataretrieval profile andany otherpersistentinformation. It is possiblethat theseuseraccountswould bemosteasilyimplementedasrealuseraccountson theSWScomputersystem,but in anyeventthey shouldbeadministeredwith asimilardegreeof carebecauseof theresourceimplicationsof eachadditionaluser.

All thefacilities listedherearetargetedat therelatively sophisticatedcommunityof serviceproviders.Forgeneralpublic accesstherecould be a ‘dummy’ accountthat carriedout a selectionof actionsof generalinterestandpostedresultingplotsor datalistingsto partof theSWSwebsite.

Otheron-linefacilitiesshouldinclude

• On-line help– this includesgeneralinformationon how to usethesystemplusstructuredaccesstothemetadatadatabase.




(a) Plasmadatafrom theWIND spacecraft,plottedusingCDAWlib

(b) Statisticalauroraloval, derivedfrom NOAA-16 databy SEC

Figure3: Exampledataplots




• A bug reportingandsuggestionssystem.• A frequentlyupdatedsourceof relevantSpaceWeathernews,alongthelinesof ESA’sexistingSWEN

newsletter.• A systemfor requestingfurtherhelpfrom SWSpersonnel.

SFR SFRDescription SIP SIP Description12.1 Enableusersto build personalsetsof data

retrieval actions12.1.1 For all actions,usersmustbeableto

specifyandstorewhentheactionshouldoccurandits repeatinterval.

12.1.2 Allow theformatof retrieveddataproductsto beuser-specifiable(CDF/XDF/plot)

12.1.3 Allow thedatatransferprotocolto beuser-specifiable(HTTP/FTP/SMTP).

12.1.4 For custommodelinvocation,allow usersto supplyamodel-specificsetofparametervalues.

12.1.5 Userprofilesmustbepassword protectedsothatthey areaccessibleonly to thecontrollinguser.

3.1 Provideon-linehelpfor users 3.1.1 Providecontext-specifichelpon how tosetupdataretrievals

3.1.2 Provideanon-linerouteto SWSsupportstaff to requestfurtherhelpwith operatingthesystem.

18.1 On-linemechanismsfor userfeedback 18.1.1 Provideabug-reportingon theon-linesystem

18.1.2 Provideausersuggestionsor discussionforum

5.2 Static materials for distrib ution

As discussedin Section2.2,besidessupportingserviceproviderstheSWSshouldhave a role in informingandeducatinga broadercommunityof interestedparties.Theconsiderationof on-linedataservicesabovesuggestedgeneratingstandardSpaceWeatheroutputsof generalinterestandpostingthemto a portion oftheSWSwebservice.Therewould alsoneedto besomestaticwebpagesintroducingthescienceof spaceweatherandtheimpactsit hasonvarioustechnologicalsectors:airlines,electricpower, geological/offshore,HF radio,insuranceandfinancialservices,military, satellites,tourism.

This groundis coveredby severalotherwebsitessuchastheSEC[36], theSpaceWeatherCenter[37] andtheLund SpaceWeatherScienceCenter[38], but if theSWSis to bea seriousplayeron theglobalSpaceWeatherscenethenthissortof generalmaterialmustbepresented.Sincemuchrelevantmaterialhasalreadybeenassembledon-line, it would make senseto build on thatwherepossible,with theLund sitebeinganobvious startingpoint. As a Europeaninitiative the SWSshouldaim to emphasiseEuropeaninstancesoftechnologicalimpactsandto highlight ESWPspacecraftthat provide relevant data– the existing generalsitestendto beorientedtowardsaUS readership.

Thereis alsoa needfor staticmaterialfor distribution suchas papernewslettersandflyers, educationalpacksfor schools,exhibits anddisplays.Thesematerialsandaccompanying activities arecoveredin detailin Section6.




SFR SFRDescription SIP SIP Description14.1 Easyon-lineaccessto introductory

materialon thescienceandimpactofspaceweather.

14.1.1 Provideasetof tutorial webpages;theLund site[38] is agoodstartingpoint.

14.1.2 Constructageneraluseraccountthatgeneratesaselectionof mainly graphicaldataproductsto accompany astatictutorial.

5.3 Human contact

Althoughtheprimaryemphasisof this documenthasbeenon describinghow to provide automatedaccessto dataproducts,therewill beoccasionswhenuserswill needdirectcontactwith SWSpersonnel.Themostfrequentcasesarelikely to bewhensomeassistanceis neededin interpretinga dataproductor instructionrequiredon how to usea model. Theseexamplesarerelatively undemandingandcouldprobablybedealtwith by supportstaff whoseprimaryrolewasin managingthecomputingsystems.

More significantwill bethelessfrequentrequestsfrom serviceprovidersneedingassistancein developingspecifictailored productsfor usersor in understandingsomeaspectof the physicsof the spaceweatherenvironment.Meetingsuchrequestsneedsa greaterdepthof understandingof thesciencethancomputingspecificsupportpersonnelarelikely to possess,so this calls for staff with scientificsupportastheir mainfocus.

Another role for scientificsupportstaff would be in providing a personalinterfacefor the userfeedbackessentialto properservicedevelopment.It would bevaluablefor SWSpersonnelto make regularpersonalcontactwith boththescientificinstitutionsin Europedeveloping(andpossiblyoperating)themodelsneededby theSWS,andrepresentativesof theserviceprovidersandend-users.In additiontomakingregularcontactvisits to individual institutionsor companies,aneffective way of doingthis would beto have a numberofscientificandapplicationsworking groups,eachfocusingon a partof thescienceor applicationsof spaceweather. Their role would beto provide a forum for discussingtheexisting SWSservicesandfor feedingin theviews of theusercommunityasto how theservicesshouldbedevelopedin thefuture. Shouldsucha working groupalreadyexist for a particularsector, relevant SWSpersonnelshouldliaise with it ratherthancreatea duplicatebody. The forward-lookingrole of the scientificsupportstaff would also involvemonitoringtherelevantscientificliteratureandattendingpertinentevents,suchastheSEC‘SpaceWeatherWeek’.

SFR SFRDescription SIP SIP Description5.1 Humanbackupto theon-linehelp

systems.5.1.1 Providea telephoneande-mailhelp-desk

with definedresponsetime characteristics18.2 SWSdevelopmentto reflecttheinterests

andconcernsof theusercommunityandtheevolving stateof STP.

18.2.1 SWSstaff to make regularvisits toscientificinstitutionsandserviceproviders

18.2.2 Runworkinggroupsfocusedonspecificpartsof thescienceandapplicationsofspaceweather.




6 Outr each

Educationand Public Outreach(EPO)hasbecomea growing concernwithin ESA, partly as a result ofa Saatchi& Saatchireport that revealedthat the generalpublic interestin spacewasnot matchedby anawarenessof the activities of ESA. Even in Europeawarenessof ESA fell far shortof that achieved byNASA.

Spaceweatheris a naturaltopic in which to interestthepublic,sinceit is concernedpreciselywith how thespaceenvironmenthasdirectphysiologicaleffectson peopleandhow it impactsthetechnologicalsystemsthatpeoplerely on every day. TheSWSwould needto take a key role in promotingawarenessandunder-standingof SpaceWeatherin Europe.SinceESA alreadyundertakessignificantPublicRelations(PR)andOutreachactivities muchof theoutreachwork of theSWSwould bedeliveredthroughthesechannels.TheSWSrole would be in developingappropriatematerialsandin co-ordinatingESA activities with thoseofthescientificinstitutionsin membercountriesthatareengagedin spacesciencework. Therearefour broadsectorsthatwouldneedto betargeted:thegeneralpublic,schools,universitiesandindustry.

6.1 Generalpublic

Generalpublic awarenessof spaceweatherwould bebestaddressedthroughcentralESA channels,in theform of advertising,PRcampaignstied to missionlaunchesor significantspaceweatherevents.TheSWSPR staff would needto feed relevant factsandgraphicsto the centralESA PR function. In the caseofsignificantspaceweatherevents,therewould needto be materialpre-preparedso that it wasavailableatshortnotice.

Therearealsoopportunitiesto presentspaceweatherin moredepth.Theseinclude

• Temporaryexhibitsfor usein museumsor othervenues.NASA’sSpaceWeatherCenterhasdevelopedsuchanexhibit [39] thatcanbehiredout to any interestedpartiesat therateof $9,500for 12weeks.

• Making presentationsat suitableevents. For example,the UK hasa NationalScienceweekorgan-isedby the researchcouncilswith an extensive programof talks andeventsfor the public at manyvenuesnationwide. The SWScould provide speakers or provide materialsandencouragementforlocal scientificinstitutionsto put forwardspeakersfor suchevents.

SFR SFRDescription SIP SIP Description15.1 To raisegeneralpublicawarenessof space

weatherandits impacts.15.1.1 Preparetemplateinformationfor press

releasesto beusedwhenaspaceweathereventsis newsworthy.

15.1.2 Commission/designportableexhibits forpublicdisplay.

15.1.3 Preparematerialandpersonnelfor givingpublicpresentations.

6.2 Schools

Thereareobviousopportunitiesto developshortstudyunitsbasedon spaceweathertopics.Thereis scopefor bothstand-aloneactivities (suchasthePopMagNetproject[40] runby theUniversityof York in theUK)andfor activities tied to the SWSon-line services.In both casesthey shouldbe supportedby classroommaterialsfor teachersandstudents.




SFR SFRDescription SIP SIP Description15.2 To promoteawarenessof spaceweather

andspacesciencein schools.15.2.1 Developshortstudyunitsbasedonspace

weathertopics.

6.3 Universities

ESA alreadyhasanoutreacheffort basedat ESTECtermedthe‘EducationOffice’ [41]. Thishasamission‘to promoteandreinforceEuropeaneducationalexcellencein the spacefield’. It doesthis by promotingspace-relatedtopicsin curricula,encouragingawarenessof spacetechnologiesamongststudents,organisingwork experienceopportunitiesat ESA, nationalspaceagenciesand industry, and seekingto coordinategeneralinformation technologyto facilitate educationaloutreach. The SWS shouldwork to ensurethatspaceweatherfeaturesin thework of thisoffice.

SFR SFRDescription SIP SIP Description15.3 To promoteawarenessof spaceweatherin

universities.15.3.1 Collaboratewith theESAEducation

Office.

6.4 Industry

The most effective path to develop industry interestwould be to work with the spaceweatherserviceprovidersthat target varioustechnologicalsectors.Theseorganisationsalreadyhave the contactsandtheexperienceto do this, but for themto beableto statethat they wereworking in partnershipwith ansupra-nationalSWS would probablybe advantageous.The sector-specificworking groupsestablishedby theSWS(Section5.3)wouldprovideoneroutefor makingconnectionswith relevantpeopleandorganisations;moregeneralopportunitieswould ariseat industryconventions,for which displaymaterialscouldbemadeavailable.

7 Operationsand Management

Thissectionconsidersthemanagementandoperationsof theSWSandincludessomeestimatesof thecostofsettingupandoperatingthevariouselementsof thesystemdiscussedin previoussectionsof thisdocument.

7.1 Management

The managementof the SWSshouldbe basedon the well-understoodprinciplesof quality systemman-agementasoutlinedin theISO 9000:2000standard[42]. This standarddescribesvariouselementsthat themanagementof a systemis well-advisedto containif thesystemis to deliver a quality product,which in-cludesthedeliveryof dataproductsandservicessuchasthoseofferedby theSWS.Therequirementsof thestandardfall into five broadcategories:Systemic,Management,Resource,RealizationandRemedial.TheSystemicandManagementrequirementsrelateto theestablishmentandoperationof aQualityManagementSystem;the generalrequirementis that therebe a documentedquality systemrequiringoperationsto becarriedoutasdescribedin documentedprocedures.

Theothercategoriesrelatemoredirectly to productdelivery, andsomeof their implicationsfor theSWSaredrawn out below. Sectionnumbersgivenin parenthesesindicatethesectionsof theISO 9001standardthatapply.




7.1.1 Resourcerequirements

Therequirementson resourcescanbesummarisedastheproviding adequateresourcingfor theSWSto beeffective. In particular:

• Competentpersonnelare vital (6.2.1). Thereare threebroadflavours of competency requiredbySWSstaff – technical,scientificandpromotional.Thecomputingsupportstaff mustbe technicallyproficientin therelevantareas,theremustscientificsupportpersonnelwhoareabletohandlescientificqueriesfrom serviceproviders,andthe promotionof public understandingof spaceweatherby theSWSshouldbecarriedoutby staff selectedwith thatrole in mind.

• Competencemustbemaintained(6.2.2),soa programmeof staff developmentandtrainingis neces-sary.

• A quality infrastructuremustbe provided (6.3). Oneimplication for theSWSis that theremustbea commitmentto resourcetheon-goingoperationanddevelopmentof the serviceover an extendedperiod. The servicewill not gain the confidenceand respectof usersif its ability to develop andrespondto their concernsdoesnotmatchits initial scope.

7.1.2 Realization requirements

Theserequirementsdealwith the substanceof serviceandproductdelivery andconsequentlygenerateanumberof specificrequirementson theSWS:

• Interactionswith usersmustbe controlled(7.2). In particulardataproductrequestsmustbe loggedanda recordkeptof thesystemactivities thatoccurredto meetthem.

• Dataproductsgeneratedshouldbevalidated,eitherindividually or ona samplebasis(7.5.2).• Datainputsfrom otherdataprovidersshouldbevalidated(7.4.3).• Theoperationsof modelsrun remotelyfor thebenefitof theSWSneedto becontrolled(7.4.1).• User-suppliedinformation,suchascontactdetailsandtheir profilesof dataretrievals mustbe kept

private from otherusers(7.5.4). The accountdetailsthat areprotectedby password shouldnot bedivulgedto otherusers.

7.1.3 Remedialrequirements

• Usersatisfactionneedsto bemonitoredandmeasured(8.2.1).• Problemreportingandcorrection(8.3). Theprincipalreportingmechanismis theon-linebug report-

ing procedure,andtheremustbea systemin placeto tracktheprogressof work on reportedbugsupandreportbackbug fixes

7.2 Operations

The consequencesof the managementrequirementsof the previous sectionaredrawn togetherherewithadditionalelementsfrom elsewherein the document,to presentan overview of the requiredoperationalelementsandproceduresof theSWS.




7.2.1 Computing Infrastructur e

TheSWSshouldbeahigh-availability systemoperating24hoursaday. Thisrequiresaclusteredcomputingsystemwith built-in fail-over capabilities,andthehardwareshouldbeunderasame-dayattendancemainte-nancecontractwith themanufacturer. Reliableandhigh-capacitynetwork links arealsoessentialgiventheconstantflow of datato andfrom thesystem.Wherea particulardatafeedis importantit is not usuallythecasethatadedicatedlink is themosteffective or cost-effective meansof guaranteeingareliableconnection.It is usuallybettereitherto determinefixedroutesfor datatraffic thatcanbereliedon,or to seekto investinthosepartsof thegeneralnetwork wherelackof capacityhasthegreatestimpactontheend-to-endcapacity.

For securityof localdata,datawill bestoredondiskandtape,with regulardaily backupof volatileandnewdatato tapefrom disk.

Thecomputersystemswill needto provide thestandardinternetdataexchangeprotocolsof HTTP, FTPandSMTP, with unnecessarycommunicationservicesdisabledfor securityreasons.Theenabledserversneedto beconfiguredwith dueregardfor security, with firewalling to preventunauthorisedaccessandsufficientloggingto enabledetectionof intrusionattempts.

SFR SFRDescription SIP SIP Description1.6 Theon-linedatamustbeavailable24

hours.1.6.1 Useclusteredcomputingsystemswith

intrinsic fail-over capabilities.1.6.2 Seekto eliminatenetwork bottlenecksby

supportingcapacityupgrades.1.6.3 Purchaseasame-daycall-outmaintenance

contractfor computinghardware.1.7 Thedataarchive mustbesecure. 1.7.1 Carryoutdaily back-upof volatile data.

1.7.2 Apply theusualsecuritymeasuresapplicableto publicly accessiblenetworkedcomputers.

7.2.2 Personnel

The computingsystemsshouldbe supportedby personnelcompetentto dealwith softwareproblemsandidentify whenexternalhardwaresupportis required.Thelocalsupportstaff shouldbecontactable24hours;ontheassumptionthatthecomputingsystemswouldbegenerallyreliable,out-of-hourscoverwouldbestbeprovidedby anon-callsystem.Thecall-outsystemcanbeautomatedby usinglocal softwareprocedurestomonitorsystemfunctions,andby usingcommercialremoteservicesto monitormachinestate.A minimumof threefull-time staff memberswouldberequiredto supportthecomputingoperations,allowing for plannedor unplannedabsence.Additionalstaff would berequiredwhile theservicewasbeingsetup becauseof thegreaterdemandsfor softwaredevelopmentduringthatphase.

Tosupportthesophisticateduserssuchastheexistingspaceweatherserviceprovidersin Europewill requiremorethanjust a fully computerisedsystem.Thereis alsoa needfor scientificallyexpertsupportstaff whocanliaisewith externalorganisationsto develop thesuiteof modelsandforecastservices,asdescribedinSection5.3.Giventherangeof modelslistedin Table6 asnecessaryfor aminimuminitial servicemultiplepersonnelwouldberequired:

• 3 for theplasmaandradiationenvironment• 1 for thesolid bodyenvironment• 2 for themagneticenvironment




• 3 for theatmosphericenvironment

Section6 presenteda wide rangeof possibleeducationandoutreachactivities andmaterialsthat theSWScouldusefullygenerate.Thenatureof this outreachwork is suchthat thegreaterpartof theproductionofmaterialswould becarriedout eitherby othersectionsof ESA or externalsub-contractors.TheSWSmustbeableto actasa sourceof ideasandmaterialandto provide co-ordinationfor theoutreachactivities andthis would requirededicatedstaff with an interestandflair for PR activities, accompaniedby a thoroughunderstandingof thescience.

All staff musthave timely accessto relevanttrainingopportunitiesdeterminedby developmentsin comput-ing technologies,thestateof solar-terrestrialphysicsandtheinformationneedsof thewidercommunity.

SFR SFRDescription SIP SIP Description13.1 Be ableto supporttheoperationof the

computingsystems13.1.1 Employ at leastthreestaff members

dedicatedto computersupport13.1.2 Computingsupportstaff shouldbe

contactable24hours.13.1.3 Uselocalandremoteautomaticsystems

to monitorsystemavailability13.2 Be ableto handlescientifically

sophisticateduserqueriesanddeveloptheserviceappropriately.

13.2.1 Haveasuitablenumberof staff dedicatedto scientificsupportandconsultancy: aminimumof 9 is suggested(Plasmaandradiationenvironment– 3, Solidbodyenvironment– 1, Magneticenvironment–2, Atmosphericenvironment– 3)

13.3 Be ableto promotepublicunderstandingof spaceweather

13.3.1 Employ at leastonestaff memberwhoseprimaryfocusis publicoutreach,intandemwith somescientificknowledge.

13.4 Staff competenciesmustdevelopin linewith changesin science,technologyandthepublicenvironment.

13.4.1 Provideon-goingtrainingfor all staffrelevantto their roles.

7.2.3 Data inputs

TheSWSwill retrievedatafrom numerouson-linesourcesworld-wide,with aninitial estimateof datatypesandvolumesmadein AppendixA. While not beingin a positionto determinethedataformatsfrom thesesources,theSWSshouldmake dataprovidersawarethat their datais beingused,andaskto begivenpriornoticeof changesin dataformator content.

Dataretrievals shouldbe logged,the retrieved datashouldbe checked for thecorrectformatandthe datavaluesscreenedfor plausibility. Any checkfailuresshouldberecordedandthesystemoperatornotifiedatanappropriatetime.

For all retrieveddatavaluesit mustbepossibleto determinethetime atwhich thevalueswereretrieved,sothat thedataenvironmentat any giventime canberecovered.On thestructuredfiles modelof datastorageproposedin Section3.2, this canbe achieved by addingretrieval time to the metadataaccompanying thedatafile.




SFR SFRDescription SIP SIP Description1.8 Aim to acquireall relevantspaceweather

datafor input to theSWS.1.8.1 Maintaingoodrelationshipswith data

suppliersby keepingtheminformedofdatausage.

11.4 Ensurethatretrieveddatais of appropriatequality

11.4.1 Log all dataretrievals

11.4.2 Only ingestincomingdatathathasbeencheckedasfreeof formaterrors

11.4.3 Endeavour to checkincomingdataforplausiblevalues

7.2.4 Data outputs

TheSWSshouldseekto outputdatain a singlestandardformat,althougha plain ASCII outputmight needto beaccommodatedin additionto a richerstructuredformat. Dataoutputsshouldbeloggedto maintainarecordof theuseof theSWS.Sincesomedataoutputscanbegeneratedondemandit maynotbepracticableto checkall dataproductsbeforedispatch,but thereshouldbe a programof validatingall modeloutputs.For largedataproductsor for thosethatareonly sampledby userrequests,only representative samplesofmodelor predictoroutputsneedbechecked.Thecheckingshouldensurethatthedatavaluesarereasonablegiven thedatainputsand,in thecaseof predictors,regular assessmentsof accuracy shouldbe madeaftertheeventto provide feedbackfor modelenhancement.

SFR SFRDescription SIP SIP Description12.2 Usersshouldbeableto determinewhat

dataproductsthey have retrieved.12.2.1 Log all dataoutputsfrom theSWS,

centrallyandby user.12.2.2 Allow usersto inspecttheirpersonal

retrieval logs.17.1 Ensurethatgenerateddataproductsfrom

modelsandforecastsareof appropriatequality

17.1.1 Validateat leastsomeof all modeloutputsagainstdata

17.1.2 Periodicallyassesstheaccuracy ofpredictions

7.2.5 Data processing

Theprincipaldataprocessingactivity of theSWSis theoperationof modelsor predictors,eitherlocally orremotelyatcooperatinginstitutions.Theconfigurationof eachmodelshouldbedocumented,with boththisdocumentandthemodel’s sourcecodeunderrevisioncontrol.

Themodelsshouldbesetup to ingestanddeliver datafrom andto theSWSin thestandardSWSstructureddataformat. In thecaseof remotelyrunmodelsthiswill requireaformal interfacespecificationto beagreedbetweentheSWSandtheinstitutionproviding theremotemodel;this shouldbedoneundertheauspicesoftheSWSquality policy, with formal changecontrol.

The otherdataprocessingactivity is handlinguseraccountsanddatarequests.As indicatedpreviously,theuseraccountsshouldbe accessibleover thewebunderpassword protection.BecausetheSWSwouldallow usersto setupapersonalprofileof dataretrieval activities, thusimposingadditionalloadontheSWScomputersystems,it would benecessaryto have userregistrationwith theseapprovedby SWSstaff ratherthanautomatically. Userdatarequestsandchangesto their personalprofilesshouldbe logged. The only




exceptionwould be for a genericpublic accountthatextractedanddisplayedselectedmodeloutputson aregularbasisfor unrestrictedaccess.

SFR SFRDescription SIP SIP Description10.4 Ensurethatgenerateddataproductsfrom

modelsandforecastsareof appropriatequality

10.4.1 Documenttheconfigurationof all models,whetherrun locally or remotely.

10.4.2 Maintainmodeldocumentationandsourcecodeunderformal revisioncontrol

7.3 Costing

This sectionpresentillustrative costingsfor establishingandrunningtheSWSin theform presentedin thisdocument.Thefiguresshouldbeconsideredonly asa roughguideto thecostsinvolved– somepartof thereal costsaredependenton the level of usageof the serviceandon the costsof procuringcomputerandnetworking facilities,which historicallyhave decreasedsubstantiallyin real termsover time. All costsareshown in KEuro.

7.3.1 Start-up costs

The set-upcostsarecomposedof the purchasecostsof the computingequipmentandthe initial costsofsoftwaredevelopment. The examplecomputingsystemis a clusterof four CompaqDS20AlphaServersrunningTru64Unix, with disk storageprovidedby a 1400GbRAID systemandtapestorageby a modularAIT-2 tapelibrary. Theinitial softwaredevelopmentcostswould dependon thedegreeto which theneces-sarymodelswereout-sourcedto co-operatinginstitutions.Themajorityof theinitial softwaredevelopmentwork wouldbecarriedoutby thepermanentstaff but theinitial developmentwould requireadditionalman-power, estimatedas5 man-yearsof effort. For this estimate,total staff costsaregivenas130KEuro/yearwhich is thesameasthedevelopmentratefor spacecraftgroundsegment[6]. It is assumedthatfor theSWSthedevelopmentandoperationalstaff wouldbethesamepersonnel,sothereis nodifferencein theratesforthetwo phases.All costsareshown in KEuro.

TheRAID systemhasthecapacityto hold severalyears’supplyof input data,asestimatedin AppendixA.No estimatehasbeenmadeof the volumeof generateddataproductsthat would be stored,but the diskstorageoptionhasbeenselectedwith plentyof sparecapacity. TheAIT tapesystemis modular, allowingup to 5 modules;eachtapestores50Gb,soa 19 slotmodulerepresentsroughly1Tbof storage.ThecostofIDL licensesis thenominallist price– discountsareusuallynegotiablefor governmentor academicusers.

7.3.2 Operating costs

Theprincipalrunningcostsarestaff salaries,theleaseof aninternetconnection,,themaintenancecontracton the computersystems,andthe depreciationon the computinghardware. Promptcall-out maintenancecontractstypically cost10%of thehardwarecostssothemaintenancecontractis costedappropriately, basedon the initial computingequipmentspendof 250 KEuro. Computerequipmentis customarilywritten offover nomorethan5 years,so20%of theinitial spendis designatedfor capitaldepreciation.

Thecostof a leasedline of a givencapacityandlengthvarieswidely acrosstheEU andhastypically beendecreasingin costby 10%a yearfor the last threeyears[15]. Thefigurequotedin thetablebelow should




Item Cost/Item Number TotalHardware

Server – CompaqDS20E(2 CPUs) 35.0 4 140.0Clusterkit – Baseunit 13.0 1 13.5Clusterkit – server cards 2.0 4 10.0Disks– 1400GbCompaqFibreChannelRAID 5400 65.0 1 65.0Tapelibrary – AIT-2, 2 drive,19slots,baseunit 20.0 1 20.0Tapelibrary – additionalmodule 6.5 1 6.5Tapes– 50Gbcapacity 0.1 40 4.0

SoftwareIDL licenses– unlimited 37.5 1 37.5

StaffSoftwaredevelopment 130.0 5 650.0

MiscellaneousPortablepublicexhibits 20.0 2 40.0Total 986.5

Table7: Set-upcosts

thereforebe regardedas illustrative ratherthan definitive. The final line, ‘Miscellaneousconsumables’,allows 3 KEuroperpersonperyearto cover any miscellaneousexpensesfor travel, photocopying etc.

Item Cost/Item Number TotalStaff – Scientific 130.0 9 1170.0Staff – Computing 130.0 3 390.0Staff – PR 130.0 1 130.0Leasedline rental 25.0 1 25.0Computermaintenance 25.0 1 25.0Depreciation 50.0 1 50.0Miscellaneousconsumables 3.0 13 39.0Total 1829.0

Table8: Runningcosts

8 Futur e Development

Theform of SpaceWeatherServiceoutlinedin this documenthasbeenonewhich couldbeestablishedinthenear-futureto serve a presentneed.Over time theneedsof theusercommunitywill developaswill thecapacityof thescienceof spaceweatherto meetthem.Thissectionsketchesoutsomeof thewaysin whichtheSWSshouldevolve,scientifically, technicallyandorganisationally.

8.1 Scientific development

Section4.2.1presentedthe initial setof modelsthat theSWSshouldoffer, derived from thesetof modelsidentifiedin theSRD.Themodelslistedin Table6 arethoseclassifiedas‘immature’ in theSRDthatwouldhave wide application– modelsto predictKp and other indicesusing other models,a numericalmodel




of solarprotoneventsbasedon time seriesanalysis,anda probabilisticmodelof geo-effective solarwindstructures.


UR

Numericalmodelsof solarprotonsbasedon time series,e.g.work by SouthamptonUniversity[43]

F S hourly 1 913 1720

Probabilisticmodelsof geo-effective solarwind structurese.g.theCargill method[44]

F S 1 minute 4 7 913

Numericalmodelsfor Kp forecasting F S 3 hourly 4 7 9

Table9: Futuremodelsfor theSWSto provide

Therelatively smallnumberof suchmodelsis testamentto theobstaclesfacingbetterspaceweatherpredic-tion at present.Theseincludepoorunderstandingof thedetaileddynamicsof theprocessesin thesunthatdrive thespaceweatherchainandthedifficulty of adequatelyinstrumentingtheregionof spacebetweenthesunandtheearth.Both of theseareasof difficulty will only beaddressedsatisfactorily on long time-scales.This posesproblemsfor somepredictiontechniques,in particulartheartificial intelligenceapproachusingneuralnetworks.

Presentneuralnetwork predictiontoolshave provedsuccessfulat characterisingthebehaviour of magneticindiceson timescalesof a few hours.It would appearthatthemagnetosphereexhibits characteristicmodesof behaviour on thesetimescaleswhich thenetworksareableto identify from thetimeseriesof theindices;beyond a few hours,the significanceof the externaldriversof the systembecomesever greater. Thereisa tendency for suchmodels,therefore,to be goodat predictionunderquiet conditionsor at trackingtheresultsof adisturbance,but pooratpredictingtheonsetof disturbedconditions.Thepresenceof solarwindmonitorsat L1 hasalleviatedthis to somedegree,sincethereis now up to a coupleof hourswarningofchangesin thesolarwind conditions.Furtherimprovementof thesamemodelswill requireinstrumentationcloserto thesun,but this approachstill limits predictionto at most2-3 daysahead.To go beyondthatwillrequireabetterunderstandingof thesolarprocessesleadingto CMEs,flaresandotherdisturbances.

WhattheSWScando is to encouragetheevolutionof modelscurrentlyusedfor scientificpurposestowardsastatewherethey canbeoperationallyuseful.Two leadingcandidatesfor thisareglobalmodelsof

• theneutralandionisedatmosphere,suchasCTIP [45]• themagnetosphere,suchasGUMICS[46].

Both of thesecited examplesarealreadyleadingthe way in their respective fields in Europe,or even theworld, andtheconcertedsupportof theSWSwould maintainor enhancetheir eminence.TheCTIP modelhasbeenincludedin anearliersectionto provide grossatmosphericcharacteristicssuchasa daily estimateof atmosphericdragon LEO spacecraft(Table6); the role envisagedhereis moredemanding,requiringpromptmodellingof moredynamicbehaviour drivenby fluctuatingmagnetosphericconditions.

With severalgoodmodelsof differentregionsof thespaceweatherenvironmentreadilyavailable,a furthernaturaldevelopmentwould beto encouragetheinterconnectionof thesevariousmodelsasthenext stepindevelopinga coordinatedmodelof the full spaceweatherenvironment,from the solarsurface(or below)throughthe solarcorona,solarwind, Earth’s magnetosphereandionospheredown to the Earth’s surfaceor crustalcurrentsystems.For a taskof this complexity, involving feedbackbetweenmodelsof differentregimesoperatingundera variety of modellingassumptions,sophisticatedtechnicalcomputingfacilitieswould berequired,asdiscussedin thefollowing section.A valuablefirst stagewould betheconnectionoftheCTIP andGUMICSmodels.




8.2 Technicaldevelopment

The volumesof datarequiredand generatedby the SWS are small by comparisonwith thosefrom theinternationalscientificcollaborationsthataredriving thecurrent’Grid’ initiativesin EuropeandtheUS[47],suchastheLHC at CERNandSLAC. Thegreatesttechnicalchallengesfacedandposedby theSWSareintheareaof modelinterconnection.

Therearenumerousprotocolsandsoftwarelibrariesavailablefor facilitating inter-processcommunicationandsynchronisation,whetherlocal or distributed.Someof themoreprominentof thesearetheHigh LevelArchitecture(HLA) [48], Globus[49], CORBA [50] andJini [51] :

HLA Developedby theUSDoD originally to allow individualsimulationsto cooperateto simulatea largersystem.It worksby building a‘federation’,whereeachfederateis adistinctsimulationor model.TheHLA stipulatesa setof rulesthata federatemustobey, providesa setof run-timeservicesandinter-facesto theseservicesin many languages(C++, Java, Ada, CORBA IDL). Thereis comprehensiveprovision for exchangingdatabetweenfederatesandfor specifyingcomplex rulesfor inter-federatecommunicationandsynchronisation.TheHLA hasbeenadoptedby the IEEE asanopenstandard,andastheFacility for DistributedSystemsby theObjectManagementGroup(OMG) which hasde-velopedCORBA.

Globus TheGlobus projectprovidessoftwaretools to facilitateresourcesharingamongdistributedcom-puters.They provide anumberof toolsandlibrariesthataddressissuesof security, resourceanddatamanagement,communicationandfaultdetection.Thetoolkit APIsareall in C but thereareJavaclasswrappersfor themostimportantcomponents.

CORBA An infrastructurefor distributedsystems.It is a lower-level architecturethantheHLA, andis alsolargely computerarchitectureandsoftwareindependent.Its InterfaceDefinition Language(IDL) hasbeenacceptedasanISO standardandhasmappingsto C, C++, Java,COBOL,Smalltalk,Ada,Lisp,PythonandIDLscript.

Jini A Java-baseddistributedsystemsarchitecture.It is specificto theJava languageandcoversmuchofthe samegroundasCORBA. The principal conceptualdifferenceis that CORBA offers accesstodistributedobjectsresidenton CORBA servers,whereasJini allows oneto offer distributedservices,with proxyobjectsrunningon theclients.

All of thesearchitecturesfor inter-processcommunicationimposeobligationsandconstraintson systemsthatusethem.Theiruseis notwidespreadin STPmodellingwork atpresent,sotheiradoptionwouldrequirea largemeasureof coordinationfrom andencouragementby theSWS. TheHLA is themostsophisticatedtechnologydescribedabovebut it is alsotheonly oneaimedspecificallyatsimulationandmodelling.Thus,althoughimplementingit makesthe greatestdemandson modeldevelopersit offers the bestprospectforconnectinglarge-scalemodelsbecauseit hasthe rigour and sophisticationto handlethe complex ruleson feedbackbetweensuchmodelsthat would be necessary. As a first step,the SWSshouldsupporttheprovisionof therelevantHLA interfaceroutinesin theCTIPandGUMICSmodelsattheirhomeinstitutions.

For relatively simplemodelstheoverheadof a sophisticatedarchitecturelike theHLA would beexcessive.It wouldbebetterfor suchmodelsto beprovidedwith appropriateinterfacesusingJini or CORBA to allowremoteinterrogationandinvocation. The SWSshouldagainact asan advocateof theserelatively simpletechnologies.

Many of the issuesraisedby operatingdistributed modelsarealsobeing tackledby the so-called‘Grid’initiatives in Europeandthe US [47]. The lower-level technologieslisted above (Globus, CORBA, Jini)




areall beingusedactively in Grid work, andany developmentsfrom this field shouldbeconsideredwhendevelopingthemodellingenvironmentof theSWS.

SFR SFRDescription SIP SIP Description19.1 Fostercollaborationbetweenrelevant

groupsin Europe.19.1.1 Supporttheuseof theHLA [48] in the

CTIP [45] andGUMICS[46] models.19.1.2 Developsimpleprotocolsfor model

invocationandinterrogationandsupporttheir implementationusingappropriatetechnologies(e.g.CORBA [50],Jini [51]).

8.3 Organisationaldevelopment

TheproposedSWSaspresentedhereis a relatively small-scaleoperation,relying on theinput andinvolve-mentof datasuppliersandmodeldevelopersto provide theservicesit offers. As theunderstandingof thephysicsof spaceweatherimproves,and in particularif it becomespossibleto model the full interactionchainfrom the solarsurfaceor below to the Earth,the complexity of usefulmodelswill increase.It willthereforebecomeincreasinglyimportantto have asingleEuropeancentrewheredefinitive versionsof suchmodelsreside,supportedby theefforts of a multinationalteamof scientistsandcomputingspecialists.Inthiscase,thesizeof theSWSoperationwouldneedto beincreasedsignificantly, with enhancedcomputinginfrastructureandmorepersonnel,someonsecondmentfrom nationalinstitutions.Thisapproachis similarto that of the EuropeanCentrefor Medium-rangeWeatherForecasting(ECMWF), which hascloselinkswith thenationalweatherforecastingservicesof many Europeannations.

GiventheEuropeansituationof geographicallydispersedandcomplementaryexpertisein thevarioussub-disciplinesof spacephysicscontributing to spaceweather, it is importantfor the SWSto provide a focusfor all partiesinterestedin spaceweatherphenomenaandeffects.Thedetailsof theorganisationalstructurewhichhousestheserviceareof muchlessimportancethantherole it plays.Two possiblescenariosare:

• An organisationalong the lines of the ECMWF, whoserole is entirely devoted to forecastingandmodellingthespaceweatherenvironment,or

• An organisationanalogoustoEUMETSAT,whichrunstheentireEuropeanspaceweatherprogramme,from satellitesthroughground-supportfor thesesatellitesthroughto disseminationof dataandfore-castproductsthetheSWS.

Eitherapproachwouldbesatisfactory, althoughin practicetheloosercollaborativeapproachof theECMWFwouldbesimplerto establishin theshortterm.

SFR SFRDescription SIP SIP Description19.1 Fostercollaborationbetweenrelevant

groupsin Europe.19.1.3 Aim to establishaprogrammeof

secondmentof personnelfrom nationalinstitutesto theSWS,alongthelinesoftheECMWF.




9 Summary

This documenthaspresenteda detailedproposalfor a EuropeanSpaceWeatherService.Thekey partsoftheanalysisare:

• A setof ServiceUserRequirementsthatany SpaceWeatherServiceshouldmeet(Table2, page13).• A high-level designschematicof thecomponentsnecessaryto meettheserequirements,andhow they

areinterrelated(Figure1, page14).• The ConsolidatedSystemMeasurementRequirementsfrom the SRD that constitutethe ‘raw’ data

feedto theSWS,with estimatesof thedatavolumesinvolved(Table10 in AppendixA).• Theinitial setof modelsthattheSWSshouldprovide (Table6, page27).• A setof detailedServiceFunctionalRequirementsandassociatedServiceImplementationProposals

describingthefeaturesthattheservicemustprovideandsuggestedmeansof implementation(Table11in AppendixB).

• Estimatesof theset-upandrunningcostsof theserviceincludingsuggestedhardwarepurchasesandstaffing levels(Tables7 and8 on page42).




References

[1] Study for an ESA SpaceWeather Programme,Proposalin responseto ESA ITT AO/1-3533/99/NL/SB,Issue1.1,28thOctober1999.

[2] SpaceWeatherMarket Analysis, Astrium Ltd. Report producedfor ESTEC under contract no.14069/99/NL/SB,Draft Issue5.0,31stJanuary2001.

[3] Lesley M Murphy andDavid J Rodgers.EuropeanSpaceWeatherProgrammeSystemRequirementsDefinition.TechnicalReportDERA/KIS/SPACE/CR010157,DERA,February2001.ESTEC/ContractNo. 14069/99/NL/SB.

[4] Michel KruglanksiandDanielHeynderickx. Spaceweatherserviceprototype.TechnicalReportWP434– BIRA-IASB, BIRA, July2001.ESTEC/ContractNo. 14069/99/NL/SB.

[5] http://www.aurorawatch.york.ac.uk/.

[6] SpacecraftInterface.Reportproducedfor ESTECundercontractNo. 14069/99/NL/SB,July2001.

[7] Mike Hapgood.The interfacebetweeninstrumentground-segmentsandtheSpaceWeatherService.TechnicalReportESWS-RAL-TN-0002,RAL, June2001.ESTEC/ContractNo. 14069/99/NL/SB.

[8] ISO. ISO/IEC9075-1:1999Informationtechnology– Databaselanguages– SQL– Part 1: Framework(SQL/Framework), 2000.

[9] ISO. ISO/IEC9075-2:1999Informationtechnology– Databaselanguages– SQL– Part 2: Foundation(SQL/Foundation), 2000.

[10] http://nssdc.gsfc.nasa.gov/cdf/cdf home.html.

[11] http://sml.gsfc.nasa.gov/XDF/XDF home.html.

[12] http://www.cacr.caltech.edu/SDA/xsil/.

[13] http://spdf.gsfc.nasa.gov/istp guide/istpguide.html.

[14] http://httpd.apache.org/.

[15] Commission of the European Communities, Brussels. Sixth Report on the Implementa-tion of the TelecommunicationsRegulatory Package, COM(2000) 814, December2000. Seehttp://europa.eu.int/comm/information society/policy/telecom/6threport/index en.htm.

[16] http://crsp3.nrl.navy.mil/creme96/cm/refs.htm.

[17] http://www.cami.jccbi.gov/aam-600/610/600Radio.html.

[18] C Stormer. . Z. Astrophys., 1:237,1930.

[19] M A Sheaand D F Smart. . In 18th International ComsicRay Conference, ConferencePapers,volume3, page415,1983.

[20] http://wwwinfo.cern.ch.asc/genat4/geant4.html.

[21] http://nssdc.gsfc.nasa.gov/space/model/magnetos/tsygan.html.

[22] S Bourdarie,D Boscher, M Blank, andJ A Sauvaud. A physical4D radiationbelt modelincludingatimedependentmagneticfield. Advancesin SpaceResearch, 25(12):2303–2306, 2000.




[23] H Klinkrad, JBendisch,H Sdunnus,PWegener, andR Westerkamp.An introductionto the1997ESAMASTER model. In Proceedingsof theSecondEuropeanConferenceon SpaceDebris, volumeESASP-393,pages217–224,May 1997.

[24] PJenniskens.Meteorstreamactivity. AstronomyandAstrophysics, 287:990–1013,1994.

[25] F Boberg, P Wintoft, andH Lundstedt.Realtime Kp predictionsfrom solarwind datausingneuralnetworks. PhysicsandChemistryof Earth,Part C, 25(4):275–280,2000.

[26] http://sol.irfl.lu.se/~fredrikb/Kp/.

[27] http://sec.noaa.gov/rpc/costello/index.html.

[28] http://nssdc.gsfc.nasa.gov/space/model/ionos/iri.html.

[29] InternationalTelecommunicationsUnion. ITU-R Methodsof basicMUF, operational MUF andray-pathprediction, RecommendationP.1240(05/97)edition,May 1997.

[30] http://www1.primushost.com/~cpibos/. ComputationalPhysics,Inc., vendorof PRISM.

[31] R E Daniell, L D Brown, D M Anderson,andM W Fox. ParametrizedIonosphericModel: A globalionosphericparametrizationbasedonfirst principlemodels.RadioScience, 30(5):1499–1510,Sep-Oct1995.

[32] http://iono.jpl.nasa.gov/gim.html.

[33] E JFremouwandJA Secan.Modelingandscientificapplicationof scintillationresults.RadioScience,19:687–694,1984.

[34] http://www.nwra-az.com/ionoscint/wbmod.html.

[35] http://www.sel.noaa.gov/pmap/index.html.

[36] http://www.sec.noaa.gov/Education/.

[37] http://www.spacescience.org/SWOP/1.html.

[38] http://www.irfl.lu.se/HeliosHome/spacew2.html.

[39] http://www.spacescience.org/SWOP/Exhibits/Mini Exhibit/overview/.

[40] http://www.aurorawatch.york.ac.uk/popmagnet/.

[41] http://www.estec.esa.nl/outreach/office.html.

[42] ISO. ISO9000:2000,Quality managementsystems– Requirements, 2000.

[43] http://www.ses.soton.ac.uk/projects/Astronautics/Gpatrick/gpatrick.html.

[44] J Chen,P J Cargill, andP J Palmadesso.Real-timeidentificationandpredictionof geoeffective solarwind structures.GeophysicsResearch Letters, 23:625,1996.

[45] http://alpha.apg.ph.ucl.ac.uk/Docs/ctip index.htm.

[46] http://www.geo.fmi.fi/~pjanhune/gumics3/.

[47] R BuyyaandM Baker, editors. Grid computing– GRID 2000: 1st IEEE/ACM InternationalWork-shop.Proceedings,Bangalore, India, 17 December2000, volume1971of Lecture Notesin ComputerScience. Springer, Berlin, 2000.




[48] http://www.dmso.mil/index.php?page=64/.

[49] http://www.globus.org/.

[50] http://www.omg.com/gettingstarted/corbafaq.htm.

[51] http://www.javacommerce.com/tutorial/j ini.

[52] MikeHapgoodandMikeOliver. A definitionof instrumentsneededfor spaceweathermeasurements.TechnicalReportESWS-RAL-TN-0001,RAL, May 2001.ESTEC/ContractNo. 14069/99/NL/SB.




A Data StorageRequirements

Table10 lists the ConsolidatedSystemMeasurementRequirements(CSMR) from the SRD andprovidesestimatesof thedaily dataratethateachmeasuremententails.WhereaCSMRduplicatesanotherexceptforits temporalsamplingrequirement,only thehighestsamplingrateis listed; all lower ratescanbe derivedfrom thehighestratesequenceby temporalaveragingor accumulation,sonofurtherbulk storageis required.CSMR18 is omittedbecauseit wasdecided,aftercompletionof theSRD,that theseobservationswerenolongerrequired(seesection2.5of theinstrumentdefinitionreport[52]).

Two assumptionshave beenmadeaboutthesizeof dataelements.Imagesarepresumedto use1Mb, thisbeingderivedasa

��%�&(')��%�&pixel arraywith onebyteperpixel. All othernumericfieldsareassumedto

be long realvaluesneeding8 bytesof storage.This maybeanoverestimatefor simpleintegerparameterssuchasF � �"! # or Sunspotnumber, but on averageit is unlikely to be in errorby a factorof morethantwo,whethernumbersarestoredin binaryor ASCII.

Thedatastorageregimeemployedwould alsoaffect thedaily requirementfor extra storage.For example,if a relationaldatabasewere usedthen therewould be someoverheadper table row associatedwith thebasedataandany relatedindexes. If insteaddatawerestoredin a file treein CDF or XDF format,thennoallowancehasbeenmadefor the metadataassociatedwith the dataitems; it is assumedthat datawill bedatabasedin sufficiently largeunitsthatany metadataoverheadimposedby thestorageformatis smallasaproportionof thetotal.

The final columnof Table10 shows the percentageof the total dataflow that eachsetof measurementsaccountsfor. It is readily apparentthat the dataratesaredominatedby only a few of the measurements,namelythosethatareimages,to theextent thatmany of theotherpercentageshave roundedto zero. Over70% of the data is attributed to the ground-basedauroral imaging for which there is an assumedneedfor 30 auroralcamerasproducinghourly imagesof 1Mb in size to meetCSMR 5 (SMR 16.5). If themeasurementrequirementthat theseauroral imagesare intendedto meetcould be satisfiedby a sparsernetwork of imagers,or by usingameridianscanningphotometerratherthananimager(assuggestedin [52]),thenthisfigurecouldbereducedby a factorof up to 1000if MSPsweresubstitutedfor imagers.This is theonly oneof theimagingrequirementswheresucheconomiesmaybepossible.

Theconclusionto bedrawn from thefigurespresentedin thetableis that thedaily datarateimplied by theneedto retrieve thespaceenvironmentdatais between200Mband1000Mbperday.

Furtherdatastoragerequirementsare imposedif the SWSalsohasto serve as the archive for calibratedphysicalparametersproducedby dedicatedESWPSpaceWeathermissions. Many of the spaceweatherrequirementson physicalparametersarefor relatively low ratetemporalsampling,for example,thevarioussolar imagesarerequiredonly hourly andthe � 10MeV ion flux is requiredonly half-hourly. It is likely,however, that any instrumentflown to producetheseparameterswill sampleat a muchhigher rate. Thefiguresin the tablecanbeusedto estimatethedaily dataratesimplied by greatertemporalresolutionandtheimplicationsfor datastoragecalculated.




CSMR Dataitem Channels Sites Bytes #/day TotalBytes % of data1 SolarEUV images 2 1 1.05e+6 24 5.03e+7 4.881 SolarX-Ray images 2 1 1.05e+6 24 5.03e+7 4.882 Coronagraphimage 1 1 1.05e+6 24 2.52e+7 2.443 StereoUV/visible images 2 2 1.05e+6 24 1.01e+8 9.774 Auroral imaging(s/c) 1 1 1.05e+6 24 2.52e+7 2.445 Auroral imaging(ground) 1 30 1.05e+6 24 7.55e+8 73.276 Auroral oval description 1 1 4.80e+1 24 1.15e+3 0.007 AO equatorwardbdy 1 24 8.00e+0 24 4.61e+3 0.008-11 X-ray flux 2 1 8.00e+0 1440 2.30e+4 0.0012 UV flux 1 1 8.00e+0 288 2.30e+3 0.0013 EUV flux 1 1 8.00e+0 288 2.30e+3 0.0014-17 F10.7 1 1 8.00e+0 288 2.30e+3 0.0019-21 2ndaryneutrons(ground) 1 40 8.00e+0 24 7.68e+3 0.0022 2ndaryneutrons(air) 1 30 8.00e+0 288 6.91e+4 0.0123-25 V *,+ 1 2 8.00e+0 1440 2.30e+4 0.0026-27 N *,+ 1 1 8.00e+0 96 7.68e+2 0.0028 Kp 1 1 8.00e+0 8 6.40e+1 0.0029 Kp* 1 1 8.00e+0 288 2.30e+3 0.0030 Ap 1 1 8.00e+0 1 8.00e+0 0.0031 Dst 1 1 8.00e+0 24 1.92e+2 0.0032 Dst* 1 1 8.00e+0 288 2.30e+3 0.0033 AE index 1 1 8.00e+0 1440 1.15e+4 0.0034-35 SSN 1 1 8.00e+0 1 8.00e+0 0.0036-38 IMF 1 2 8.00e+0 1440 2.30e+4 0.0039-43 B (magnetosphere) 1 20 2.40e+1 1440 6.91e+5 0.0744 B (ground) 1 100 2.40e+1 8640 2.07e+7 2.0145 IPS 1 3 8.00e+0 24 5.76e+2 0.0046-47 foF2, foE, foF1 3 40 8.00e+0 288 2.76e+5 0.0348-49 TEC 1 40 8.00e+0 288 9.22e+4 0.0150 Cross-tailE-field 1 1 8.00e+0 8 6.40e+1 0.0051 Ion drift velocity 1 10 8.00e+0 8640 6.91e+5 0.0752 Cold ions 1 4 8.00e+0 1440 4.61e+4 0.0053 1-10keV electrons 10 4 8.00e+0 1440 4.61e+5 0.0454-55 10-100keV electrons 10 4 8.00e+0 1440 4.61e+5 0.0456-58 � 10MeV ions 25 1 8.00e+0 48 9.60e+3 0.0059-61 � 10MeVprotons 5 3 8.00e+0 96 1.15e+4 0.0062-65 � 100MeVions(spectra) 4 1 8.00e+0 24 7.68e+2 0.0066-67 Relativistic electrons 10 3 8.00e+0 24 5.76e+3 0.0068 Atmosphericscaleheight 1 20 8.00e+0 24 3.84e+3 0.0069 Debrissize+ v-dist 1 1 8.00e+2 1 8.00e+2 0.0070-71 Meteoroidsize+ v-dist 1 1 8.00e+2 1 8.00e+2 0.0072 Doserate+ LET spectrum 5 10 8.00e+0 288 1.15e+5 0.0173 Totaldose 1 1 8.00e+0 1 8.00e+0 0.0074 Satelliteposition 1 20 2.40e+1 48 2.30e+4 0.00

GRAND TOTAL (Mb) 1.03e+9 100.00

Table10: Measurementdatarate




B Service Implementation Proposals

Table11 bringstogetherall theServiceFunctionalRequirements(SFRs)andServiceImplementationPro-posals(SIPs)from throughoutthedocument.Thelastcolumnis a cross-referenceto thedocumentsectionwhereeachSFRandSIPwasidentifiedanddiscussed.Themajornumberin eachSFRis theServiceUserRequirement(SUR)of which theSFRformsapart.

Table11: ConsolidatedServiceImplementationProposals

SFR SFR Description SIP SIP Description Sec

1.1 Allow dataretrieval usingtheFTP,HTTP andSMTPprotocols

1.1.1 UsetheFTP/HTTPagentGNU wgetfor FTPandHTTP retrieval andmirroring

3.1.2

1.1.2 OnUnix, therequirementsaremetbystandardMail Delivery Agentssuchassendmailin conjunctionwith procmailandthemail clientmail.

1.2 Dataretrieval mechanismsshouldbesufficiently flexible to copewith thesystemsof many differentdatasuppliers.

1.2.1 Provide for dataacquisitionusingallmethodsfrom Table3 excepttheRemote/Localcombination.

3.1.2

1.3 Datadistribution mechanismsshouldbesufficiently flexible to copewiththeneedsof many differentusers.

1.3.1 Provide for datadistribution usingallmethodsfrom Table3 excepttheRemote/Localcombination.

3.3

1.4 Allow datadistribution usingtheFTP, HTTP andSMTPprotocols

1.4.1 Offer active FTPdelivery of data(PUToperation)by usingappropriatesoftware,suchasexpector thePerlLWPandNet::FTPmodules.

3.3

1.4.2 Offer dynamicHTTP accessbyrunninga forms-drivenwebserviceontheapachewebserver [14].

1.4.3 Offer e-maildatadistribution usingastandardMail TransportAgentsuchassendmail, Mail UserAgentsuchasmail, or wrappedby higher-levelsoftwaresuchasthePerlMail::Mailermodule.

1.4.4 Considernovel datadistributionchannels(e.g.SMS)for selectedproducts(e.g.alerts)

1.5 Provide sufficientnetwork capacitytoallow promptaccessto SWSdataproductsby users

1.5.1 Procurea leasedline connectionwithbandwidthof at least5Mbits/s.

3.3

continued. . .




Table11: (continued)


1.6 Theon-linedatamustbeavailable24hours.

1.6.1 Useclusteredcomputingsystemswithintrinsic fail-over capabilities.

7.2.1

1.6.2 Seekto eliminatenetwork bottlenecksby supportingcapacityupgrades.

1.6.3 Purchasea same-daycall-outmaintenancecontractfor computinghardware.

1.7 Thedataarchive mustbesecure. 1.7.1 Carryoutdaily back-upof volatiledata.

7.2.1

1.7.2 Apply theusualsecuritymeasuresapplicableto publicly accessiblenetworkedcomputers.

1.8 Aim to acquireall relevantspaceweatherdatafor input to theSWS.

1.8.1 Maintaingoodrelationshipswith datasuppliersby keepingtheminformedofdatausage.

7.2.3

2.1 Dataretrieval mustbeableto proceedon apre-determinedschedulewithouthumanintervention.

2.1.1 TheUnix cron or Windows schedulefacilitiesshouldbeusedto startretrieval jobsasrequired.

3.1.2

2.2 Dataprocessinganddistribution mustbeableto proceedon apre-determinedschedulewithouthumanintervention.

2.2.1 TheUnix cron or Windows schedulefacilitiesshouldbeusedto startdataprocessingor distribution jobsasrequired.

3.2.3

3.1 Provide on-linehelpfor users 3.1.1 Providecontext-specifichelpon howto setupdataretrievals

5.1

3.1.2 Provideanon-linerouteto SWSsupportstaff to requestfurtherhelpwith operatingthesystem.

4.1 Storedatain amannerthatfacilitatestheassociationof datawith thenecessarymetadatafor its properuse.

4.1.1 Storedatain structuredfileswithintegratedprovision for metadata

3.2.4

5.1 Humanbackupto theon-linehelpsystems.

5.1.1 Providea telephoneande-mailhelp-deskwith definedresponsetimecharacteristics

5.3

6.1 Outputall datausingagenericandaccessibledataformat,forconsistency of access.

6.1.1 Outputdatain CDF or XDF formats. 3.3

6.1.2 FacilitatesimpleASCII outputbystructuringXDF filessothatthedataelementhassufficient internalstructurefor it to beintelligible in theabsenceof themetadataelements.

7.1 Allow thedataholdingsto besearchedby metadataattributes

7.1.1 Extractdatafilemetadataandstorein arelationaldatabase– a ‘DataDictionary’

3.2.6

8.1 Allow easyconfigurationandaccessto locationmetadata

8.1.1 Storelocationmetadatain a relationaldatabase– a ‘Yellow PagesDirectory’.

3.2.6

continued. . .






9.1 Enableusersto filter databy time 9.1.1 All datarequestsshouldspecifya timerange.

4.1

9.2 Enableusersto applysimpleaggregatingoperationsto datastreams.

9.2.1 Implementasuiteof aggregationfunctionsto computemean,median,minimum,maximumandsumonarbitrarydatafieldsover regularrepeatingintervalsof timeor space.

4.1

10.1 Offer awide rangeof modelsandforecastingtools.

10.1.1 Themodelsin Table6 shouldall beprovided.

3.3

10.2 Allow modelsto beinterconnected 10.2.1 For themodelsin Table6, passingstandardoutputdatafiles in CDF orXDF formatis sufficient.

4.2.2

10.2.2 Runmodelsat themostappropriatelocation,whichmaybeata remoteinstitution.

10.2.3 Maintainawarenessof developmentsin theGrid field, andon possibleapplicationswithin spaceweathermodelling.

10.3 Offer modelsto berun routinelyoron demandasappropriate

10.3.1 Themodelswith S(tandard)modeinTable6 shouldgeneratestandardoutputsthatarearchivedroutinely.

4.2.3

10.4 Ensurethatgenerateddataproductsfrom modelsandforecastsareofappropriatequality

10.4.1 Documenttheconfigurationof allmodels,whetherrun locally orremotely.

7.2.5

10.4.2 Maintainmodeldocumentationandsourcecodeunderformal revisioncontrol

11.1 Maintainlocal databasesof retrieveddata,estimatedto accumulateat200-1000Mb/day.

11.1.1 UseaRAID systemwith capacityofat leastseveralmonthsof dataforimmediateon-lineaccess,withnear-on-linetapestorageholdingtheremainingdata.

3.2.1

11.2 Storedatasothattheoperationaldataenvironmentatany point in time isrecoverableat any latertime.

11.2.1 Updatedatanon-destructively. 3.2.2

11.2.2 Timestampall datafilesor valueswiththeir timeof retrieval.

11.3 Storedatain amannerthatsimplifiesingestion,processinganddistribution.

11.3.1 Storedatain structuredfiles ratherthanrelationalor object-orienteddatabasesystems

3.2.4

continued. . .






11.4 Ensurethatretrieveddatais ofappropriatequality

11.4.1 Log all dataretrievals 7.2.3

11.4.2 Only ingestincomingdatathathasbeencheckedasfreeof formaterrors

11.4.3 Endeavour to checkincomingdataforplausiblevalues

12.1 Enableusersto build personalsetsofdataretrieval actions

12.1.1 For all actions,usersmustbeabletospecifyandstorewhentheactionshouldoccurandits repeatinterval.

5.1

12.1.2 Allow theformatof retrieveddataproductsto beuser-specifiable(CDF/XDF/plot)

12.1.3 Allow thedatatransferprotocolto beuser-specifiable(HTTP/FTP/SMTP).

12.1.4 For custommodelinvocation,allowusersto supplyamodel-specificsetofparametervalues.

12.1.5 Userprofilesmustbepasswordprotectedsothatthey areaccessibleonly to thecontrollinguser.

12.2 Usersshouldbeableto determinewhatdataproductsthey haveretrieved.

12.2.1 Log all dataoutputsfrom theSWS,centrallyandby user.

7.2.4

12.2.2 Allow usersto inspecttheirpersonalretrieval logs.

13.1 Be ableto supporttheoperationofthecomputingsystems

13.1.1 Employ at leastthreestaff membersdedicatedto computersupport

7.2.2

13.1.2 Computingsupportstaff shouldbecontactable24 hours.

13.1.3 Uselocalandremoteautomaticsystemsto monitorsystemavailability

13.2 Be ableto handlescientificallysophisticateduserqueriesanddeveloptheserviceappropriately.

13.2.1 Haveasuitablenumberof staffdedicatedto scientificsupportandconsultancy: aminimumof 9 issuggested(Plasmaandradiationenvironment– 3, Solid bodyenvironment– 1, Magneticenvironment– 2, Atmosphericenvironment– 3)

7.2.2

13.3 Be ableto promotepublicunderstandingof spaceweather

13.3.1 Employ at leastonestaff memberwhoseprimaryfocusis publicoutreach,in tandemwith somescientificknowledge.

7.2.2

continued. . .






13.4 Staff competenciesmustdevelopinline with changesin science,technologyandthepublicenvironment.

13.4.1 Provideon-goingtrainingfor all staffrelevantto their roles.

7.2.2

14.1 Easyon-lineaccessto introductorymaterialon thescienceandimpactofspaceweather.

14.1.1 Provideasetof tutorialwebpages;theLundsite[38] is agoodstartingpoint.

5.2

14.1.2 Constructageneraluseraccountthatgeneratesaselectionof mainlygraphicaldataproductsto accompanyastatictutorial.

15.1 To raisegeneralpublicawarenessofspaceweatherandits impacts.

15.1.1 Preparetemplateinformationfor pressreleasesto beusedwhenaspaceweathereventsis newsworthy.

6.1

15.1.2 Commission/designportableexhibitsfor publicdisplay.

15.1.3 Preparematerialandpersonnelforgiving publicpresentations.

15.2 To promoteawarenessof spaceweatherandspacesciencein schools.

15.2.1 Developshortstudyunitsbasedonspaceweathertopics.

6.2

15.3 To promoteawarenessof spaceweatherin universities.

15.3.1 Collaboratewith theESAEducationOffice.

6.3

16.1 Provide thecapabilityof generatingplotsfrom dataoutputs

16.1.1 UsethegenericdatavisualisationpackageIDL.

4.3

16.1.2 If usingCDF, usetheCDAWlibplotting routineson topof IDL.

17.1 Ensurethatgenerateddataproductsfrom modelsandforecastsareofappropriatequality

17.1.1 Validateat leastsomeof all modeloutputsagainstdata

7.2.4

17.1.2 Periodicallyassesstheaccuracy ofpredictions

18.1 On-linemechanismsfor userfeedback

18.1.1 Provideabug-reportingon theon-linesystem

5.1

18.1.2 Provideausersuggestionsordiscussionforum

18.2 SWSdevelopmentto reflecttheinterestsandconcernsof theusercommunityandtheevolving stateofSTP.

18.2.1 SWSstaff to make regularvisits toscientificinstitutionsandserviceproviders

5.3

18.2.2 Runworkinggroupsfocusedonspecificpartsof thescienceandapplicationsof spaceweather.

continued. . .






19.1 Fostercollaborationbetweenrelevantgroupsin Europe.

19.1.1 Supporttheuseof theHLA [48] in theCTIP [45] andGUMICS[46] models.

8.2

19.1.2 Developsimpleprotocolsfor modelinvocationandinterrogationandsupporttheir implementationusingappropriatetechnologies(e.g.CORBA [50], Jini [51]).

19.1.3 Aim to establishaprogrammeofsecondmentof personnelfromnationalinstitutesto theSWS,alongthelinesof theECMWF.

8.3