unclassified net-centric spectrum data transformation fred nelson defense spectrum organization...

UNCLASSIFIEDUNCLASSIFIED

Net-Centric Spectrum Data Net-Centric Spectrum Data TransformationTransformation

Fred Nelson

Defense Spectrum Organization

Joint Spectrum Center/J5

9 December 2009

2

A Combat Support Agency


DisclaimerDisclaimer

*******************************************************************************

The information provided in this briefing is for general information purposes only. It does not constitute a commitment on behalf of the United States Government to provide any of the capabilities, systems or equipment presented and in no way obligates the United States Government to enter into any future agreements with regard to the same. The information presented is for the purposes of presentation at the 2009 DoD Spectrum Workshop and may not be disseminated further without the express consent of the United States Government.

*******************************************************************************

3



Outline

• Overview of Net-Centric Spectrum Data Transformation

• Why DSO is Doing This• Joint Spectrum Data Repository (JSDR) and

Interfaces• Transition Process and Milestones

4



Overview of Net-Centric Spectrum Data Transformation

• Provide initial Military Communications Electronics Board (MCEB) Pub 8/Spectrum Management Allied Data Exchange Format (SMADEF) XML compliant data repository– New interfaces to Net-Centric JSDR– Registration of data and services

• Metadata Registry: Go to https://metadata.dod.mil/mdr/homepage.htm and establish your account

– Pub 8 Version 1.2.4b XSD registered under namespace called “SSRF” under "DODENT: DoD Enterprise" tab

• Pub 8 XSD will also be registered under the DoD IT Standards Registry (DISR) (https://disronline.disa.mil/)

• JSDR Service Interface (SI) will be registered under the NCES Service Registry (https://service.nces.dod.mil/wasp/uddi/bsc/web)

• JSDR, JSDR SI, and JSC Data Access Web Server (JDAWS) are registered in DoD Information Technology Portfolio Repository (DITPR) (https://ditpr.dod.mil/)

– Measured Data Capability (MDC)• Focus is on supporting Multi-spectral Ambient Noise Collection &

Analysis Tool (MANCAT) and other measured data compliant with Pub 8 XML standard

5



WHY DSO IS DOING THIS!

6



Tool Capability and Data Quality Are Out of Balance

• Improved tools, algorithms, and workflow being developed to meet the coming spectrum management challenges

– SXXI Online, NSTs, HNSWDO, CJSMPT, SPEED, AESOP, WIN-T, etc.– Significant enhancements of algorithms

• More accurate calculations, less need for margin• Decreased need for human intervention, faster response

– Utilize automated business process tools to baseline processes and identify targets for improvement

• Develop/generate/track metrics to prioritize and guide activities/initiatives

• These tools rely on accurate/trusted and complete data– Inventory of emitters and receivers– Detailed equipment characteristics

• Data is not keeping up with the tools– Accuracy– Currency– Completeness– Assurance

ToolsTools

DataData

7



Spectrum DataTransformation

Today

Pub 8*

2010+

SFAFDD 1494Allocation TablesChannel PlansAllotment PlansInterference Reports

Pub 8 will substantially change the way spectrum operations is done.

* Military Communications Electronics Board (MCEB) Pub 8 defines the XML-based SSRF

…but the user doesn’t need to know Pub 8.

• Improve Information Exchange and Interfaces (Harmonized XML)– NATO - Spectrum Management Allied Data Exchange Format (SMADEF)– US NTIA – Office of Spectrum Management Data Dictionary (OSMDD)– US DoD – MCEB Pub 8 Standard Spectrum Resource Format (SSRF)

8



Net-Centric Data Strategy

Ensure widely used standard data/internet formats and protocols are implemented (e.g., Pub 8/SMADEF XML and UDDI, WSDL, Web 2.0, XML, SOAP, etc. controlled by international consortiums such as WWC and OASIS)

Accessible

Understandable

Trusted

Interoperable

Responsive

Institutionalize

Visible Make information resources discoverable by most users (MDR, DISR, NCES Service Registry, DITPR)

Make spectrum data available on the network (JDAWS v7) and available to tools using a machine-to-machine interface (JSDR SI)

Improve the accuracy and currency of the data available to users by using new/enhanced data capture tools (Stepstone v2.0 Editor & SXXI Online) and authoritative data sources

Ensure it can be intelligibly used by the consumers and the data format/structure is well documented in MCEB Pub 8

Ensure the data can be imported/exported between tools and databases without costly translation costs and schedule delays

Ensure the data is what users need and robust user feedback mechanisms are in place to improve it (One-stop Helpdesk)

Need to comply with the DoD Net-centric Data Strategy to make the Department’s information resources:

9



JSDR AND INTERFACES

10



Joint Spectrum Data Repository

Priority Databases that will be transformed:• Frequency Environment (FE)

– Frequency Resource Record System (FRRS)– Government Master File (GMF)– Background Environment Information (BEI) [Underlying BEI Data Sources]

• JSC Equipment, Tactical and Space (JETS) [Underlying JETS Data Sources ]

• Joint Spectrum Interference Resolution (JSIR) database

Databases that will not be transformed:• Electronic Order of Battle (EOB) [Future Objective - Data from E-SPACE]

– Links provided to correlate EOB records with Blue/USA records in JETS by ELNOT

• Historical Spectrum Certification System (SCS) [DoD J/F-12 Records]• DoD EL-CID records• Joint Ordnance E3 Risk Assessment Database (JOERAD) susceptibility

data record• Area Studies (Posted on DKO portal not in JSDR -

https://www.us.army.mil/suite/folder/9906701)• Host Nation Spectrum Worldwide Database Online (HNSWDO)

11



JSDR InterfacesJoint Spectrum Data

Repository (JSDR)

Pub 8/SMADEF(XML)

Pub 8/SMADEF(XML)

JSDR Service Interface

AESOP Afloat EM Spectrum Operational ProgramDKO Defense Knowledge OnlineEL-CID Equipment Location - Certification Information DatabaseEOB Electronic Order of Battle DatabaseFE Frequency Environmental DatabaseHNSWDO Host Nation Spectrum Worldwide Database OnlineJETS JSC Equipment, Tactical, & Space DatabaseJSIR Joint Spectrum Interference Resolution DatabaseMANCAT Multi-spectral Ambient Noise Collection & Analysis ToolSCS Spectrum Certification SystemSMADEF Spectrum Management Allied Data Exchange FormatJDAWS JSC Data Access Web ServerNCES Net-Centric Enterprise ServicesNST Net-Centric Spectrum ToolsSMTI Spectrum Management Transition InitiativeSPEED Systems Planning Engineering & Evaluation DeviceWIN-T Warfighter Information Network - Tactical

JETSJETS

FEFE

EOBEOB

JSIRJSIR

SCS Historical Archive

SCS Historical Archive

HNSWDOHNSWDO

JDAWS Version 7(Ad hoc query

whole database)

JDAWS Version 7(Ad hoc query

whole database) DKO Link

Will be registered service under NCES

NOTE: Pub 8 XML master schema has been registered under NCES Meta Data Registry (MDR)

NOTE: The Space and Extended TX/RX elements in Pub 8 Volume IV will not be included in the JSDR Service Interface

SPEEDSPEED

SXXI OnlineSXXI Online

WIN-TWIN-T

AESOPAESOP

Stepstone v2.0Stepstone v2.0

MANCAT/MDCMANCAT/MDC

NSTNST

Stepstone Version 2.0

EL-CIDEL-CID

MANCAT/MDCMANCAT/MDC

Pu

b 8

Com

pli

ant

Dat

aN

on-P

ub

8 C

omp

lian

t D

ata

Prototype

12



Cross Domain Solution (CDS)

ExternalCDSFilter

Extract, Transform, &Load (ETL)

Pub 8 XMLInterface

JSDR EnterpriseDatabase

JSDR EnterpriseDatabase

JDAWS V7JSDR Service

Interface

MDCOLTP DB

Stepstone V2.0

HNSWDOOLTP DB

SXXIOOLTP DB

Pub 8 XML Data & Text (Attachments only from low to high)

SIPRNET NIPRNET

MDCOLTP DB

StepstoneV2.0

HNSWDOOLTP DB

SXXIOOLTP DB

JDAWS V7JSDR Service

InterfaceU U U U

KEY:

Data flow for future capability

Data flow for future service interface

Data flow capability planned under Net-Centric Spectrum Transformation project

- Green figures represent capability planned under Net-Centric Spectrum Transformation project- Purple figures represent future capability- HNSWDO is an existing capability

Note: Also will provide filter to push data/text one way from SIPRNET to JWICS.

13



Data Quality Metrics

• JSC Data Access Web Server (JDAWS) Version 7 will be used to view and monitor metrics relating to data quality issues in Pub 8 compliant portion of JSDR– A set of default reports will be created to represent common

data quality metrics analysis needs– Ad hoc queries will permit users to create their own custom

reports to focus on their area of concern• Example: Spectrum Operations Permanent Working Group

(SO PWG) members will be able to create custom reports that display the metrics which help them identify specific data quality problems with FRRS records and is organized by responsible service (e.g., searchable by Agency Serial Number)

14



Other Planned Initiatives

• Add three new major data sources to the JSDR and expose in Pub 8 XML– Worldwide Table of Allocations (TOAs)

– Federal Aviation Administration (FAA) flight-line data

– NGA Digital Aeronautical Flight Information File (DAFIF)

• Expand and improve quality of dataset for FCC database records – Expose in Pub 8 XML via JDAWS v7 and JSDR SI

– Expose in SFAF via JDAWS Community Site on DKO• URL will direct users to a PerCon server to query and download FCC

and ITU records

• Develop and demonstrate a prototype of an enhanced JSDR SI to automate the process for obtaining Pub 8 data directly from Authorized Data Sources (ADSs) or Data Capture tools (e.g., Stepstone v2.0) without the need for human intervention– Proof of Concept; not an operational capability

15



TRANSITION PROCESS AND MILESTONES

16



Transition Process

• Execute transition process to move from legacy to new Net-Centric JSDR– Users move from legacy JDAWS v6 to JDAWS v7

• Legacy JDAWS v6 only available on SIPRNET and JWICS• JDAWS v7 will be available on NIPRNET, SIPRNET, and JWICS• JDAWS application (Version 7.2) available on SIPRNET in Jan 2010• Target date for retiring legacy JDAWS v6 is Sept 2010

– Applications must be modified to exchange data in SSRF and interface to new Net-Centric JSDR through JSDR Service Interface

• Continue to coordinate with USN, USMC, USAF, and USA to address implications of service-specific applications that utilize legacy database formats– Parallel transition process for supporting old/new database formats

• Will continue to distribute SFAF records via the Frequency Assignment Retrieval System (FARS) CDs semiannually for FRRS records

– Plan to transition from ITU and FCC FARS CDs to on-line access by Mar 2010• Plan to transition from SCS/EL-CID CDs to on-line access by Feb 2010

– Target date for dropping SFAF: End of FY2012

17



Key Planned Dates

• JSDR v1.0, JDAWS v7.2, and JSDR SI v1.0 IOC on SIPRNET – Jan 2010– Implements Pub 8 Version 1.2.4b (Volumes I, II, III, and VI)

• Measured Data Capability (MDC) on SIPRNET – Apr 2010• JSDR v2.0, JDAWS v7.3, JSDR SI v2.0, CDS, and MDC (NIPRNET) –

Aug 2010– Will implement draft of SMADEF v2.0 (Volumes I, II, III, IV, and VI)

– CDS filters will be bi-directional between NIPRNET and SIPRNET; and one way from SIPRNET to JWICS

– Will include a more accurate and expanded FCC dataset

– Will include Tables of Allocation for about half of the 191 ITU countries, FAA Flight-line data, and National Geospatial-Intelligence Agency (NGA) Digital Aeronautical Flight Information File (DAFIF) in Pub 8 XML

– Informal user testing (Not PATE) – Start around June 2010


www.disa.mil

19



BACKUP SLIDES

20



Underlying BEI Data Sources

• Source Data for BEI Database– International Telecommunication Union

(ITU)/International Frequency List (IFL) – International Telecommunication Union

(ITU)/Space Radio Systems (SRS)– Federal Communications Commission (FCC)– Canadian Tactical and Administrative Frequency

List (TAFL)– Radio Astronomy (RA)

21



Underlying JETS Data Sources• JETS Database Sources

– Data Sources Mapped into JETS Database• Global Status of Resource and Training System (GSORTS)• Army Master Data File (AMDF)• Army Continuous Balance System - Expanded (CBS-X)• The Army Authorization Document System (TAADS)• Air Force Avionics Planning Baseline (APB) [May be cancelled due to funding cuts]• Air Force Aerospace Vehicle Inventory (AVI)• Air Force Ground (AFG) Logistics Detail (LOGDET)• Navy Afloat Master Planning System (AMPS)• Marine Corps Total Force Structure Management System (TFSMS)• Naval Avionics Installation Plan (NAIP) [Cancelled – Using ASEMICAP as new source]• Satellite Two-Line Element Datasets from USSTRATCOM• Space International Frequency Information Circular (IFIC) from ITU-Radio

communications Bureau – Sources Used to Manually Augment JETS Records Referenced within Bibliography

• United States Electromagnetic Systems (USELMS)• Electronic Warfare Information Reprogramming (EWIR)• Electronic Intelligence Parameter List (EPL)• Spectrum Certification System (SCS)• Federal Logistics Database (FEDLOG)• ITU Space Network systems (SNS) [Also source of IFIC]• The Satellite Encyclopedia (TSE) from Tag’s Broadcasting Service (TBS)• Miscellaneous secondary references (e.g., technical manuals, manufacturer

brochures, etc.)

Platform/Unit data sources

Future objective is to use E-SPACE as federate data source

22



Tables of Allocation

ALBANIAANTIGUA AND BARBUDAARGENTINAARMENIAAUSTRALIAAUSTRIABAHAMASBANGLADESHBARBADOSBELARUSBELGIUMBOLIVIABOSNIA AND HERZEGOVINABRAZILBULGARIACANADACHILECHINACOLOMBIACOSTA RICA

CROATIACYPRUSCZECH REPUBLICDENMARKDOMINICAN REPUBLICEUROPEAN COMMON ALLOCATION (ECA)ECUADOREL SALVADORESTONIAETHIOPIAEUROPEAN UNIONFEDERATED STATES OF MICRONESIAFINLANDFRANCEGEORGIAGERMANYGIBRALTORGUATEMALAGUYANAHONDURAS

HONG KONGHUNGARYICELANDINDIAIRELANDITALYITU-R1ITU-R2ITU-R3JAMAICAJAPANKAZAKHSTANLATVIALIECHTENSTEINLITHUANIALUXEMBOURGMALAYSIAMALTAMAURITIUSMEXICOMOLDOVANETHERLANDS

NEW ZEALANDNICARAGUANORWAYORGANIZATION OF EASTERN CARIBBEAN STATES (OECS)PANAMAPARAGUAYPERUPORTUGALQATARROMANIARUSSIASAMOASENEGALSEYCHELLESSINGAPORESLOVAKIASLOVENIASOUTH AFRICASOUTH KOREA

SPAINSURINAMESWEDENSWITZERLANDTAIWAN TAJIKISTANTHAILANDTHE FORMER YUGOSLAVREPUBLIC OF TRINIDAD AND TOBAGOTURKEYUGANDAUKRAINEUNITED KINGDOMUNITED STATESURUGUAYUZBEKISTANVENEZUELAVIETNAM

Key:NATO Countries highlighted in purple. Greece and Poland are the only 2 of the 28 NATO countries not completed yet.Six Organizational TOAs highlighted in greenNote: All five CCEB countries have been completed (US, UK, NZ, AUST, and CAN).

23



Milestones2008

1 2 3 4 5 6 7 8 9 10 11 12

20091 2 3 4 5 6 7 8 9 10 11 12

20101 2 3 4 5 6 7 8 9 10 11 12

Implementation Plan

Drafts of JSDR SI Consumer Development Guide

Measured Data Capability(MDC) IOC

SCS to EL-CID Data Conversion (Increment 2)

Metadata Registration

Prototype Service Interface to demonstrate upload of Pub 8 XML data

Query Outputs in SFAF Discontinued (Legacy JDAWS Retired)

JSDR v1.0, JDAWS v7.2 & JSDR SI v1.0 IOC (SIPRNET)

JSDR v2.0, JDAWS v7.3 & JSDR SI v2.0

CDS IOC

unclassified net-centric spectrum data transformation fred nelson defense spectrum organization...

Documents