update on the collaborative radar acquisition field test (craft): planning for the next steps

Update on the Collaborative Update on the Collaborative Radar Acquisition Field Radar Acquisition Field

Test (CRAFT): Test (CRAFT): Planning for the Next StepsPlanning for the Next Steps

Kelvin K. DroegemeierKelvin K. DroegemeierUniversity of OklahomaUniversity of Oklahoma

Presented toPresented toNEXRAD Technical Advisory CommitteeNEXRAD Technical Advisory Committee

3 December 20013 December 2001

The Collaborative Radar The Collaborative Radar Acquisition Field Test Acquisition Field Test

(CRAFT)(CRAFT) A Collaborative project to demonstrate the real A Collaborative project to demonstrate the real

time compression and Internet-based time compression and Internet-based transmission of WSR-88D base (Level II) datatransmission of WSR-88D base (Level II) data

Motivated by Motivated by – Storm-scale NWP research at CAPS (now for WRF)Storm-scale NWP research at CAPS (now for WRF)– Decision support system development (NSSL)Decision support system development (NSSL)– Desire to improve long-term archive rate (NCDC)Desire to improve long-term archive rate (NCDC)

Philosophy: Leverage existing infrastructures Philosophy: Leverage existing infrastructures (Internet), public-domain software (Unidata (Internet), public-domain software (Unidata LDM), and low-cost services (56K phone lines)LDM), and low-cost services (56K phone lines)

CRAFT was established by the OU Center CRAFT was established by the OU Center for Analysis and Prediction of Storms for Analysis and Prediction of Storms (CAPS) in collaboration with the (CAPS) in collaboration with the – UCAR Unidata Program OfficeUCAR Unidata Program Office– NOAA WSR-88D Operational Support Facility NOAA WSR-88D Operational Support Facility

(now Radar Operations Center)(now Radar Operations Center)– University of WashingtonUniversity of Washington– National Severe Storms LaboratoryNational Severe Storms Laboratory

Now coordinated by CAPS, NSSL, ROC, Now coordinated by CAPS, NSSL, ROC, Unidata, and NCDC Unidata, and NCDC

ParticipantsParticipants

Current Technical StrategyCurrent Technical Strategy

Repeater Hub

RIDDS Linux PC + LDM + Compression

($1000)

LDM Server

Dedicated 56K line or ($2000 - $6000/year)

or NWS LANOptional Router

($2000)

WSR-88D

At the radar site

Internet or Abilene

BZIP2 off-the-shelf compression algorithmBZIP2 off-the-shelf compression algorithm– maximum 16:1 in clear air; 8:1 in the worst case; maximum 16:1 in clear air; 8:1 in the worst case;

average of about 12:1average of about 12:1 56K lines found to be inadequate in some cases 56K lines found to be inadequate in some cases

(squall lines) – maybe 10% of time(squall lines) – maybe 10% of time– Exceed available bandwidth by 10-20%Exceed available bandwidth by 10-20%– Can create latencies of 10s of minutesCan create latencies of 10s of minutes– Up to 26% improvement in compression possible in Up to 26% improvement in compression possible in

BZIP2, but requires changing code at user endBZIP2, but requires changing code at user end– A pre-processing algorithm (NSSL) provides 10% A pre-processing algorithm (NSSL) provides 10%

improvement – no changes in BZIP requiredimprovement – no changes in BZIP required– 56K line overrun may not be an issue – used for initial 56K line overrun may not be an issue – used for initial

testing because of cost and securitytesting because of cost and security– New scan strategies and availability of cheaper, faster New scan strategies and availability of cheaper, faster

links (even cable modems) may render 56K lines mootlinks (even cable modems) may render 56K lines moot

Technical HighlightsTechnical Highlights

Radars Now Delivering Real Radars Now Delivering Real Time Level II Data to NCDC/OUTime Level II Data to NCDC/OU

Cedar City

TulsaLas Vegas

Flagstaff

Tucson

Amarillo

Lubbock

Oklahoma CityFort Smith

Dallas/Fort Worth

WilmingtonYuma Phoenix

Seattle

Huntsville

Norfolk

Charleston

Morehead City

Columbia

Raleigh

Denver

Courtesy Steve DelGreco (NCDC)

Old WSR-88D Level II Data Processing at Old WSR-88D Level II Data Processing at NCDCNCDC

WSR-88DWSR-88D

Abilene T3Abilene T3

Compressed Level II DataCompressed Level II DataFrom 21 RadarsFrom 21 Radars

LDM HDSS

MassMass Storage Storage

At NCDC TodayAt NCDC Today

Modified from Steve DelGreco (NCDC)

via FTPvia FTP

Processing/Processing/RecompressionRecompression

WSR-88D Level II Archival at WSR-88D Level II Archival at NCDCNCDC% Rcpts % Rcpts #Sites#Sites

Level II NWSLevel II NWS 64.964.9 120120Level II DoDLevel II DoD 35.735.7 2626Level II FAALevel II FAA 9.59.5 1212

Total 88D NetworkTotal 88D Network 55.755.7158158

WSR-88D Level II Archival at WSR-88D Level II Archival at NCDCNCDC

% Rcpts % Rcpts #Sites#Sites

Level II NWSLevel II NWS 64.964.9 120120Level II DoDLevel II DoD 35.735.7 2626Level II FAALevel II FAA 9.59.5 1212

Total 88D NetworkTotal 88D Network 55.755.7158158

CRAFT EFFICIENCY IS ABOVE 95%CRAFT EFFICIENCY IS ABOVE 95%

University of Iowa requested 19.4 gigabytes University of Iowa requested 19.4 gigabytes (uncompressed) of Level II data(uncompressed) of Level II data

The NCDC was able to retrieve it from the The NCDC was able to retrieve it from the HDSS, compress it, and stage it to FTP in HDSS, compress it, and stage it to FTP in 2 2 hours and 5 minuteshours and 5 minutes

Data was then moved over the Internet in a Data was then moved over the Internet in a matter of minutesmatter of minutes

The old 8 mm tape system would have The old 8 mm tape system would have required required – 58 tapes58 tapes– 9 workstations each running for 26 hours9 workstations each running for 26 hours– A total of A total of 232 hours232 hours

Improvement is more than a factor of 100 in Improvement is more than a factor of 100 in time, not including personnel coststime, not including personnel costs

A Notable NCDC Success A Notable NCDC Success StoryStory

0 5 10 15 20 25 30

Radar Data LatencySEAAMACMBFLGFTWSRPTULLBBFSMOKCTCS

PreliminaryPreliminary Latency DataLatency Data

ESDIM is fundingESDIM is funding– Analysis of latency, bandwidth, etcAnalysis of latency, bandwidth, etc– Network simulation (reliability, Network simulation (reliability,

redundancy)redundancy) Additional analysis being conducted in Additional analysis being conducted in

collaboration with the private sectorcollaboration with the private sector

Networking R&DNetworking R&D

Additions by End of CY 2001Additions by End of CY 2001

Cedar City

TulsaLas Vegas

Flagstaff

Tucson

Amarillo

Lubbock

Dallas/Fort Worth

Seattle

Huntsville

Norfolk

Charleston

Morehead City

Columbia

Raleigh

Denver

Rapid City

Melbourne

Buffalo

Chicago

Pittsburgh

Cleveland

Cincinnati

Detroit

N. Indiana

21+ 21+ 1111

Cheyenne

Goodland

Cedar City

TulsaLas Vegas

Flagstaff

Tucson

Amarillo

Lubbock

Dallas/Fort Worth

Seattle

Huntsville

Norfolk

Charleston

Morehead City

Columbia

Raleigh

Denver

Rapid City

Melbourne

Buffalo

Chicago

Pittsburgh

Cleveland

Cincinnati

Detroit

N. IndianaCheyenne

Goodland

Dodge CityWichita Springfield

Little Rock

Shreveport

Kansas City

Additions by Spring, 2002Additions by Spring, 2002

21+ 21+ 11 + 11 + 66

IHOP Field ProgramIHOP Field ProgramPlus CAPS FAAPlus CAPS FAAResearchResearch

Cedar City

TulsaLas Vegas

Flagstaff

Tucson

Amarillo

Lubbock

Dallas/Fort Worth

Seattle

Huntsville

Norfolk

Charleston

Morehead City

Columbia

Raleigh

Denver

Rapid City

Melbourne

Buffalo

ChicagoPittsburgh

Cleveland

Cincinnati

Detroit

N. Indiana

Little Rock

Shreveport

Kansas City

Cheyenne

Goodland

Military RadarsMilitary Radars

21+21+11+11+66+2+2

Cedar City

TulsaLas Vegas

Flagstaff

Tucson

Amarillo

Lubbock

Dallas/Fort Worth

Seattle

Huntsville

Norfolk

Charleston

Morehead City

Columbia

Raleigh

Denver

Rapid City

Melbourne

Buffalo

ChicagoPittsburgh

Cleveland

Cincinnati

Detroit

N. Indiana

Little Rock

Shreveport

Kansas City

Cheyenne

Goodland

FAA-MIT/LL Additions FY 2002FAA-MIT/LL Additions FY 2002

Central Ill

Milwaukee

Quad Cities

St. Louis

Paducah

Louisville

Charleston

Baltimore

Central PAPhiladelphia

New York City

AlbanyBoston

Binghamton

4040+14=+14=5454

Current Technical StrategyCurrent Technical Strategy

Repeater Hub

RIDDS Linux PC + LDM + Compression

($1000)

Server

Dedicated 56K line or ($2000 - $6000/year)

or NWS LANOptional Router

($2000)

WSR-88D

At the radar site

Internet or Abilene

Interim BDDSInterim BDDS

Repeater Hub

BDDS Linux PC + LDM + Compression

($1000)

Server

Dedicated 56K lines or or NWS LAN

Optional Router ($2000)

WSR-88D

At the radar site

Internet or Abilene

Final BDDSFinal BDDS

Repeater Hub

BDDS+LDM/Compression

ServerDedicated 56K lines or or

NWS LAN

WSR-88D

At the radar site

Internet or Abilene

How Does This How Does This Experiment Experiment Become an Become an Operational Operational

System?System?

Abilene NetworkAbilene NetworkJanuary 1999 January 1999

New York

Atlanta

Indianapolis

Kansas City

Houston

Denver

Los Angeles

Sacramento

Seattle

Abilene Router Node

Abilene Access Node

Operational January 1999

Planned 1999

One Possible StrategyOne Possible Strategy

NOAA Facility

via phone linesor commodity

Internet

Silver SpringSilver Spring

Internet

Abilene NetworkNOAA Facility

NOAA Facility NOAA Facility

NOAA Facility

NormanNorman

BoulderBoulder

Silver SpringSilver Spring SeattleSeattle

MiamiMiami

AtlantaAtlanta

Internet

NOAA Facility

NormanNorman

BoulderBoulder

Silver SpringSilver Spring SeattleSeattle

MiamiMiami

AtlantaAtlanta

““Super Sites”: Each LDM on Abilene Super Sites”: Each LDM on Abilene ““bus” Carries all 88D data -- redundancybus” Carries all 88D data -- redundancy

NOAA Facility

Abilene Network

Commodity InternetCommodity Internet

Commodity InternetCommodity InternetCommodity InternetCommodity Internet

Commodity InternetCommodity Internet

odity Internet

modity In

ternet

NOAA Facility

Private Sector

Academia

NOAA Facility

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

NOAA Facility

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

NOAA Facility

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Private Sector

Academia

Meeting all of the NeedsMeeting all of the Needs

88D 88D 88D 88D 88D

Abilene-based network of (aggregating & relaying) LDMs, each carrying all 88D data, with redundant interconnections

Private 3

Academia 2Private 3

Academia 1

Private 4

Private 1

NOAA 3

NOAA 2

NOAA 1

Courtesy D. Fulker, UCAR

Meeting all of the NeedsMeeting all of the Needs

88D 88D 88D 88D 88D

Abilene-based network of (aggregating & relaying) LDMs, each carrying all 88D data, with redundant interconnections

Private 3

Academia 2Private 3

Academia 1

Private 4

Private 1

NOAA 3

NOAA 2

NOAA 1

Courtesy D. Fulker, UCAR

– NOAA runs its operational ingest system but links to NOAA runs its operational ingest system but links to private sector and academic systems, each providing private sector and academic systems, each providing redundancy for the otherredundancy for the other

– Entire community benefits – builds upon the Entire community benefits – builds upon the collaborative spirit of CRAFTcollaborative spirit of CRAFT

– Configuration completely scalableConfiguration completely scalable– Leverages existing infrastructure (LDM sites, Abilene, Leverages existing infrastructure (LDM sites, Abilene,

WFO connection to commodity Internet)WFO connection to commodity Internet)– Significant capacity for future growth (dual-pol, Significant capacity for future growth (dual-pol,

phased array)phased array)

Benefits of This ConceptBenefits of This Concept

Weber (2000)

Longer Term: Addition of FAA Longer Term: Addition of FAA RadarsRadars

update on the collaborative radar acquisition field test (craft): planning for the next steps

Documents

writer’s craft notebooks analyzing author’s craft to...

craft research and craft based project

red atc radar harmonised standards aeronautical radar...

pulsed radar (ii) radar equation

radio and radar: radar

the collaborative radar acquisition field test (craft): a...

1592318817680445 - dodo...

responsive systems comparison method: dynamic insights into...

design & craft, noida, craft items

earth observation services in collaborative...

collaborative adaptive sensing of the atmosphere...

by authority of - public.resource.org · performance...

22 m fast rescue craft - swede ship marine, · pdf file22 m...

workforce concerns? construction career collaborative (c3) a...

marconi radar history / marconi radar

casa – collaborative adaptive sensing of the atmosphere...

american craft council | american craft council

assessing advances in the assimilation of radar data ......1...

coconut craft - craft mark craft.pdf · coconut craft ....

the atmospheric imaging radar – ubiquitous radar ·...