factors(aﬀec+ng(weather( forecast(skill(in(the(arc+c( › faculties › environment › ceos ›...

Factors affec+ng weather forecast skill in the Arc+c

Gordon McBean and Ken Porter University of Western Ontario

STAR Final Workshop Winnipeg, June 14th 2010

Factors affecting weather forecast skill in the Arctic

•  Observational information on the “state” of the weather

•  Variability of the weather •  Understanding of the weather •  And ability to “see ahead” – forecast •  Knowledge of the individual forecaster •  Location of the forecaster

Inuvik

Iqaluit Rankin Inlet

Cambridge Bay

Resolute

Calgary Winnipeg Regina

The Pas Edmonton

Fort McMurray

Site Lat Long Radar Radiosonde

Edmonton 53.32 113.58 Yes Yes

Calgary 51.11 114.02 Yes

Fort McMurray 56.65 111.22 Yes

Regina 50.43 104.67 Yes

Winnipeg 49.92 97.23 Yes

Saskatoon 52.17 106.72 Yes

The Pas 53.97 101.1 Yes

Site Lat Long Radar Radiosonde

Iqaluit 63.75 68.55 Yes

Inuvik 68.3 133.48 Yes

Cambridge Bay 69.12 105.14 Yes

Resolute 74.72 94.99 Yes

Rankin Inlet 62.82 92.12

Observational information

0

10

20

30

40

50

60

0 50 100 150 200 250

Wea

ther

Sta

tions

reco

gniz

ed b

y En

viro

nmen

t Can

ada

Radial Distance (km)

Spatial Proximity of Weather Stations Recognized by Environment Canada (Prairies vs. Arctic)

Arctic sites AVG

Prairie sites AVG

Observational information

-18.0

-13.0

-8.0

-3.0

2.0

7.0

1940 1950 1960 1970 1980 1990 2000 2010

Mea

n A

nnua

l Air

Tem

pera

ture

(°C

)

Time (Year)

Change in Mean Annual Temperature (Prairies vs. Arctic)

Arctic sites AVG

Prairie sites AVG

Arctic 11 year running mean

Prairie 11 year running mean

Variability of the weather

-5.0

-4.0

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

5.0

1940 1950 1960 1970 1980 1990 2000 2010

Air

Tem

pera

ture

(°C

)

Time (Year)

Departure from 1951-1980 Mean Annual Air Temperature (Prairies vs. Arctic)

Arctic Departure from 1951-1980 mean

Prairie Departure from 1951-1980 mean

Arctic Departure from 1951-1980: 11 year running mean

Prairie Departure from 1951-1980: 11 year running mean

0.0

100.0

200.0

300.0

400.0

500.0

600.0

1940 1950 1960 1970 1980 1990 2000 2010

Tot

al A

nnua

l Pre

cipi

tatio

n (m

m)

Time (Year)

Change in Total Annual Precipitation (Prairies vs. Arctic)

Arctic sites AVG

Prairie sites AVG

Arctic 11 year running mean

Prairie 11 year running mean

-100.0

-50.0

0.0

50.0

100.0

150.0

200.0

1940 1950 1960 1970 1980 1990 2000 2010

Pre

cipi

tatio

n (m

m)

Time (Year)

Departure from 1951-1980 Mean Total Annual Precipitation (Prairies vs. Arctic)

Arctic Departure from 1951-1980 mean

Prairie Departure from 1951-1980 mean

Arctic Departure from 1951-1980: 11 year running mean

Prairie Departure from 1951-1980: 11 year running mean

Time Period Temperature Dew Point Temperature

RH Surface Pressure

01/01/1958-08/31/1961 13.97 °C 13.54 °C 10.88 % 1.05 mb 01/01/1978-08/31/1981 13.62 °C 13.89 °C 12.13 % 1.08 mb 01/01/1998-8/31/2001 14.01 °C 14.70 °C 13.23 % 1.06 mb

RMS values for hourly data from Iqaluit for select 7me periods

Is the weather becoming variable?

Edmonton Prairies

Arctic

Forecasts for period November 1998 to June 2004 Alternate Limits/ Visual Flight Rules (VFR) = for air flight planning

ALT_above = above alternate limits (dependent on station)

ALT_below = below alternate limits (dependent on station)

VFR_above = above VFR limits (a ceiling of 1000 ft. or visibility of 3 statute miles)

POD- Probability of Detection: What fraction of the observed "yes" events were correctly forecast? (Perfect score = 1)

POD = hits / (hits + misses)

FAR- False Alarm Ratio: What fraction of the predicted "yes" events actually did not occur (i.e., were false alarms)? (Perfect score = 0)

FAR = false alarms / (hits + false alarms)

CSI- Critical Success Index: How well did the forecast "yes" events correspond to the observed "yes" events? (Perfect score = 1)

CSI = hits / (hits + misses + false alarms)

HSS – Heidke Skill Score: What was the accuracy of the forecast relative to that of random chance? (Perfect score = 1) HSS = (C-E)/(T-E)

C = correct number of forecasts = hits + correct negatives T = total number of forecasts = hits + misses + false alarms + correct negatives E = expected number of correct forecasts by chance = ((hits + false alarms)(hits +misses)+(false alarms +

correct negatives)(misses + correct negatives)) / T

Edmonton Forecasts

Forecasts

Prairies

Arctic

ALT_above

VFR_above ALT_below

ALT_above ALT_below VFR_above

Nov/98 – Jun/04

Nov/98 – Jun/04

Environment Canada Data 1948 - 2006

POD – Probability of Detection FAR – False Alarm Ratio

Analysis and Comparison

CSI – Critical Success Index (Threat Score) HSS – Heidke Skill Score

Hits Misses Observed Yes

False Alarms

Correct Negatives

Observed No

Forecast Yes

Forecast No

Total

Forecasts

Obs

erve

d Yes No

Yes

No

Total

Tota

l

-40

-30

-20

-10

0

10

20

30

Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08

Prairie Mean Temperature Arctic Mean Temperature

0

20

40

60

80

100

120


Prairie Mean Total Precipitation Arctic Mean Total Precipitation

Mea

n M

onth

ly T

empe

ratu

re (°

C)

Mea

n M

onth

ly P

reci

pita

tion

(mm

) Mean Monthly Temperatures and Precipitation Levels During Skill Score Period of Nov/98 – Jun/04

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1


Prairie POD Arctic POD

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18


Prairie FAR Arctic FAR

Skill

Sco

re (%

) Sk

ill S

core

(%)

Skill Score Data for ALT_(Above Alternate) Limits

Skill

Sco

re (%

) Sk

ill S

core

(%)

0.7

0.75

0.8

0.85

0.9

0.95

1


Prairie CSI

Arctic CSI

Skill Score Data for ALT_(Above Alternate) Limits

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55


Prairie HSS Arctic HSS

For ALT_Above

Prairie stations were on average • 4.2% higher in Probability of Detection, • 6.0% lower in False Alarm Ratio, • 9% higher in Critical Success Index • 2.2% lower in Heidke Skill Score.

VFR_Above ALT_Above FAR POD CSI HSS FAR POD CSI HSS

Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377 Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399

Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic) Sk

ill S

core

(%)

0

0.02

0.04

0.06

0.08

0.1

0.12

Spring Summer Fall Winter

Prairie ALT_above_FAR

Arctic ALT_above_FAR

0.8

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98


Arctic ALT_above_CSI

Prarie ALT_above_CSI

0.3

0.32

0.34

0.36

0.38

0.4

0.42

0.44

0.46

0.48


Arctic ALT_above_HSS

Prarie ALT_above_HSS

Skill

Sco

re (%

)

Seasonal Skill Scores Between 1998 – 2004 (Prairies vs. Arctic)

0.3

0.4

0.5

0.6

0.7

0.8

0.9



0

0.1

0.2

0.3

0.4

0.5

0.6

0.7



Skill

Sco

re (%

) Sk

ill S

core

(%)

Skill Score Data for ALT_(Below Alternate) Limits

ALT_Below

FAR POD CSI HSS

Prairie AVG 0.539 0.459 0.261 0.377

Arctic AVG 0.463 0.443 0.323 0.399 For ALT_below, Prairie stations were on average • 1.5% higher in Probability of Detection • 7.6% higher in False Alarm Ratio • 6.2% lower in Critical Success Index

FAR is higher for below alternate conditions because it is much safer to forecast below and actually be above as opposed to forecast above and be below; POD and CSI are lower for below alternate conditions because like FAR it is much safer to forecast below and have a higher miss rate for conditions that are above). By deriving accuracy relative to random chance for HSS, ALT above and below will have the same skill score. This means once again Prairie stations had a 2.2% lower HSS.

0.3

0.32

0.34

0.36

0.38

0.4

0.42

0.44

0.46

0.48

0.5


Prairie ALT_below_POD

Arctic ALT_below_POD

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7


Prairie ALT_below_FAR

Arctic ALT_below_FAR


ill S

core

(%)

Forecasts for below alternative flight limits - Probability of Detection From 1998-2004, there appears to be an improvement for Edmonton forecasts (positive linear slope = 2E-05) and a slight decline in Iqaluit forecasts (negative linear slope = -2E-06).

Forecasts for below alternative flight limits - Probability of Detection On a seasonal basis, Edmonton forecasts are improving in the spring and summer where as Iqaluit forecasts are improving in the spring and fall. Edmonton is experiencing a decline in forecasting in the winter and fall where as Iqaluit is experiencing a decline in the summer and winter.

0

0.05

0.1

0.15

0.2

0.25



0.78

0.82

0.86

0.9

0.94

0.98



Skill

Sco

re (%

) Sk

ill S

core

(%)

Skill Score Data for VFR_(Above) Limits

VFR_Above ALT_Above FAR POD CSI HSS FAR POD CSI HSS

Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377 Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399

For VFR_Above

Prairie stations were on average • 5.9% higher in Probability of Detection, • 7.7% lower in False Alarm Ratio, • 11.3% higher in Critical Success Index • 5.6% higher in Heidke Skill Score.

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98


Prairie VFR_above_POD

Arctic VFR_above_POD

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18


Prairie VFR_above_FAR

Arctic VFR_above_FAR


ill S

core

(%)

0

10

20

30

40

50

60

0 50 100 150 200 250

Wea

ther

Sta

tions

reco

gniz

ed b

y En

viro

nmen

t Can

ada

Radial Distance (km)

Spatial Proximity of Weather Stations Recognized by Environment Canada (Prairies vs. Arctic)

Arctic sites AVG

Prairie sites AVG

VFR_Above ALT_Above

FAR POD CSI HSS FAR POD CSI HSS

Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377

Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399

ALT_Below

FAR POD CSI HSS

Prairie AVG 0.539 0.459 0.261 0.377

Arctic AVG 0.463 0.443 0.323 0.399

Number of Recognized Environment Canada Surface Weather Sta+ons Within 200 km

Skill Score (%

) Skill Score (%

)

Skill depending on number

of stations

ALT_Below

FAR POD CSI HSS

Prairie AVG 0.539 0.459 0.261 0.377

Arctic AVG 0.463 0.443 0.323 0.399

Comparison VFR_Above ALT_Above

FAR POD CSI HSS FAR POD CSI HSS

Prairie AVG 0.070 0.945 0.883 0.461 0.042 0.967 0.930 0.377

Arctic AVG 0.146 0.886 0.770 0.405 0.102 0.928 0.840 0.399

Factors affecting weather forecast skill in the Arctic

•  Observational information on the “state” of the weather

•  Variability of the weather •  Understanding of the weather •  And ability to “see ahead” – forecast •  Knowledge of the individual forecaster •  Location of the forecaster

Thank you for listening.

Weather Forecasting Skill in the Arctic

BLACK MOUNTAIN •  TOUR DATES

06/24/10 Calgary, AB - Republikw/ Lord Beginner

06/25/10 Edmonton, AB - Starlite Roomw/ Lord Beginner

06/26/10 Saskatoon, SK - Amigo'sw/ Lord Beginner

06/27/10 Regina, SK - The Exchangew/ Lord Beginner, Orbital Express

06/28/10 Winnipeg, MB - The Garrick Centrew/ Lord Beginner, Sleepy Sun

07/03/10 London, ON - Call the Office w/ One Hundred Dollars

Europe

•  07/23/10 Toronto, ONT - Horseshoe Tavernw/ Quest for Fire

factors(aﬀec+ng(weather( forecast(skill(in(the(arc+c( › faculties › environment › ceos ›...

Documents