beginners guide to weather and climate data
TRANSCRIPT
A Beginners guide to weather and climate data
Bristol Data Scientists Meetup | 24 January 2017
Margriet Groenendijk | Developer Advocate | IBM Watson Data Platform
@MargrietGr
https://medium.com/ibm-watson-data-lab
Observations + ModelsForecast =
Temperature
Humidity
Windspeed and direction
Air pressure
Rainfall
Radiation
http://www.metoffice.gov.uk/public/weather/climate-network/#?tab=climateNetwork
Historic weather
Mean daily maximum temperature
Mean daily minimum temperature
Days of air frost
Total rainfall
Total sunshine duration
http://www.metoffice.gov.uk/datapoint/
https://business.weather.com/products/the-weather-company-data-packages
Note: data not cleaned
Historic weather
Example: London City airport 1997-2017
https://climexp.knmi.nl
http://www.cru.uea.ac.uk/data/
https://developer.ibm.com/clouddataservices/2016/04/18/predict-temperatures-using-dashdb-python-and-r/
pseudo-code… test before using!
Models
3D grid
Atmosphere
Energy
Water
Differential equations
Boundary conditions
Land
Ocean
Lots of parameters
http://www.climateprediction.net/
Important to separate the different PFTs and large cover fraction, because these are all influenced20
the f0 parameters and by the evaporation from the soil.21
Where can the differences between models be related to?22
What are the uncertainties?23
Any constraints of the models with observations?24
2 Theory25
At the leaf-scale CO2 (A) and water vapour (E) fluxes across stomata are given by (Katul et al., 2009):26
A = gs(ca � ci) (1)27
E = 1.6gs(ei � ea) (2)
where gs is the stomatal conductance of CO2, ca and ci the ambient and intercellular CO2 concentra-28
tions, ea and ei the ambient and intercellular water vapour concentration, and 1.6 the relative diffu-29
sivity of water with respect to carbon. The ratio of these fluxes is defined as the leaf-scale water use30
efficient (WUEleaf ):31
WUEleaf =A
E(3)
Additionally, WUEleaf can be derived from the gradients of the CO2 concentration and water vapour32
between the ambient air and air within the leaf. When combining Eqs. 1 to 3 it follows that:33
WUEleaf =ca � ci
1.6(es � ea)(4)
where ca and ci are the ambient and internal partial pressures of CO2 [Pa] and es and ea the sat-34
urated and ambient water vapour pressure [Pa] The assumption is used that the intercellular water35
vapour concentration is saturated and therefore ei in Eq. 1 is replaced with es. [See Katul papers for36
conversions between units and why vpd is a good estimate in this equation]37
To simulate the ecosystem-scale water and carbon fluxes the model developed for the leaf-scale is38
used (e.g., Farquhar et al., 1980). This implies that in theory WUE at the ecosystem-scale can be39
estimated with both eqs. 3 and 4 and that they should be equal:40
WUEe =GPP
ET(5)
2GPP ET
http://www.earth-syst-dynam.net/7/525/2016/esd-7-525-2016.pdf
0 50 100 1500
100
200
300
400
500
600
vcm,25 (µmol m−2 s−1)
j m,2
5 (µm
ol m−2
s−1 )
1 2 3 4 5 6 7
0 50 100 1500
0.2
0.4
0.6
0.8
1
vcm,25 (µmol m−2 s−1)
α (m
ol m
ol−1
)
0 50 100 1500
200
400
600
800
1000
vcm,25 (µmol m−2 s−1)
λ (m
ol m
ol−1
)
0 200 400 6000
0.2
0.4
0.6
0.8
1
jm,25 (µmol m−2 s−1)α
(mol
mol−1
)
0 200 400 6000
200
400
600
800
1000
jm,25 (µmol m−2 s−1)
λ (m
ol m
ol−1
)
0 0.5 10
200
400
600
800
1000
α (mol mol−1)
λ (m
ol m
ol−1
)
k-means clustering
0 50 100 1500
100
200
300
400
500
600
vcm,25 (µmol m−2 s−1)
j m,2
5 (µm
ol m−2
s−1 )
1 2 3 4 5 6 7
0 50 100 1500
0.2
0.4
0.6
0.8
1
vcm,25 (µmol m−2 s−1)α
(mol
mol−1
)
0 50 100 1500
200
400
600
800
1000
vcm,25 (µmol m−2 s−1)
λ (m
ol m
ol−1
)
0 200 400 6000
0.2
0.4
0.6
0.8
1
jm,25 (µmol m−2 s−1)
α (m
ol m
ol−1
)
0 200 400 6000
200
400
600
800
1000
jm,25 (µmol m−2 s−1)
λ (m
ol m
ol−1
)0 0.5 1
0
200
400
600
800
1000
α (mol mol−1)λ
(mol
mol−1
)
1 2 3 4 5 6 70
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Group
Rela
tive
vege
tatio
n di
strib
utio
n (−
)
CROSAVDBFEBFENFGRAMFO
1 2 3 4 5 6 70
0.05
0.1
0.15
0.2
0.25
0.3
0.35
GroupRe
lativ
e cl
imat
e di
strib
utio
n (−
)
ARBOTMTETCTR
k-means clusters vs. vegetation and climate classifications
0.25
0.15
0.20
A CB
http://www.earth-syst-dynam.net/7/525/2016/esd-7-525-2016.pdf
Climate models
=
Algorithms + Observations + Parameters
=
Machine Learning
invehiclehaildamageclaimseveryyear
increaseintemperaturemeans$24Mmorein
electricityspendingperday
drop insalesforareaswithmorethana10%drop
intemperature
Insurance
Energy & Utility
Retail
Some applications
Next… find the data!
Open Data
Open data in Bristol
https://opendata.bristol.gov.uk
Air quality River water quality
Water levels Flood alerts
M32 traffic flow Road works
Crime stats Census data
Energy consumption by ward Hospital admissions
NHS
Weather + Driving Difficulty Index API
+
M32 traffic flow + Road works
+
air quality observations
=
Predict air quality
Weather and Climate Data
Historical time series
Historical maps
Climate model data
Real-time APIs
Forecast APIs
Thanks!https://github.com/MargrietGroenendijk/Bristol
http://www.slideshare.net/MargrietGroenendijk/presentations
@MargrietGr
https://medium.com/ibm-watson-data-lab