Download - Smoothing mortality rates using R
![Page 1: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/1.jpg)
Smoothing mortality rates using R
Gary Brown & Julie Mills
![Page 2: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/2.jpg)
Overview
• Introduction• Context• Methodology• Implementation• Results• Summary• What’s next?
![Page 3: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/3.jpg)
Introduction
• Future mortality rates are published every two years – Until 2004, by the Government Actuary’s
Department (GAD)– Since 2006, by ONS (with GAD consultants)
• The methodology was designed by GAD in the 1990s and runs in Excel
• In 2010, ONS reviewed the current process– Implementation and testing still ongoing
![Page 4: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/4.jpg)
Context
• Mortality rates, estimated 75 years into the future, are a key factor in National Population Projections (others: births and net migration)– Natural change (births – deaths) accounts for 1/3
of total population change
• Population projections used as inputs/control totals for other government projections, such as numbers of school children or pensioners
• Robustness of mortality rates is crucial
![Page 5: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/5.jpg)
Methodology - current
Mortality rate = deaths/pop
![Page 6: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/6.jpg)
Methodology - current
Mortality rate = deaths/popconstrained survivor ratio
![Page 7: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/7.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
constrained survivor ratio
![Page 8: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/8.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
constrained survivor ratio
extrapolate to age 120
![Page 9: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/9.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
constrained survivor ratio
extrapolate to age 120
![Page 10: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/10.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
![Page 11: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/11.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
![Page 12: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/12.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
![Page 13: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/13.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
![Page 14: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/14.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
![Page 15: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/15.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
![Page 16: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/16.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
… further adjustments
![Page 17: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/17.jpg)
Methodology - current
Mortality rate = deaths/pop
Smooth within years (to 103/104)
Estimate year T+1 for each age
Smooth improvement rate in T+1
Improvement rates up to T+26
Mortality rates for T+1 to T+26
constrained survivor ratio
extrapolate to age 120
exponential smoothingx2
1x1 3x1 5x1 3x1 1x1 MAs
T+26 expert opinions
… further adjustments
![Page 18: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/18.jpg)
Methodology - new
• Replace two-stage smoothing process
• Smooth mortality rates surface simultaneously over ages and years
• Estimate improvement rate using existing smoothed years – ie do not estimate T+1
• Requires longer path to T+26 opinions!
![Page 19: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/19.jpg)
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
![Page 20: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/20.jpg)
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
![Page 21: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/21.jpg)
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
![Page 22: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/22.jpg)
Methodology – 2 dimensional p-spline
• Thoroughly tested, and recommended, by Continuous Mortality Investigation
• Best advice - read “Smoothing and forecasting mortality rates”, Currie et al, 2004!
T Ta r r r y c c c
a y ages years
t t+5
PL( )=L( )-0.5 P( )
where P( )= I D D + I D D
where D , D act on rows/columns of B B
where B is matrix of B-splines, each fitting (x , x )
![Page 23: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/23.jpg)
Implementation
• Difficult to understand (and explain) … but easy to implement!
![Page 24: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/24.jpg)
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
![Page 25: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/25.jpg)
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
Mort2Dsmooth(x=ages,y=years,Z=deaths,offset=log(pop))
![Page 26: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/26.jpg)
Implementation
• Difficult to understand (and explain) … but easy to implement!
• MortalitySmooth (Carlo G Camarda) in R
• Smoothed values = 21st entry in list of R output
Mort2Dsmooth(x=ages,y=years,Z=deaths,offset=log(pop))
![Page 27: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/27.jpg)
Results - testing
Males Females
Sensitivity – adding annual data
Ages 0 - 1031961 – 2003
1961 – 2004
1961 – 2005
1961 – 2006
1961 – 2007
1961 – 2008
1961 – 2009
Ages 0 - 1041961 – 2003
1961 – 2004
1961 – 2005
1961 – 2006
1961 – 2007
1961 – 2008
1961 – 2009
Sensitivity
- adding years of age
1961 – 20090 – 90
0 – 100
0 – 103
1961 – 2005 1961 - 2009 0 – 104 0 – 90
0 – 105 0 – 100
0 – 106 0 – 104
0 – 105
0 – 106
![Page 28: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/28.jpg)
Results - testing
0
1
2
3
4
5
6
102
%
Age
0 20 40 60 8010 30 50 70 90
61-04
61-05
61-06
61-09
Mortality improvement rates by age, 2003/04
61-07
61-08
![Page 29: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/29.jpg)
Results – mortality rates in the base year
Issues• New method does not project rates forward to
base year• Edge effects
Solution• Step back 2 years into the data set
2010 mortality rates = 2007 mortality rates x (1 – 2006-07 improvement rates/100) ^
3
![Page 30: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/30.jpg)
Results – mortality rates in the base year
Year
Age
100
52 yrs
49
48 48 yrs
0
1961………………………………………………….. 2007 2008 2009 2010
![Page 31: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/31.jpg)
Past improvements in smoothed mortality rates, males – old method
![Page 32: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/32.jpg)
Past improvements in smoothed mortality rates, males – new method
![Page 33: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/33.jpg)
Past improvements in smoothed mortality rates, Scotland males – new method
![Page 34: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/34.jpg)
Past improvements in smoothed mortality rates, females – old method
![Page 35: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/35.jpg)
Past improvements in smoothed mortality rates, females – new method
![Page 36: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/36.jpg)
Comparison of projected smooth % changes in death rates by age, UK 2009-10
Males
-1
0
1
2
3
4
5
6
0 10 20 30 40 50 60 70 80 90 100 110
Age
Per
cent
age
redu
ctio
n
2008-basedprojections
Proposed 2010-based projections
![Page 37: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/37.jpg)
Comparison of projected smooth % changes in death rates by age, UK 2009-10
Females
0
1
2
3
4
5
6
0 10 20 30 40 50 60 70 80 90 100 110
Age
2008-basedprojections
Proposed 2010-based projections
![Page 38: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/38.jpg)
Comparison of actual and projected expectation of life at birth
65
70
75
80
85
90
1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050 2055 2060 2065 2070 2075 2080 2085
Old 08
New 08
Old 08
New 08
Females
Males
![Page 39: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/39.jpg)
Summary
• New smoothing method used to produce the 2010-based ‘proposed’ mortality assumptions
• Introduced in the 2010-based consultation with devolved administrations and government departments
![Page 40: Smoothing mortality rates using R](https://reader035.vdocument.in/reader035/viewer/2022062308/56813437550346895d9b29d7/html5/thumbnails/40.jpg)
What’s next?
• More testing/evaluation:Over-smoothingAdding 2010 dataDerivation of base year rates
• Using R to project mortality rates