robust income distribution analysis · 2007-04-13 · robust income distribution analysis robust...
TRANSCRIPT
![Page 1: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/1.jpg)
Robust income distribution analysis
Robust income distributionanalysis
Philippe Van Kerm
CEPS/INSTEADLuxembourg
5th German Stata Users Group Meeting (RWI Essen, April 2, 2007)
![Page 2: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/2.jpg)
Robust income distribution analysis
[ outline ]
1 The problem of data contamination/extreme incomes
2 Robust estimation strategies
3 Stata Implementation of OBRE
4 A brief empirical illustration
5 Concluding remarks
![Page 3: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/3.jpg)
Robust income distribution analysis
The problem of data contamination/extreme incomes
The problem of data contamination/extreme incomes
• Income distribution analysis:1 summary measures of inequality (and other distributional
features)2 dominance checks (stochastic dominance, Lorenz
dominance)• Both very sensitive to extreme incomes (‘valid’ outliers?
contamination?)• unbounded influence function (Cowell & Victoria-Feser,
Econometrica 1996, 2002)
![Page 4: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/4.jpg)
Robust income distribution analysis
The problem of data contamination/extreme incomes
The problem of data contamination/extreme incomes
• Income distribution analysis:1 summary measures of inequality (and other distributional
features)2 dominance checks (stochastic dominance, Lorenz
dominance)• Both very sensitive to extreme incomes (‘valid’ outliers?
contamination?)• unbounded influence function (Cowell & Victoria-Feser,
Econometrica 1996, 2002)
![Page 5: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/5.jpg)
Robust income distribution analysis
The problem of data contamination/extreme incomes
The problem of data contamination/extreme incomes
PSELL-3 (equivalised) household income data (waves 1-3):2002 2003 2004Top 10 incomes
37,260 16,925 41,83034,242 15,280 32,56928,292 15,132 18,341
... ... ...15,407 10,464 11,095
Summary measuresRaw Trim Wins. Raw Trim Wins. Raw Trim Wins.
µ 2,689 2,635 2,666 2,674 2,631 2,667 2,734 2,685 2,715Gini 0.272 0.259 0.266 0.262 0.252 0.259 0.262 0.250 0.257CV 2
2 0.192 0.129 0.147 0.138 0.116 0.129 0.159 0.112 0.123
![Page 6: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/6.jpg)
Robust income distribution analysis
The problem of data contamination/extreme incomes
The problem of data contamination/extreme incomes
PSELL-3 (equivalised) household income data (waves 1-3):2002 2003 2004Top 10 incomes
37,260 16,925 41,83034,242 15,280 32,56928,292 15,132 18,341
... ... ...15,407 10,464 11,095
Summary measuresRaw Trim Wins. Raw Trim Wins. Raw Trim Wins.
µ 2,689 2,635 2,666 2,674 2,631 2,667 2,734 2,685 2,715Gini 0.272 0.259 0.266 0.262 0.252 0.259 0.262 0.250 0.257CV 2
2 0.192 0.129 0.147 0.138 0.116 0.129 0.159 0.112 0.123
![Page 7: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/7.jpg)
Robust income distribution analysis
The problem of data contamination/extreme incomes
The problem of data contamination/extreme incomes
PSELL-3 (equivalised) household income data (waves 1-3):2002 2003 2004Top 10 incomes
37,260 16,925 41,83034,242 15,280 32,56928,292 15,132 18,341
... ... ...15,407 10,464 11,095
Summary measuresRaw Trim Wins. Raw Trim Wins. Raw Trim Wins.
µ 2,689 2,635 2,666 2,674 2,631 2,667 2,734 2,685 2,715Gini 0.272 0.259 0.266 0.262 0.252 0.259 0.262 0.250 0.257CV 2
2 0.192 0.129 0.147 0.138 0.116 0.129 0.159 0.112 0.123
![Page 8: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/8.jpg)
Robust income distribution analysis
Robust estimation strategies
[ outline ]
1 The problem of data contamination/extreme incomes
2 Robust estimation strategies
3 Stata Implementation of OBRE
4 A brief empirical illustration
5 Concluding remarks
![Page 9: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/9.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 10: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/10.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 11: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/11.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 12: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/12.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 13: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/13.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 14: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/14.jpg)
Robust income distribution analysis
Robust estimation strategies
Remedial actions
1 Remove extremely high incomes, or impose a top code• Easy, but not efficient and dependence to trimming fractions
2 Use functional form assumptions:• model tails of distribution parametrically (e.g. Pareto
distribution)1
• model the full distribution parametrically (e.g. log-Normal,Gamma, Singh-Maddala)
• But... classical ML estimators are themselves non-robust toextreme incomes!
1See, e.g., Cowell & Victoria-Feser, Journal of Economic Inequality 2007;Cowell & Flachaire, Journal of Econometrics 2007
![Page 15: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/15.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE)A robust alternative to classical ML
• OBRE is an M-estimator: θ solution to∑N
i=1 ψ(xi , θ) = 0• (For ML: ψ(xi , θ
ML) = s(xi , θML) is the score function)
• OBRE estimator is the solution to
ψ(xi , θOB) = (s(xi , θ
OB)− a(θOB))Wc(xi ; θOB)
where
Wc(xi ; θOB) = min
(1;
cG(s(xi , θOB),a(θOB),A(θOB))
)• Wc(x ; θOB) imposes a bound on influence function by
downweighting extreme values (values deviating frommodel)
• c is a ‘robustness’ parameter to be determined(efficiency-robustness trade-off)
![Page 16: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/16.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE)A robust alternative to classical ML
• OBRE is an M-estimator: θ solution to∑N
i=1 ψ(xi , θ) = 0• (For ML: ψ(xi , θ
ML) = s(xi , θML) is the score function)
• OBRE estimator is the solution to
ψ(xi , θOB) = (s(xi , θ
OB)− a(θOB))Wc(xi ; θOB)
where
Wc(xi ; θOB) = min
(1;
cG(s(xi , θOB),a(θOB),A(θOB))
)• Wc(x ; θOB) imposes a bound on influence function by
downweighting extreme values (values deviating frommodel)
• c is a ‘robustness’ parameter to be determined(efficiency-robustness trade-off)
![Page 17: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/17.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE)A robust alternative to classical ML
• OBRE is an M-estimator: θ solution to∑N
i=1 ψ(xi , θ) = 0• (For ML: ψ(xi , θ
ML) = s(xi , θML) is the score function)
• OBRE estimator is the solution to
ψ(xi , θOB) = (s(xi , θ
OB)− a(θOB))Wc(xi ; θOB)
where
Wc(xi ; θOB) = min
(1;
cG(s(xi , θOB),a(θOB),A(θOB))
)• Wc(x ; θOB) imposes a bound on influence function by
downweighting extreme values (values deviating frommodel)
• c is a ‘robustness’ parameter to be determined(efficiency-robustness trade-off)
![Page 18: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/18.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE)A robust alternative to classical ML
• OBRE is an M-estimator: θ solution to∑N
i=1 ψ(xi , θ) = 0• (For ML: ψ(xi , θ
ML) = s(xi , θML) is the score function)
• OBRE estimator is the solution to
ψ(xi , θOB) = (s(xi , θ
OB)− a(θOB))Wc(xi ; θOB)
where
Wc(xi ; θOB) = min
(1;
cG(s(xi , θOB),a(θOB),A(θOB))
)• Wc(x ; θOB) imposes a bound on influence function by
downweighting extreme values (values deviating frommodel)
• c is a ‘robustness’ parameter to be determined(efficiency-robustness trade-off)
![Page 19: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/19.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE)A robust alternative to classical ML
• OBRE is an M-estimator: θ solution to∑N
i=1 ψ(xi , θ) = 0• (For ML: ψ(xi , θ
ML) = s(xi , θML) is the score function)
• OBRE estimator is the solution to
ψ(xi , θOB) = (s(xi , θ
OB)− a(θOB))Wc(xi ; θOB)
where
Wc(xi ; θOB) = min
(1;
cG(s(xi , θOB),a(θOB),A(θOB))
)• Wc(x ; θOB) imposes a bound on influence function by
downweighting extreme values (values deviating frommodel)
• c is a ‘robustness’ parameter to be determined(efficiency-robustness trade-off)
![Page 20: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/20.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE) (ctd.)A robust alternative to classical ML
• a(θOB) and A(θOB) are such that
E(ψ(x , θOB)ψ(x , θOB)′) = (A(θOB)A(θOB)′)−1
E(ψ(x , θOB)) = 0
The resulting estimator is the optimal (minimum variance)M-estimator with bounded influence function2
• If c →∞ then θOB = θML
2For a thorough treatment, see Hampel et al. (1986), Robust Statistics:The approach based on influence functions.
![Page 21: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/21.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE) (ctd.)A robust alternative to classical ML
• a(θOB) and A(θOB) are such that
E(ψ(x , θOB)ψ(x , θOB)′) = (A(θOB)A(θOB)′)−1
E(ψ(x , θOB)) = 0
The resulting estimator is the optimal (minimum variance)M-estimator with bounded influence function2
• If c →∞ then θOB = θML
2For a thorough treatment, see Hampel et al. (1986), Robust Statistics:The approach based on influence functions.
![Page 22: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/22.jpg)
Robust income distribution analysis
Robust estimation strategies
Optimal B-Robust Estimators (OBRE) (ctd.)A robust alternative to classical ML
• a(θOB) and A(θOB) are such that
E(ψ(x , θOB)ψ(x , θOB)′) = (A(θOB)A(θOB)′)−1
E(ψ(x , θOB)) = 0
The resulting estimator is the optimal (minimum variance)M-estimator with bounded influence function2
• If c →∞ then θOB = θML
2For a thorough treatment, see Hampel et al. (1986), Robust Statistics:The approach based on influence functions.
![Page 23: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/23.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
[ outline ]
1 The problem of data contamination/extreme incomes
2 Robust estimation strategies
3 Stata Implementation of OBRE
4 A brief empirical illustration
5 Concluding remarks
![Page 24: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/24.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation
• Given number of implicit definitions of parameters andconstraints, estimation is not easy
• But relatively detailed algorithms are available(fortunately!). I follow Ronchetti & Victoria-Feser(Canadian Journal of Statistics, 1994).
• Estimation involves1 matrix operations2 numerical integration
=⇒ Mata!
![Page 25: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/25.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation
• Given number of implicit definitions of parameters andconstraints, estimation is not easy
• But relatively detailed algorithms are available(fortunately!). I follow Ronchetti & Victoria-Feser(Canadian Journal of Statistics, 1994).
• Estimation involves1 matrix operations2 numerical integration
=⇒ Mata!
![Page 26: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/26.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation
• Given number of implicit definitions of parameters andconstraints, estimation is not easy
• But relatively detailed algorithms are available(fortunately!). I follow Ronchetti & Victoria-Feser(Canadian Journal of Statistics, 1994).
• Estimation involves1 matrix operations2 numerical integration
=⇒ Mata!
![Page 27: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/27.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation
• Given number of implicit definitions of parameters andconstraints, estimation is not easy
• But relatively detailed algorithms are available(fortunately!). I follow Ronchetti & Victoria-Feser(Canadian Journal of Statistics, 1994).
• Estimation involves1 matrix operations2 numerical integration
=⇒ Mata!
![Page 28: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/28.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation (ctd.)
• Implementation is relatively easy with Mata (but familiaritywith matrix algebra can help!)
• Builds on suite of commands by Stephen Jenkins to fitfunctional forms to unit record data by ML3
• just replace ML engine by home-brewed OBRE engine (calla Mata function, rather than ml model)
• I implemented Pareto Type I distribution and 3-parametersSingh-Maddala distribution4
• Compatible with Nick Cox’s diagnostic commands psm andqsm
3ssc describe smfit4paretofit to fit Pareto by ML will soon be added to Jenkins’ collection.
![Page 29: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/29.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation (ctd.)
• Implementation is relatively easy with Mata (but familiaritywith matrix algebra can help!)
• Builds on suite of commands by Stephen Jenkins to fitfunctional forms to unit record data by ML3
• just replace ML engine by home-brewed OBRE engine (calla Mata function, rather than ml model)
• I implemented Pareto Type I distribution and 3-parametersSingh-Maddala distribution4
• Compatible with Nick Cox’s diagnostic commands psm andqsm
3ssc describe smfit4paretofit to fit Pareto by ML will soon be added to Jenkins’ collection.
![Page 30: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/30.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation (ctd.)
• Implementation is relatively easy with Mata (but familiaritywith matrix algebra can help!)
• Builds on suite of commands by Stephen Jenkins to fitfunctional forms to unit record data by ML3
• just replace ML engine by home-brewed OBRE engine (calla Mata function, rather than ml model)
• I implemented Pareto Type I distribution and 3-parametersSingh-Maddala distribution4
• Compatible with Nick Cox’s diagnostic commands psm andqsm
3ssc describe smfit4paretofit to fit Pareto by ML will soon be added to Jenkins’ collection.
![Page 31: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/31.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation (ctd.)
• Implementation is relatively easy with Mata (but familiaritywith matrix algebra can help!)
• Builds on suite of commands by Stephen Jenkins to fitfunctional forms to unit record data by ML3
• just replace ML engine by home-brewed OBRE engine (calla Mata function, rather than ml model)
• I implemented Pareto Type I distribution and 3-parametersSingh-Maddala distribution4
• Compatible with Nick Cox’s diagnostic commands psm andqsm
3ssc describe smfit4paretofit to fit Pareto by ML will soon be added to Jenkins’ collection.
![Page 32: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/32.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Implementation (ctd.)
• Implementation is relatively easy with Mata (but familiaritywith matrix algebra can help!)
• Builds on suite of commands by Stephen Jenkins to fitfunctional forms to unit record data by ML3
• just replace ML engine by home-brewed OBRE engine (calla Mata function, rather than ml model)
• I implemented Pareto Type I distribution and 3-parametersSingh-Maddala distribution4
• Compatible with Nick Cox’s diagnostic commands psm andqsm
3ssc describe smfit4paretofit to fit Pareto by ML will soon be added to Jenkins’ collection.
![Page 33: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/33.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Practical programming issues
• Precision of numerical integration functions revealed veryimportant
• Difficulty to set multiple tolerance and precisionparameters – trade-off between speed and accuracy (stillsubject to changes...)
• As in ML estimation, using re-parameterization θ̃ = ln(θ)can help convergence (in all models considered, θ > 0)
![Page 34: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/34.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Practical programming issues
• Precision of numerical integration functions revealed veryimportant
• Difficulty to set multiple tolerance and precisionparameters – trade-off between speed and accuracy (stillsubject to changes...)
• As in ML estimation, using re-parameterization θ̃ = ln(θ)can help convergence (in all models considered, θ > 0)
![Page 35: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/35.jpg)
Robust income distribution analysis
Stata Implementation of OBRE
Practical programming issues
• Precision of numerical integration functions revealed veryimportant
• Difficulty to set multiple tolerance and precisionparameters – trade-off between speed and accuracy (stillsubject to changes...)
• As in ML estimation, using re-parameterization θ̃ = ln(θ)can help convergence (in all models considered, θ > 0)
![Page 36: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/36.jpg)
Robust income distribution analysis
A brief empirical illustration
[ outline ]
1 The problem of data contamination/extreme incomes
2 Robust estimation strategies
3 Stata Implementation of OBRE
4 A brief empirical illustration
5 Concluding remarks
![Page 37: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/37.jpg)
Robust income distribution analysis
A brief empirical illustration
Empirical illustration
• Data from panel survey PSELL-3 (Panel ‘Living inLuxembourg’), 2003–2005
• Representative of Luxembourg residents• Single-adult-equivalent real household income (incomes of
2002-2004)
![Page 38: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/38.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesPareto Type I parameters
ML OBREc = 200 c = 5 c = 3 c = 2
Pareto Type I 2002 3.635 3.635 3.633 3.720 3.926(upper 5%) 2003 4.075 4.075 4.060 4.007 3.911
2004 4.306 4.306 4.383 4.425 4.498
![Page 39: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/39.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesPareto Type I parameters
ML OBREc = 200 c = 5 c = 3 c = 2
Pareto Type I 2002 3.635 3.635 3.633 3.720 3.926(upper 5%) 2003 4.075 4.075 4.060 4.007 3.911
2004 4.306 4.306 4.383 4.425 4.498
![Page 40: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/40.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesEmpirical CDF and estimated Pareto Type I CDF
.994
.996
.998
1
10000 20000 30000 4000010000 20000 30000 4000010000 20000 30000 40000
1 2 3
Niveau de vie exprimé en prix constants de 2005Graphs by wave
![Page 41: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/41.jpg)
Robust income distribution analysis
A brief empirical illustration
OBRE robustness weightsPareto Type I distribution
0.5
1
0 10000 20000 30000 0 10000 20000 30000 0 10000 20000 30000
1 2 3
Niveau de vie exprimé en prix constants de 2005Graphs by wave
![Page 42: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/42.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesSingh-Maddala parameters
ML OBREc = 200 c = 10 c = 5 c = 4
Singh-Maddala 2002 4.131 4.141 4.170 4.417 4.7262,159 2,159 2,146 2,022 1,9120.797 0.797 0.784 0.664 0.555
2003 3.643 3.463 3.713 4.035 4.3262,477 2,477 2,428 2,214 2,0601.094 1.094 1.040 0.822 0.666
2004 3.666 3.666 3.716 3.980 4.2622,529 2,529 2,496 2,278 2,1241.091 1.091 1.058 0.841 0.684
![Page 43: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/43.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesSingh-Maddala parameters
ML OBREc = 200 c = 10 c = 5 c = 4
Singh-Maddala 2002 4.131 4.141 4.170 4.417 4.7262,159 2,159 2,146 2,022 1,9120.797 0.797 0.784 0.664 0.555
2003 3.643 3.463 3.713 4.035 4.3262,477 2,477 2,428 2,214 2,0601.094 1.094 1.040 0.822 0.666
2004 3.666 3.666 3.716 3.980 4.2622,529 2,529 2,496 2,278 2,1241.091 1.091 1.058 0.841 0.684
![Page 44: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/44.jpg)
Robust income distribution analysis
A brief empirical illustration
ML vs. OBRE parameter estimatesNon-parametric estimates and estimated Singh-Maddala PDFs
0.0
005
0 2000 4000 6000 0 2000 4000 6000 0 2000 4000 6000
1 2 3
Niveau de vie exprimé en prix constants de 2005Graphs by wave
![Page 45: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/45.jpg)
Robust income distribution analysis
A brief empirical illustration
OBRE robustness weightsSingh-Maddala distribution
0.5
1
0 10000 20000 30000 0 10000 20000 30000 0 10000 20000 30000
1 2 3
Niveau de vie exprimé en prix constants de 2005Graphs by wave
![Page 46: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/46.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks
• Mata makes estimators such as OBRE feasible withinStata
• In theory, OBRE estimators have great relevance inincome distribution analysis... implementation in Stata mayhelp putting this claim to broader practical assessment
• At present, it is a prototype (but looks ok). Minordevelopments still needed for
• fixing precision and tolerance thresholds• allowing svy: prefix (?)• adding additional distributions (log-normal, gamma,
Dagum) (?) – transplanting code to other distributions iseasy
![Page 47: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/47.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks
• Mata makes estimators such as OBRE feasible withinStata
• In theory, OBRE estimators have great relevance inincome distribution analysis... implementation in Stata mayhelp putting this claim to broader practical assessment
• At present, it is a prototype (but looks ok). Minordevelopments still needed for
• fixing precision and tolerance thresholds• allowing svy: prefix (?)• adding additional distributions (log-normal, gamma,
Dagum) (?) – transplanting code to other distributions iseasy
![Page 48: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/48.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks
• Mata makes estimators such as OBRE feasible withinStata
• In theory, OBRE estimators have great relevance inincome distribution analysis... implementation in Stata mayhelp putting this claim to broader practical assessment
• At present, it is a prototype (but looks ok). Minordevelopments still needed for
• fixing precision and tolerance thresholds• allowing svy: prefix (?)• adding additional distributions (log-normal, gamma,
Dagum) (?) – transplanting code to other distributions iseasy
![Page 49: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/49.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks
• Mata makes estimators such as OBRE feasible withinStata
• In theory, OBRE estimators have great relevance inincome distribution analysis... implementation in Stata mayhelp putting this claim to broader practical assessment
• At present, it is a prototype (but looks ok). Minordevelopments still needed for
• fixing precision and tolerance thresholds• allowing svy: prefix (?)• adding additional distributions (log-normal, gamma,
Dagum) (?) – transplanting code to other distributions iseasy
![Page 50: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/50.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks
• Mata makes estimators such as OBRE feasible withinStata
• In theory, OBRE estimators have great relevance inincome distribution analysis... implementation in Stata mayhelp putting this claim to broader practical assessment
• At present, it is a prototype (but looks ok). Minordevelopments still needed for
• fixing precision and tolerance thresholds• allowing svy: prefix (?)• adding additional distributions (log-normal, gamma,
Dagum) (?) – transplanting code to other distributions iseasy
![Page 51: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/51.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks (ctd.)
• Now entering more thorough testing phase:• Monte-Carlo simulations• Benchmarking against the software IneQ (by Cowell and
Gomulka)• Further applications
• Then need to develop add-on software to help exploitthese tools for deriving complete, robust inequality/povertyestimates
![Page 52: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/52.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks (ctd.)
• Now entering more thorough testing phase:• Monte-Carlo simulations• Benchmarking against the software IneQ (by Cowell and
Gomulka)• Further applications
• Then need to develop add-on software to help exploitthese tools for deriving complete, robust inequality/povertyestimates
![Page 53: Robust income distribution analysis · 2007-04-13 · Robust income distribution analysis Robust estimation strategies Remedial actions 1 Remove extremely high incomes, or impose](https://reader034.vdocument.in/reader034/viewer/2022042203/5ea4114e09c29a6126346aa7/html5/thumbnails/53.jpg)
Robust income distribution analysis
Concluding remarks
Concluding remarks (ctd.)
• Now entering more thorough testing phase:• Monte-Carlo simulations• Benchmarking against the software IneQ (by Cowell and
Gomulka)• Further applications
• Then need to develop add-on software to help exploitthese tools for deriving complete, robust inequality/povertyestimates