Mathematical Statistics for Economics and Business

Springer New York Berlin Heidelberg Barcelona Hong Kong London Milan Paris Singapore Tokyo

Ron C. Mittelhammer

Mathematical Statistics for Economics and Business

With 83 Illustrations

Springer

Ron C. Mittelhammer Program in Statistics and Department of Agricultural Economics Washington State University Pullman, WA 99164-6210 USA

library of Congress Cataloging-in-Publication Data Mittelhammer, Ron.

Mathematical statistics for economics and business / Ron Mittelhammer.

p. cm. Includes bibliographical references and inde~.

ISBN-13: 978-1-4612-8461-1 e-ISBN-13: 978-1-4612-3988-8 DOl: 10.1007/978-1-4612-3988-8 1. Commercial statistics. 2. Mathematical statistics. I. Title.

HF1017.M53 1995 519.5-dc20 95-37686

Printed on acid-free paper.

© 1996 Springer-Verlag New York Inc.

Softcover reprint of the hardcover 1 st edition 1996

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

Production managed by Laura Carlson; manufacturing supervised by jacqui Ashri. Typeset by The Bartlett Press, Inc.

9 8 7 6 5 4 3 (Corrected third printing, 1999)

To my wife Linda, and to Carl, Edith, Dolly and Ralph

Preface

This book is designed to provide beginning graduate stu­dents and advanced undergraduates with a rigorous and accessible foundation in the principles of probability and mathematical statistics underlying statis­tical inference in the fields of business and economics. The book assumes no prior knowledge of probability or statistics and effectively builds the subject "from the ground up." Students who complete their studies of the topics in this text will have acquired the necessary background to achieve a mature and enduring understanding of statistical and econometric methods of inference and will be well equipped to read and comprehend graduate-level economet­rics texts. Additionally, this text serves as an effective bridge to more advanced study of both mathematical statistics and econometric theory and methods. The book will also be of interest to researchers who desire a decidedly business and economics-oriented treatment of the subject in terms of its topics, depth, breadth, examples, and problems.

Without the unifying foundations that come with training in probability and mathematical statistics, students in statistics and econometrics classes too often perceive the subject matter as a potpourri of formulae and techniques applied to a collection of special cases. The details of the cases and their solu­tions quickly fade for those who do not understand the reasons for using the procedures they attempt to apply. Many institutions now recognize the need for a more rigorous study of probability and the mathematical statistics principles

viii Preface

in order to prepare students for a higher level and a longer lasting understanding of the statistical techniques employed in the fields of business and economics. Furthermore, quantitative analysis in these fields has progressed to the point where a deeper understanding of the principles of probability and statistics is now virtually necessary for one to read and contribute successfully to quanti­tative research in economics and business. Contemporary students themselves know this and need little convincing from advisors that substantial statistical training must be acquired in order to compete successfully with their peers and to become effective researchers. Despite these observations, there are very few rigorous books on probability and mathematical statistics foundations that are also written with the needs of business and economics student in mind.

This book is the culmination of 15 years of teaching graduate level statis­tics and econometrics classes for students who are beginning graduate programs in business Iprimarily finance, marketing, accounting, and decision sciences), economics, and agricultural economics. When I originally took on the teach­ing assignment in this area, I cycled through a number of very good texts in mathematical statistics searching for an appropriate exposition for beginning graduate students. With the help of my students, I ultimately realized that the available textbook presentations were optimizing the wrong objective func­tions for our purposes! Some books were too elementary, other presentations did not cover multivariate topics in sufficient detail, and proofs of important results were omitted occasionally because they were "obvious" or "clear" or "beyond the scope of the text." In most cases they were neither obvious nor clear to students, and in many cases, useful and accessible proofs of the most important results can and should be provided at this level of instruction. Suf­ficient asymptotic theory was often lacking and/or tersely developed. At the extreme, material was presented in a sterile mathematical context at a level that was inaccessible to most beginning graduate students while nonetheless leaving notable gaps in topic coverage of particular interest to business and eco­nomics students. I then began to teach the course from lecture notes that I had created and iteratively refined them as I interacted with scores of students who provided me with feedback regarding what was working-and what wasn't­with regard to topics, proofs, problems, and exposition. I am deeply indebted to the hundreds of students who persevered through, and contributed to, the many revisions and continual sophistication of my notes. Their influence has had a substantial impact on the text: It is a time-tested and class-tested prod­uct. Other students at a similar stage of development should find it honest, accessible, and informative.

Instructors attempting to teach a rigorous course in mathematical statistics soon learn that the typical new graduate student in economics and business is thoroughly intelligent, but often lacks the sophisticated mathematical training that facilitates understanding and assimilation of the mathematical concepts involved in mathematical statistics. My experience has been that these stu­dents can understand and become functional with sophisticated concepts in mathematical statistics if their backgrounds are respected and the material is presented carefully and thoroughly, using a realistic level of mathematics.

Preface ix

Furthermore, it has been my experience that most students are actually eager to see proofs of propositions, as opposed to merely accepting statements on faith, so long as the proofs do not insult the integrity of the nonmathemati­ciano Additionally, students almost always remark that the understanding and the long-term memory of a stated result is enhanced by first having worked through a formal proof of a proposition, and then working through examples and problems that require the result to be applied.

With the preceding observations in mind, the prerequisites for the book include only the usual introductory college-level courses in basic calculus (in­cluding univariate integration and differentiation, partial differentiation, and multivariate integration of the iterated integral type) and basic matrix algebra. The text is largely self-contained for students with this preparation. A signifi­cant effort has been made to present proofs in ways that are accessible. Care has been taken to choose methods and types of proofs that exercise and extend the learning process regarding statistical results and concepts learned prior to the introduction of the proof. A generous number of examples are presented with a substantial amount of detail to illustrate the application of major theories, concepts, and methods. The problems at the end of the chapters are chosen to provide an additional perspective to the learning process. The majority of the problems are word problems designed to challenge the reader to become adept at what is generally the most difficult hurdle-translating descriptions of statistical problems arising in business and economic settings into a form that lends itself to solutions based on mathematical statistics principles. I have also warned students through the use of asterisks (*) when a proof, concept, ex­ample, or problem may be stretching the bounds of the prerequisites so as not to frustrate the otherwise diligent reader, and to indicate when the help of the instructor or additional readings may be useful.

The book is designed to be versatile. The course that inspired this book is a 4-credit, semester-long, intensive mathematical statistics foundation course. I do not lecture on all of the topics contained in the book in the 50 contact hours available in the semester. The topics that I do not cover are taught in the first half of a subsequent semester-long 3-credit course in statistics and econo­metric methods. I have tended to treat chapters 1 through 4 in detail, and I recommend that this material be thoroughly understood before venturing into the statistical inference portion of the book. Thereafter, the choice of topics is flexible. For example, the instructor can control the depth at which asymp­totic theory is taught by her choice of whether the starred topics in chapter 5 are discussed. While random sampling, empirical distribution functions, and sample moments should be covered in chapter 6, the instructor has leeway in the degree of emphasis that she places on other topics in the chapter. Point estimation and hypothesis testing topics can then be mixed and matched with a minimal amount of back-referencing between the respective chapters.

Distinguishing features of this book include the care with which topics are introduced, motivated, and built upon one another; use of the appropriate level of mathematics; the generous level of detail provided in the proofs; and a familiar business and economics context for examples and problems. This text

x Preface

Acknowledgments

is a bit longer than some of the others in the field. The additional length comes from additional explanation, and the detail in examples, problems, and proofs and not from a proliferation of topics which are merely surveyed rather than fully developed. As I see it, a survey of statistical techniques is useful only after one has the fundamental statistical background to appreciate what is being surveyed. And this book provides the necessary background.

I am indebted to large number of people for their encouragement and comments. Millard Hastay, now retired from the Washington State University economics faculty, is largely responsible for my unwavering curiosity and enthusiasm for the field of theoretical and applied statistics and econometrics. George Judge has been a constant source of encouragement for the book project and over the years has provided me with very valuable and selfless advice and support in all endeavors in which our paths have crossed. I thank Jim Chalfant for giving earlier drafts of chapters a trial run at Berkeley, and for providing me with valuable student and instructor feedback. Thomas Severini at Northwestern provided important and helpful critiques of content and exposition. Martin Gilchrist at Springer-Verlag provided productive and pleasurable guidance to the writing and revision of the text. I also acknowledge the steadfast support of Washington State University in the pursuit of the writing of this book. Of the multitude of past students who contributed so much to the final product and that are too numerous to name explicitly, lowe a special measure of thanks to Don Blayney, now of the Economic Research Service, and Brett Crow, currently a promising Ph.D candidate in Economics at WSU, for reviewing drafts of the text literally character by character and demanding clarification in a number of proofs and examples. I also wish to thank many past secretaries who toiled faithfully on the book project. In particular, I wish to thank Brenda Campbell, who at times literally typed morning, noon, and night to bring the manuscript to completion, and without whom completing the project would have been infinitely more difficult. Finally, I thank my wife Linda, who proofread many parts of the text, provided unwavering support, sustenance, and encouragement to me throughout the project, and, despite all of the trials and tribulations, remains my best friend.

Contents

Preface vii

1. Elements of Probability Theory 1 1.1. Introduction 1 1.2. Experiment, Sample Space, Outcome, and Event 2 1.3. Nonaxiomatic Probability Definitions 5 1.4. Axiomatic Definition of Probability 8 1.5. Some Probability Theorems 15 1.6. A Digression on Events 20 1.7. Conditional Probability 22 1.8. Independence 29 1.9. Bayes's Rule 34 Key Words, Phrases, and Symbols 36 Problems 37

2. Random Variables, Densities, and Cumulative Distribution Functions 43 2.1. Introduction 43 2.2. Univariate Random Variables and Density Functions 44

xii Contents

Probability Space Induced by a Random Variable 45 Discrete Random Variables and Probability Density

Functions 48 Continuous Random Variables and Probability Density

Functions 51 Classes of Discrete and Continuous PDFs 55 Mixed Discrete-Continuous Random Variables 58

2.3. Univariate Cumulative Distribution Functions 60 CDF Properties 63 Duality Between CDFs and PDFs 65

2.4. Multivariate Random Variables, PDFs, and CDFs 67 Multivariate Random Variable Properties and Classes of

PDFs 69 Multivariate CDFs and Duality with PDFs 72 Multivariate Mixed Discrete-Continuous and Composite

Random Variables 76 2.5. Marginal Probability Density Functions and CDFs 77

Bivariate Case 77 N-Variate Case 81 Marginal Cumulative Distribution Functions (MCDFs) 83

2.6. Conditional Density Functions 83 Bivariate Case 84 Conditioning on Elementary Events in Continuous Cases 86 N -Variate Case 88 Conditional CDFs 89

2.7. Independence of Random Variables 90 Bivariate Case 90 N-Variate 93 Independence Between Random Vectors and Between

Functions of Random Vectors 95 2.8. Extended Example of Multivariate Concepts in the

Continuous Case 97 2.9. Events Occurring with Probability Zero 100 Key Words, Phrases, and Symbols 101 Problems 101

3. Mathematical Expectation and Moments 109 3.1. Expectation of a Random Variable 109 3.2. Expectation of a Function of Random Variables 116

Expectation Properties 121 Multivariate Extensions 122

3.3. Conditional Expectation 125 Regression Function 129 Conditional Expectation and Regression in the Multivariate

Case 130

Contents xiii

3.4. Moments of a Random Variable 132 Relationship Between Moments About the Origin and Mean 137 Existence of Moments 138 Nonmoment Measures of Probability Density Characteristics 139

3.5. Moment- and Cumulant-Generating Functions 141 Uniqueness and Inversion of MGFs 143 Cumulant-Generating Function 145 Multivariate Extensions 146

3.6. Joint Moments, Covariance, and Correlation 148 Covariance and Correlation 148 Correlation, Linear Association, and Degeneracy 152

3.7. Means and Variances of Linear Combinations of Random Variables 156

3.8. Necessary and Sufficient Conditions for Positive Semidefiniteness 161

Key Words, Phrases, and Symbols 162 Problems 162

4. Parametric Families of Density Functions 169 4.1. Parametric Families of Discrete Density Functions 170

Family Name: Uniform 170 Family Name: Bernoulli 171 Family Name: Binomial 172 Family Name: Multinomial 174 Family Name: Negative Binomial and Geometric 175 Family Name: Poisson 177 Family Name: Hypergeometric 183 Family Name: Multivariate Hypergeometric 185

4.2. Parametric Families of Continuous Density Functions 186 Family Name: Uniform 186 Family Name: Gamma 187 Gamma Subfamily Name: Exponential 190 Gamma Subfamily Name: Chi-Square 192 Family Name: Beta 194

4.3. The Normal Family of Densities 197 Family Name: Univariate Normal 197 Family Name: Multivariate Normal Density 202

4.4. The Exponential Class of Densities 213 Key Words, Phrases, and Symbols 215 Problems 216

5. Basic Asymptotics 221 5.1. Introduction 221 5.2. Elements of Real Analysis 222

xiv Contents

Limit of a Sequence 224 Continuous Functions 228 Convergence of Function Sequence 230 Order of Magnitude of a Sequence 231

5.3. Types of Random-Variable Convergence 233 Convergence in Distribution 234 Convergence in Probability 241 Convergence in Mean Square lor Convergence in Quadratic

Mean) 249 .. Almost-Sure Convergence lor Convergence with

Probability 1) 253 Relationships Between Convergence Modes 258

5.4. Laws of Large Numbers 258 Weak Laws of Large Numbers IWLLN) 259 ·Strong Laws of Large Numbers ISLLN) 263

5.5. Central Limit Theorems 268 Independent Scalar Random Variables 269 '"Dependent Random Variables 281 Multivariate Central Limit Results 282

5.6. Asymptotic Distributions of Differentiable Functions of Asymptotically Normally Distributed Random Variables 286

Key Words, Phrases, and Symbols 290 Problems 291

6. Sampling, Sample Moments, Sampling Distributions, and Simulation 297

6.1. Introduction 297 6.2. Random Sampling 299

Random Sampling from a Population Distribution 300 Random Sampling Without Replacement 303 Sample Generated by a Composite Experiment 305 Commonalities in Probabilistic Structure of Random

Samples 306 Statistics 307

6.3. Empirical or Sample Distribution Function 308 EDF: Scalar Case 308 EDF: Multivariate Case 313

6.4. Sample Moments and Sample Correlation 314 Scalar Case 314 Multivariate Case 320

6.5. Properties of Xn and S~ When Random Sampling from a Normal Distribution 328

6.6. Sampling Distributions: Deriving Probability Densities of Functions of Random Variables 331 MGF Approach 332

Contents xv

CDF Approach 332 Equivalent Events Approach /Discrete Case) 334 Change of Variables /Continuous Case) 335

6.7. t-and F-Densities 341 t-Density 341 Family Name: t-Family 343 F-Density 345 Family Name: F-Family 346

6.8. Random Sample Simulation and the Probability Integral Transformation 348

6.9. Order Statistics 351 Key Words, Phrases, and Symbols 355 Problems 356

7. Elements of Point Estimation Theory 363

7.1. Introduction 363 7.2. Statistical Models 365 7.3. Estimators and Estimator Properties 370

Estimators 370 Estimator Properties 371 Finite Sample Properties 373 Asymptotic Properties 383

Class of Consistent Asymptotically Normal/CAN) Estimators and Asymptotic Properties 385

7.4. Sufficient Statistics 389 Minimal Sufficient Statistics 393 Sufficient Statistics in the Exponential Class 397 Sufficiency and the MSE Criterion 398 Complete Sufficient Statistics 400 Sufficiency, Minimality, and Completeness of Functions of

Sufficient Statistics 404 7.5. Results on MVUE Estimation 405

Cramer-Rao Lower Bound 408 Complete Sufficient Statistics and MVUEs 420

Key Words, Phrases, and Symbols 421 Problems 422

8. Point Estimation Methods 427

8.1. Introduction 427 8.2. Least Squares and the General Linear Model 428

The Classical GLM Assumptions 429 Estimator for f3 Under Classical GLM Assumptions 433 Estimator for a 2

AUnder Classical GLM Assumptions 437

Consistency of f3 439

xvi Contents

Consistency of S2 441 Asymptotic Normality of ~ 443 Asymptotic Normality of S2 447 Summary of Estimator Properties 449 Violations of Classic GLM Assumptions 452 GLM Assumption Violations: Property Summary and

Epilogue 459 Least Squares Under Normality 460

MVUE Property of {3 and 52 462 8.3. The Method of Maximum Likelihood 464

MLE Mechanics 465 MLE Properties: Finite Sample 470 MLE Properties: Large Sample 472 MLE Invariance Principle 485 MLE Property Summary 489

8.4. The Method of Moments 490 Method of Moments Estimator 491 MOM Estimator Properties 493 Generalized Method of Moments (GMM)

Estimator 495 GMM Properties 497

Key Words, Phrases, and Symbols 502 Problems 502

9. Elements of Hypothesis-Testing Theory 509 9.1. Introduction 509 9.2. Statistical Hypotheses 510 9.3. Basic Hypothesis-Testing Concepts 514

Statistical Hypothesis Tests 514 Type I Error, Type II Error, and Ideal Statistical

Tests 516 Controlling Type I and II Errors 519 Type I/Type II Error Tradeoff 522 Test Statistics 526 Null and Alternative Hypotheses 527

9.4. Parametric Hypothesis Tests and Test Properties 527 Maintained Hypothesis 529 Power Function 530 Properties of Statistical Tests 531 P Values 535 Asymptotic Tests 538

9.5. Results on UMP Tests 539 Neyman-Pearson Approach 539 Monotone Likelihood Ratio Approach 550 Exponential Class of Densities 556

Contents xvii

*Conditioning in the Multiple Parameter Case 571 Concluding Remarks 586

9.6. Noncentral t-Distribution 586 Family Name: Noncentral t-Distribution 586

Key Words, Phrases, and Symbols 588 Problems 589

10. Hypothesis-Testing Methods 595 10.1. Introduction 595 10.2. Heuristic Approach 596 10.3. Generalized Likelihood Ratio Tests 601

Test Properties: Finite Sample 602 Test Properties: Asymptotics 608

10.4. Lagrange Multiplier Tests 616 10.5. Wald Tests 622 10.6. Tests in the GLM 625

Tests When e Is Multivariate Normal 625 Testing R{3 = r, R{3 :::: r, Or R{3 ::: r When R Is 11 x k):

T-Tests 628 Bonferroni Joint Tests of Ri{3 = ri, Ri{3 :::: ri, or Ri{3 ::: ri,

i=I, ... ,m 631 Testing When e Is Not Multivariate Normal 635

Testing R{3 = r or RI{3) = r When R Has q Rows: Asymptotic X2-Tests 636

Testing RIf3J = I, RIf3J :::: I, or RIf3J ::: I When R Is a Scalar Function: Asymptotic Normal Tests 636

10.7. Confidence Intervals and Regions 640 Defining Confidence Regions via Duality with Critical

Re~ons 642 Properties of Confidence Regions 647 Confidence Regions from Pivotal Quantities 649 Confidence Regions as Hypothesis Tests 654

10.8. Nonparametric Tests of Distributional Assumptions 654 Functional Forms of Probability Distributions 655 IID Assumption 663

10.9. Noncentral X2- and F-Distributions 666 Family Name: Noncentral X2-Distribution 666 Family Name: Noncentral F-Distribution 667

Key Words, Phrases, and Symbols 668 Problems 669

Appendix A. Math Review: Sets, Functions, Permutations, Combinations, and Notation 677

A.1. Introduction 677

xviii Contents

A.2. Definitions, Axioms, Theorems, Corollaries, and Lemmas 677 A.3. Elements of Set Theory 679

Set-Defining Methods 680 Set Classifications 681 Special Sets, Set Operations, and Set Relationships 682 Rules Governing Set Operations 684

A.4. Relations, Point Functions, and Set Functions 687 Cartesian Product 688 Relation (Binary) 689 Function 691 Real-Valued Point Versus Set Functions 693

A.S. Combinations and Permutations 696 A.6. Summation, Integration and Matrix Differentiation Notation 699 Key Words, Phrases, and Symbols 701 Problems 702

Appendix B. Useful Tables 705

B.1. Cumulative Normal Distribution 706 B.2. Student's t Distribution 707 B.3. Chi-square Distribution 708 B.4. F-Distribution: 5% Points 710 B.S. F-Distribution: 1 % Points 712

Index 715