Theory and Methods of Statistics

Theory and Methods of Statistics covers essential topics for advanced graduate students and professional research statisticians. This comprehensive resource covers many important areas in one manageable volume, including core subjects such as probability theory, mathematical statistics, and linear models, and various special topics, including nonparametrics, curve estimation, multivariate analysis, time series, and resampling. The book presents subjects such as "maximum likelihood and sufficiency," and is written with an intuitive, heuristic approach to build reader comprehension. It also includes many probability inequalities that are not only useful in the context of this text, but also as a resource for investigating convergence of statistical procedures.

• Codifies foundational information in many core areas of statistics into a comprehensive and definitive resource
• Serves as an excellent text for select master’s and PhD programs, as well as a professional reference
• Integrates numerous examples to illustrate advanced concepts
• Includes many probability inequalities useful for investigating convergence of statistical procedures

Graduate (Masters/PhD) students and research statisticians.

1: Probability Theory

• Abstract
• 1.1 Random Experiments and Their Outcomes
• 1.2 Set Theory
• 1.3 Axiomatic Definition of Probability
• 1.4 Some Simple Propositions
• 1.5 Equally Likely Outcomes in Finite Sample Space
• 1.6 Conditional Probability and Independence
• 1.7 Random Variables and Their Distributions
• 1.8 Expected Value, Variance, Covariance, and Correlation Coefficient
• 1.9 Moments and the Moment Generating Function
• 1.10 Independent Random Variables and Conditioning When There Is Dependence
• 1.11 Transforms of Random Variables and Their Distributions
• Exercises

2: Some Common Probability Distributions

• Abstract
• 2.1 Discrete Distributions
• 2.2 Continuous Distributions
• Exercises

3: Infinite Sequences of Random Variables and Their Convergence Properties

• Abstract
• 3.1 Introduction
• 3.2 Modes of Convergence
• 3.3 Probability Inequalities
• 3.4 Asymptotic Normality: The Central Limit Theorem and Its Generalizations
• Exercises

4: Basic Concepts of Statistical Inference

• Abstract
• 4.1 Population and Random Samples
• 4.2 Parametric and Nonparametric Models
• 4.3 Problems of Statistical Inference
• 4.4 Statistical Decision Functions
• 4.5 Sufficient Statistics
• 4.6 Optimal Decision Rules
• Exercises

5: Point Estimation in Parametric Models

• Abstract
• 5.1 Optimality Under Unbiasedness, Squared-Error Loss, UMVUE
• 5.2 Lower Bound for the Variance of an Unbiased Estimator
• 5.3 Equivariance
• 5.4 Bayesian Estimation Using Conjugate Priors
• 5.5 Methods of Estimation
• Exercises

6: Hypothesis Testing

• Abstract
• 6.1 Early History
• 6.2 Basic Concepts
• 6.3 Simple Null Hypothesis vs Simple Alternative: Neyman-Pearson Lemma
• 6.4 UMP Tests for One-Sided Hypotheses Against One-Sided Alternatives in Monotone Likelihood Ratio Families
• 6.5 Unbiased Tests
• 6.6 Generalized Neyman-Pearson Lemma
• 6.7 UMP Unbiased Tests for Two-Sided Problems
• 6.8 Locally Best Tests
• 6.9 UMP Unbiased Tests in the Presence of Nuisance Parameters: Similarity and Completeness
• 6.10 The p-Value: Another Way to Report the Result of a Test
• 6.11 Sequential Probability Ratio Test
• 6.12 Confidence Sets
• Exercises

7: Methods Based on Likelihood and Their Asymptotic properties

• Abstract
• 7.1 Asymptotic Properties of the MLEs: Consistency and Asymptotic Normality
• 7.2 Likelihood Ratio Test
• 7.3 Asymptotic Properties of MLE and LRT Based on Independent Nonidentically Distributed Data
• 7.4 Frequency X2
• Exercises

8: Distribution-Free Tests for Hypothesis Testing in Nonparametric Families

• Abstract
• 8.1 Ranks and Order Statistics
• 8.2 Locally Most Powerful Rank Tests
• 8.3 Tests Based on Empirical Distribution Function
• Exercises

9: Curve Estimation

• Abstract
• 9.1 Introduction
• 9.2 Density Estimation
• 9.3 Regression Estimation
• 9.4 Nearest Neighbor Approach
• 9.5 Curve Estimation in Higher Dimension
• 9.6 Curve Estimation Using Local Polynomials
• 9.7 Estimation of Survival Function and Hazard Rates Under Random Right-Censoring
• Exercises

10: Statistical Functionals and Their Use in Robust Estimation

• Abstract
• 10.1 Introduction
• 10.2 Functional Delta Method
• 10.3 The L-Estimators
• 10.4 The M-Estimators
• 10.5 A Relation Between L-Estimators and M-Estimators
• 10.6 The Remainder Term Rn
• 10.7 The Jackknife and the Bootstrap
• Exercises

11: Linear Models

• Abstract
• 11.1 Introduction
• 11.2 Examples of Gauss-Markov Models
• 11.3 Gauss-Markov Models: Estimation
• 11.4 Decomposition of Total Sum of Squares
• 11.5 Estimation Under Linear Restrictions on β
• 11.6 Gauss-Markov Models: Inference
• 11.7 Analysis of Covariance
• 11.8 Model Selection
• 11.9 Some Alternate Methods for Regression
• 11.10 Random- and Mixed-Effects Models
• 11.11 Inference: Examples From Mixed Models
• Exercises

12: Multivariate Analysis

• Abstract
• 12.1 Introduction
• 12.2 Wishart Distribution
• 12.3 The Role of Multivariate Normal Distribution
• 12.4 One-Sample Inference
• 12.5 Two-Sample Problem
• 12.6 One-Factor MANOVA
• 12.7 Two-Factor MANOVA
• 12.8 Multivariate Linear Model
• 12.9 Principal Components Analysis
• 12.10 Factor Analysis
• 12.11 Classification and Discrimination
• 12.12 Canonical Correlation Analysis
• Exercises

13: Time Series

• Abstract
• 13.1 Introduction
• 13.2 Concept of Stationarity
• 13.3 Estimation of the Mean and the Autocorrelation Function
• 13.4 Partial Autocorrelation Function (PACF)
• 13.5 Causality and Invertibility
• 13.6 Forecasting
• 13.7 ARIMA Models and Forecasting
• 13.8 Parameter Estimation
• 13.9 Selection of an Appropriate ARMA model
• 13.10 Spectral Analysis
• Exercises

Appendix A: Results From Analysis and Probability

• A.1 Some Important Results in Integration Theory
• A.2 Convex Functions
• A.3 Stieltjes Integral
• A.4 Characteristic Function, Weak Law of Large Number, and Central Limit Theorem
• A.5 Weak Convergence of Probabilities on C[0,1]

Appendix B: Basic Results From Matrix Algebra

• B.1 Some Elementary Facts
• B.2 Eigenvalues and Eigenvectors
• B.3 Functions of Symmetric Matrices
• B.4 Generalized Eigenvalues
• B.5 Matrix Derivatives
• B.6 Orthogonal Projection
• B.7 Distribution of Quadratic Forms