Theory and Methods of Statistics

1: Probability Theory

• Abstract
• 1.1 Random Experiments and Their Outcomes
• 1.2 Set Theory
• 1.3 Axiomatic Definition of Probability
• 1.4 Some Simple Propositions
• 1.5 Equally Likely Outcomes in Finite Sample Space
• 1.6 Conditional Probability and Independence
• 1.7 Random Variables and Their Distributions
• 1.8 Expected Value, Variance, Covariance, and Correlation Coefficient
• 1.9 Moments and the Moment Generating Function
• 1.10 Independent Random Variables and Conditioning When There Is Dependence
• 1.11 Transforms of Random Variables and Their Distributions
• Exercises

2: Some Common Probability Distributions

• Abstract
• 2.1 Discrete Distributions
• 2.2 Continuous Distributions
• Exercises

3: Infinite Sequences of Random Variables and Their Convergence Properties

• Abstract
• 3.1 Introduction
• 3.2 Modes of Convergence
• 3.3 Probability Inequalities
• 3.4 Asymptotic Normality: The Central Limit Theorem and Its Generalizations
• Exercises

4: Basic Concepts of Statistical Inference

• Abstract
• 4.1 Population and Random Samples
• 4.2 Parametric and Nonparametric Models
• 4.3 Problems of Statistical Inference
• 4.4 Statistical Decision Functions
• 4.5 Sufficient Statistics
• 4.6 Optimal Decision Rules
• Exercises

5: Point Estimation in Parametric Models

• Abstract
• 5.1 Optimality Under Unbiasedness, Squared-Error Loss, UMVUE
• 5.2 Lower Bound for the Variance of an Unbiased Estimator
• 5.3 Equivariance
• 5.4 Bayesian Estimation Using Conjugate Priors
• 5.5 Methods of Estimation
• Exercises

6: Hypothesis Testing

• Abstract
• 6.1 Early History
• 6.2 Basic Concepts
• 6.3 Simple Null Hypothesis vs Simple Alternative: Neyman-Pearson Lemma
• 6.4 UMP Tests for One-Sided Hypotheses Against One-Sided Alternatives in Monotone Likelihood Ratio Families
• 6.5 Unbiased Tests
• 6.6 Generalized Neyman-Pearson Lemma
• 6.7 UMP Unbiased Tests for Two-Sided Problems
• 6.8 Locally Best Tests
• 6.9 UMP Unbiased Tests in the Presence of Nuisance Parameters: Similarity and Completeness
• 6.10 The p-Value: Another Way to Report the Result of a Test
• 6.11 Sequential Probability Ratio Test
• 6.12 Confidence Sets
• Exercises

7: Methods Based on Likelihood and Their Asymptotic properties

• Abstract
• 7.1 Asymptotic Properties of the MLEs: Consistency and Asymptotic Normality
• 7.2 Likelihood Ratio Test
• 7.3 Asymptotic Properties of MLE and LRT Based on Independent Nonidentically Distributed Data
• 7.4 Frequency X²
• Exercises

8: Distribution-Free Tests for Hypothesis Testing in Nonparametric Families

• Abstract
• 8.1 Ranks and Order Statistics
• 8.2 Locally Most Powerful Rank Tests
• 8.3 Tests Based on Empirical Distribution Function
• Exercises

9: Curve Estimation

• Abstract
• 9.1 Introduction
• 9.2 Density Estimation
• 9.3 Regression Estimation
• 9.4 Nearest Neighbor Approach
• 9.5 Curve Estimation in Higher Dimension
• 9.6 Curve Estimation Using Local Polynomials
• 9.7 Estimation of Survival Function and Hazard Rates Under Random Right-Censoring
• Exercises

10: Statistical Functionals and Their Use in Robust Estimation

• Abstract
• 10.1 Introduction
• 10.2 Functional Delta Method
• 10.3 The L-Estimators
• 10.4 The M-Estimators
• 10.5 A Relation Between L-Estimators and M-Estimators
• 10.6 The Remainder Term R_n
• 10.7 The Jackknife and the Bootstrap
• Exercises

11: Linear Models

• Abstract
• 11.1 Introduction
• 11.2 Examples of Gauss-Markov Models
• 11.3 Gauss-Markov Models: Estimation
• 11.4 Decomposition of Total Sum of Squares
• 11.5 Estimation Under Linear Restrictions on β
• 11.6 Gauss-Markov Models: Inference
• 11.7 Analysis of Covariance
• 11.8 Model Selection
• 11.9 Some Alternate Methods for Regression
• 11.10 Random- and Mixed-Effects Models
• 11.11 Inference: Examples From Mixed Models
• Exercises

12: Multivariate Analysis

• Abstract
• 12.1 Introduction
• 12.2 Wishart Distribution
• 12.3 The Role of Multivariate Normal Distribution
• 12.4 One-Sample Inference
• 12.5 Two-Sample Problem
• 12.6 One-Factor MANOVA
• 12.7 Two-Factor MANOVA
• 12.8 Multivariate Linear Model
• 12.9 Principal Components Analysis
• 12.10 Factor Analysis
• 12.11 Classification and Discrimination
• 12.12 Canonical Correlation Analysis
• Exercises

13: Time Series

• Abstract
• 13.1 Introduction
• 13.2 Concept of Stationarity
• 13.3 Estimation of the Mean and the Autocorrelation Function
• 13.4 Partial Autocorrelation Function (PACF)
• 13.5 Causality and Invertibility
• 13.6 Forecasting
• 13.7 ARIMA Models and Forecasting
• 13.8 Parameter Estimation
• 13.9 Selection of an Appropriate ARMA model
• 13.10 Spectral Analysis
• Exercises

Appendix A: Results From Analysis and Probability

• A.1 Some Important Results in Integration Theory
• A.2 Convex Functions
• A.3 Stieltjes Integral
• A.4 Characteristic Function, Weak Law of Large Number, and Central Limit Theorem
• A.5 Weak Convergence of Probabilities on C[0,1]

Appendix B: Basic Results From Matrix Algebra

• B.1 Some Elementary Facts
• B.2 Eigenvalues and Eigenvectors
• B.3 Functions of Symmetric Matrices
• B.4 Generalized Eigenvalues
• B.5 Matrix Derivatives
• B.6 Orthogonal Projection
• B.7 Distribution of Quadratic Forms