A First Course in Stochastic Processes

A First Course in Stochastic Processes focuses on several principal areas of stochastic processes and the diversity of applications of stochastic processes, including Markov chains, Brownian motion, and Poisson processes.

The publication first takes a look at the elements of stochastic processes, Markov chains, and the basic limit theorem of Markov chains and applications. Discussions focus on criteria for recurrence, absorption probabilities, discrete renewal equation, classification of states of a Markov chain, and review of basic terminologies and properties of random variables and distribution functions. The text then examines algebraic methods in Markov chains and ratio theorems of transition probabilities and applications.

The manuscript elaborates on the sums of independent random variables as a Markov chain, classical examples of continuous time Markov chains, and continuous time Markov chains. Topics include differentiability properties of transition probabilities, birth and death processes with absorbing states, general pure birth processes and Poisson processes, and recurrence properties of sums of independent random variables. The book then ponders on Brownian motion, compounding stochastic processes, and deterministic and stochastic genetic and ecological processes.

The publication is a valuable source of information for readers interested in stochastic processes.

Preface

Chapter 1 Elements of Stochastic Processes

1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions

2. Two Simple Examples of Stochastic Processes

3. Classification of General Stochastic Processes

Problems

References

Chapter 2 Markov Chains

1. Definitions

2. Examples of Markov Chains

3. Transition Probability Matrices of a Markov Chain

4. Classification of States of a Markov Chain

5. Recurrence

6. Examples of Recurrent Markov Chains

7. More on Recurrence

Problems

References

Chapter 3 The Basic Limit Theorem of Markov Chains and Applications

1. Discrete Renewal Equation

2. Proof of Theorem 1.1

3. Absorption Probabilities

4. Criteria for Recurrence

5. A Queueing Example

6. Another Queueing Model

7. Random Walk

Problems

References

Chapter 4 Algebraic Methods in Markov Chains

1. Preliminaries

2. Relations of Eigenvalues and Recurrence Classes

3. Periodic Classes

4. Special Computational Methods in Markov Chains

5. Examples

6. Applications to Coin Tossing

Problems

References

Chapter 5 Ratio Theorems of Transition Probabilities and Applications

1. Taboo Probabilities

2. Ratio Theorems

3. Existence of Generalized Stationary Distributions

4. Interpretation of Generalized Stationary Distributions

5. Regular, Superregular, and Subregular Sequences for Markov Chains

Problems

References

Chapter 6 Sums of Independent Random Variables as a Markov Chain

1. Recurrence Properties of Sums of Independent Random Variables

2. Local Limit Theorems

3. Right Regular Sequences for the Markov Chain {Sn}

Problems

References

Chapter 7 Classical Examples of Continuous Time Markov Chains

1. General Pure Birth Processes and Poisson Processes

2. More about Poisson Processes

3. A Counter Model

4. Birth and Death Processes

5. Differential Equations of Birth and Death Processes

6. Examples of Birth and Death Processes

7. Birth and Death Processes with Absorbing States

8. Finite State Continuous Time Markov Chains

Problems

References

Chapter 8 Continuous Time Markov Chains

1. Differentiability Properties of Transition Probabilities

2. Conservative Processes and the Forward and Backward Differential Equations

3. Construction of a Continuous Time Markov Chain from Its Infinitesimal Parameters

4. Strong Markov Property

Problems

References

Chapter 9 Order Statistics, Poisson Processes, and Applications

1. Order Statistics and Their Relation to Poisson Processes

2. The Ballot Problem

3. Empirical Distribution Functions and Order Statistics

4. Some Limit Distributions for Empirical Distribution Functions

Problems

References

Chapter 10 Brownian Motion

1. Background Material

2. Joint Probabilities for Brownian Motion

3. Continuity of Paths and the Maximum Variables

Problems

References

Chapter 11 Branching Processes

1. Discrete Time Branching Processes

2. Generating Function Relations for Branching Processes

3. Extinction Probabilities

4. Examples

5. Two-Type Branching Processes

6. Multi-Type Branching Processes

7. Continuous Time Branching Processes

8. Extinction Probabilities for Continuous Time Branching Processes

9. Limit Theorems for Continuous Time Branching Processes

10. Two-Type Continuous Time Branching Process

11. Branching Processes with General Variable Lifetime

Problems

References

Chapter 12 Compounding Stochastic Processes

1. Multidimensional Homogeneous Poisson Processes

2. An Application of Multidimensional Poisson Processes to Astronomy

3. Immigration and Population Growth

4. Stochastic Models of Mutation and Growth

5. One-Dimensional Geometric Population Growth

6. Stochastic Population Growth Model in Space and Time

7. Deterministic Population Growth with Age Distribution

8. A Discrete Aging Model

Problems

References

Chapter 13 Deterministic and Stochastic Genetic and Ecological Processes

1. Genetic Models; Description of the Genetic Mechanism

2. Inbreeding

3. Polyploidy

4. Markov Processes Induced by Direct Product Branching Processes

5. Multi-Type Population Frequency Models

6. Eigenvalues of Markov Chains Induced by Direct Product Branching Processes

7. Eigenvalues of Multi-Type Mutation Model

8. Probabilistic Interpretations of the Eigenvalues

Problems

References

Chapter 14 Queueing Processes

1. General Description

2. The Simplest Queueing Processes (M/M/1)

3. Some General One-Server Queueing Models

4. Embedded Markov Chain Method Applied to the Queueing Model (M/GI/1)

5. Exponential Service Times (G/M/1)

6. Gamma Arrival Distribution and Generalizations (Ek/M/1)

7. Exponential Service with 8 Servers (GI/M/8)

8. The Virtual Waiting Time and the Busy Period

Problems

References

Appendix. Review of Matrix Analysis

1. The Spectral Theorem

2. The Frobenius Theory of Positive Matrices

Miscellaneous Problems

Index