A First Course in Stochastic Processes

A First Course in Stochastic Processes

1st Edition - January 1, 1968

Write a review

  • Author: Samuel Karlin
  • eBook ISBN: 9781483268095

Purchase options

Purchase options
DRM-free (PDF)
Sales tax will be calculated at check-out

Institutional Subscription

Free Global Shipping
No minimum order


A First Course in Stochastic Processes focuses on several principal areas of stochastic processes and the diversity of applications of stochastic processes, including Markov chains, Brownian motion, and Poisson processes. The publication first takes a look at the elements of stochastic processes, Markov chains, and the basic limit theorem of Markov chains and applications. Discussions focus on criteria for recurrence, absorption probabilities, discrete renewal equation, classification of states of a Markov chain, and review of basic terminologies and properties of random variables and distribution functions. The text then examines algebraic methods in Markov chains and ratio theorems of transition probabilities and applications. The manuscript elaborates on the sums of independent random variables as a Markov chain, classical examples of continuous time Markov chains, and continuous time Markov chains. Topics include differentiability properties of transition probabilities, birth and death processes with absorbing states, general pure birth processes and Poisson processes, and recurrence properties of sums of independent random variables. The book then ponders on Brownian motion, compounding stochastic processes, and deterministic and stochastic genetic and ecological processes. The publication is a valuable source of information for readers interested in stochastic processes.

Table of Contents

  • Preface

    Chapter 1 Elements of Stochastic Processes

    1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions

    2. Two Simple Examples of Stochastic Processes

    3. Classification of General Stochastic Processes



    Chapter 2 Markov Chains

    1. Definitions

    2. Examples of Markov Chains

    3. Transition Probability Matrices of a Markov Chain

    4. Classification of States of a Markov Chain

    5. Recurrence

    6. Examples of Recurrent Markov Chains

    7. More on Recurrence



    Chapter 3 The Basic Limit Theorem of Markov Chains and Applications

    1. Discrete Renewal Equation

    2. Proof of Theorem 1.1

    3. Absorption Probabilities

    4. Criteria for Recurrence

    5. A Queueing Example

    6. Another Queueing Model

    7. Random Walk



    Chapter 4 Algebraic Methods in Markov Chains

    1. Preliminaries

    2. Relations of Eigenvalues and Recurrence Classes

    3. Periodic Classes

    4. Special Computational Methods in Markov Chains

    5. Examples

    6. Applications to Coin Tossing



    Chapter 5 Ratio Theorems of Transition Probabilities and Applications

    1. Taboo Probabilities

    2. Ratio Theorems

    3. Existence of Generalized Stationary Distributions

    4. Interpretation of Generalized Stationary Distributions

    5. Regular, Superregular, and Subregular Sequences for Markov Chains



    Chapter 6 Sums of Independent Random Variables as a Markov Chain

    1. Recurrence Properties of Sums of Independent Random Variables

    2. Local Limit Theorems

    3. Right Regular Sequences for the Markov Chain {Sn}



    Chapter 7 Classical Examples of Continuous Time Markov Chains

    1. General Pure Birth Processes and Poisson Processes

    2. More about Poisson Processes

    3. A Counter Model

    4. Birth and Death Processes

    5. Differential Equations of Birth and Death Processes

    6. Examples of Birth and Death Processes

    7. Birth and Death Processes with Absorbing States

    8. Finite State Continuous Time Markov Chains



    Chapter 8 Continuous Time Markov Chains

    1. Differentiability Properties of Transition Probabilities

    2. Conservative Processes and the Forward and Backward Differential Equations

    3. Construction of a Continuous Time Markov Chain from Its Infinitesimal Parameters

    4. Strong Markov Property



    Chapter 9 Order Statistics, Poisson Processes, and Applications

    1. Order Statistics and Their Relation to Poisson Processes

    2. The Ballot Problem

    3. Empirical Distribution Functions and Order Statistics

    4. Some Limit Distributions for Empirical Distribution Functions



    Chapter 10 Brownian Motion

    1. Background Material

    2. Joint Probabilities for Brownian Motion

    3. Continuity of Paths and the Maximum Variables



    Chapter 11 Branching Processes

    1. Discrete Time Branching Processes

    2. Generating Function Relations for Branching Processes

    3. Extinction Probabilities

    4. Examples

    5. Two-Type Branching Processes

    6. Multi-Type Branching Processes

    7. Continuous Time Branching Processes

    8. Extinction Probabilities for Continuous Time Branching Processes

    9. Limit Theorems for Continuous Time Branching Processes

    10. Two-Type Continuous Time Branching Process

    11. Branching Processes with General Variable Lifetime



    Chapter 12 Compounding Stochastic Processes

    1. Multidimensional Homogeneous Poisson Processes

    2. An Application of Multidimensional Poisson Processes to Astronomy

    3. Immigration and Population Growth

    4. Stochastic Models of Mutation and Growth

    5. One-Dimensional Geometric Population Growth

    6. Stochastic Population Growth Model in Space and Time

    7. Deterministic Population Growth with Age Distribution

    8. A Discrete Aging Model



    Chapter 13 Deterministic and Stochastic Genetic and Ecological Processes

    1. Genetic Models; Description of the Genetic Mechanism

    2. Inbreeding

    3. Polyploidy

    4. Markov Processes Induced by Direct Product Branching Processes

    5. Multi-Type Population Frequency Models

    6. Eigenvalues of Markov Chains Induced by Direct Product Branching Processes

    7. Eigenvalues of Multi-Type Mutation Model

    8. Probabilistic Interpretations of the Eigenvalues



    Chapter 14 Queueing Processes

    1. General Description

    2. The Simplest Queueing Processes (M/M/1)

    3. Some General One-Server Queueing Models

    4. Embedded Markov Chain Method Applied to the Queueing Model (M/GI/1)

    5. Exponential Service Times (G/M/1)

    6. Gamma Arrival Distribution and Generalizations (Ek/M/1)

    7. Exponential Service with 8 Servers (GI/M/8)

    8. The Virtual Waiting Time and the Busy Period



    Appendix. Review of Matrix Analysis

    1. The Spectral Theorem

    2. The Frobenius Theory of Positive Matrices

    Miscellaneous Problems


Product details

  • No. of pages: 514
  • Language: English
  • Copyright: © Academic Press 1968
  • Published: January 1, 1968
  • Imprint: Academic Press
  • eBook ISBN: 9781483268095

About the Author

Samuel Karlin

Affiliations and Expertise

Stanford University and The Weizmann Institute of Science

Ratings and Reviews

Write a review

There are currently no reviews for "A First Course in Stochastic Processes"