# Probabilistic Reasoning in Intelligent Systems

**Networks of Plausible Inference**

**By**

- Judea Pearl

The author distinguishes syntactic and semantic approaches to uncertainty--and offers techniques, based on belief networks, that provide a mechanism for making semantics-based systems operational. Specifically, network-propagation techniques serve as a mechanism for combining the theoretical coherence of probability theory with modern demands of reasoning-systems technology: modular declarative inputs, conceptually meaningful inferences, and parallel distributed computation. Application areas include diagnosis, forecasting, image interpretation, multi-sensor fusion, decision support systems, plan recognition, planning, speech recognition--in short, almost every task requiring that conclusions be drawn from uncertain clues and incomplete information.

Probabilistic Reasoning in Intelligent Systems will be of special interest to scholars and researchers in AI, decision theory, statistics, logic, philosophy, cognitive psychology, and the management sciences. Professionals in the areas of knowledge-based systems, operations research, engineering, and statistics will find theoretical and computational tools of immediate practical use. The book can also be used as an excellent text for graduate-level courses in AI, operations research, or applied probability.

### Book information

- Published: September 1988
- Imprint: MORGAN KAUFMANN
- ISBN: 978-1-55860-479-7

### Table of Contents

**Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference**

by Judea Pearl

Revised Second Printing

by Judea Pearl

Revised Second Printing

**Chapter 1 Uncertainty In AI Systems: An Overview**- 1.1 Introduction
- 1.1.1 Why bother with Uncertainty?
- 1.1.2 Why Is It a Problem?
- 1.1.3 Approaches to Uncertainty
- 1.1.4 Extensional vs. Intensional Approaches

- 1.2 Extensional Systems: Merits, Deficiencies, and Remedies
- 1.2.1 Computational Merits
- 1.2.2 Semantic Deficiencies
- 1.2.3 Attempted Remedies and their Limitations

- 1.3 Intensional Systems and Network Representations
- 1.3.1 Why Networks?
- 1.3.2 Graphoids and the Formalization of Relevance and Causality

- 1.4 The Case for Probabilities
- 1.4.1 Why Should Beliefs Combine Like Frequencies?
- 1.4.2 The Primitive Relationships of Probability Language
- 1.4.3 Probability as a Faithful Guardian of Common Sense

- 1.5 Qualitative Reasoning With Probabilities
- 1.5.1 Softened Logic vs. Hardened Probabilities
- 1.5.2 Probabilities and the Logic of "Almost True"

- 1.6 Bibliographical and Historical Remarks

- 1.1 Introduction
**Chapter 2 Bayesian Inference**- 2.1 Basic Concepts
- 2.1.1 Probabilistic Formulation and Bayesian Inversion
- 2.1.2 Combining Predictive and Diagnostic Supports
- 2.1.3 Pooling of Evidence
- 2.1.4 Recursive Bayesian Updating
- 2.1.5 Multi-Valued Hypotheses

- 2.2 Hierarchical Modeling
- 2.2.1 Uncertain Evidence (Cascaded Inference)
- 2.2.2 Virtual (Intangible) Evidence
- 2.2.3 Predicting Future Events
- 2.2.4 Multiple Causes and "Explaining Away"
- 2.2.5 Belief Networks and the Role of Causality

- 2.3 Epistemological Issues of Belief Updating
- 2.3.1 Patterns of Plausible Inference: Polya vs. Bayes?
- 2.3.2 The Three Prisoners Paradox: When the Bare Facts Won't Do
- 2.3.3 Jeffrey's Rule and the Problem of Autonomous Inference Agents

- 2.4 Bibliographical and Historical Remarks
- Exercises

- 2.1 Basic Concepts
**Chapter 3 Markov and Bayesian Networks: Two Graphical Representations of Probabilistic Knowledge**- 3.1 From Numeral to Graphical Representations
- 3.1.1 Introduction
- 3.1.2 An Axiomatic Basis for Probabilistic Dependencies
- 3.1.3 On Representing Dependencies by Undirected Graphs
- 3.1.4 Axiomatic Characterization of Graph-Isomorph Dependencies

- 3.2 Markov Networks
- 3.2.1 Definitions and Formal Properties
- 3.2.2 Illustrations
- 3.2.3 Markov Network as a Knowledge Base
- 3.2.4 Decomposable Models

- 3.3 Bayesian Networks
- 3.3.1 Dependence Semantics for Bayesian Networks
- 3.3.2 Bayesian Network as a Knowledge Base
- 3.3.3 How Expressive are Bayesian Networks?

- 3.4 Bibliographical and Historical Remarks
- Exercises
- Appendix 3-A Proof of Theorem 3
- Appendix 3-B Proof of Theorem 4

- 3.1 From Numeral to Graphical Representations
**Chapter 4 Belief Updating by Network Propagation**- 4.1 Autonomous Propagation as a Computational Paradigm
- 4.1.1 Constraint Propagation and Rule-based Computation
- 4.1.2 Why Probabilistic Reasoning Seems to Resist Propagation

- 4.2 Belief Propagation in Causal Trees
- 4.2.1 Notation
- 4.2.2 Propagation in Chains
- 4.2.3 Propagation in Trees

- 4.3 Belief Propagation in Causal Polytrees (Singly Connected Networks)
- 4.3.1 Propagation Rules
- 4.3.2 Canonical Models of Multicausal Interactions

- 4.4 Coping with Loops
- 4.4.1 Clustering Methods
- 4.4.2 The Method of Conditioning (Reasoning by Assumptions)
- 4.4.3 Stochastic Simulation

- 4.5 What Else Can Bayesian Networks Compute?
- 4.5.1 Answering Queries
- 4.5.2 Introducing Constraints
- 4.5.3 Answering Conjunctive Queries

- Exercises
- Appendix 4-A Auxiliary Derivations for Section 4.5.3

- 4.1 Autonomous Propagation as a Computational Paradigm
**Chapter 5 Distributed Revision of Composite Beliefs**- 5.1 Introduction
- 5.2 Illustrating the Propagation Scheme
- 5.2.1 Belief Updating (A Review)
- 5.2.2 Belief Revision

- 5.3 Belief Revision in Singly Connected Networks
- 5.3.1 Deriving the Propagation Rules
- 5.3.2 Summary of Propagation Rules
- 5.3.3 Reaching Equilibrium and Assembling a Composite Solution
- 5.3.4 Comparison to Belief Updating
- 5.3.5 Coping with Loops

- 5.4 Diagnosis of Systems with Multiple Faults
- 5.4.1 An Example: Electronic Circuit
- 5.4.2 Initialization
- 5.4.3 Fault Interpretation
- 5.4.4 Finding the Second-Best Interpretation

- 5.5 Application to Medical Diagnosis
- 5.5.1 The Causal Model
- 5.5.2 Message Propagation
- 5.5.3 Choosing the Best Interpretation
- 5.5.4 Generating Explanations
- 5.5.5 Reversibility vs. Perseverance

- 5.6 The Nature of Explanations
- 5.6.1 Accepting vs. Assessing Beliefs
- 5.6.2 Is a Most-Probable Explanation Adequate?
- 5.6.3 Circumscribing Explanations

- 5.7 Conclusions
- 5.8 Bibliographical and Historical Remarks
- Exercises

**Chapter 6 Decision and Control**- 6.1 From Beliefs to Actions: Introduction to Decision Analysis
- 6.1.1. Rational Decisions and Quality Guarantees
- 6.1.2 Consequences, Payoffs, and Lotteries
- 6.1.3 Calibrating the Value of a Lottery
- 6.1.4 The Axioms of Utility Theory
- 6.1.5 Utility Functions

- 6.2 Decision Trees and Influence Diagrams
- 6.2.1 Decision Trees
- 6.2.2 Planning with Decision Trees
- 6.2.3 Influence Diagrams

- 6.3 The Value of Information
- 6.3.1 Information Sources and Their Values
- 6.3.2 Myopic Assessments of Information Sources

- 6.4 Relevance-Based Control
- 6.4.1 Focusing Attention
- 6.4.2 Utility-Free Assessment of Information Sources
- 6.4.3 Controlling Attention
- 6.4.4 Summary

- 6.5 Bibliographical and Historical Remarks
- Exercises

- 6.1 From Beliefs to Actions: Introduction to Decision Analysis
**Chapter 7 Taxonomic Hierarchies, Continuous Variables, and Uncertain Probabilities**- 7.1 Evidential Reasoning in Taxonomic Hierarchies
- 7.1.1 Taxonomic vs. Causal Hierarchies
- 7.1.2 Evidence Propagation in Taxonomic Hierarchies
- 7.1.3 Probabilistic Justification
- 7.1.4 Psychological and Computational Characteristics

- 7.2 Managing Continuous Variables
- 7.2.1 Plausible Reasoning about Uncertain Quantities
- 7.2.2 Propagating Estimates and Ranges
- 7.2.3 Qualitative Patterns of Reasoning
- 7.2.4 Discussion

- 7.3 Representing Uncertainty About Probabilities
- 7.3.1 The Semantics of Probabilities of Probabilities
- 7.3.2 De Finetti's Paradigm of Uncertain Contingencies
- 7.3.3 A Formal Definition of Network-Induced Confidence Measures
- 7.3.4 The Effect of Evidence on Confidence: An Example
- 7.3.5 Conclusions

- 7.4 Bibliographical and Historical Remarks
- Exercises
- Appendix 7-A Derivation of Propagation Rules for Continuous Variables

- 7.1 Evidential Reasoning in Taxonomic Hierarchies
**Chapter 8 Learning Structure from Data**- 8.1 Causality, Modularity, and Tree Structurs
- 8.2 Structuring the Observables
- 8.2.1 Chow's Method of Constructing Trees
- 8.2.2 Structuring Causal Polytrees
- 8.2.3 Conclusions: Causation or Covariation?

- 8.3 Learning Hidden Causes
- 8.3.1 Problem Definition and Nomenclature
- 8.3.2 Star-Decomposable Triplets
- 8.3.3 A Tree-Reconstruction Procedure
- 8.3.4 Extensions to Normal Variables
- 8.3.5 Conclusions and Open Questions

- 8.4 Bibliographical and Historical Remarks
- Exercises
- Appendix 8-A Proof of theorems 1 and 2
- Appendix 8-B Conditions for Star-Decomposability (After Lazarfeld [1966])

**Chapter 9 Non-Bayesian Formalisms for Managing Uncertainty**- 9.1 The Dempster-Shafer Theory
- 9.1.1 Basic Concepts
- 9.1.2 Comparing Bayesian and Dempster-Shafer Formalisms
- 9.1.3 Dempster's Rule of Combination
- 9.1.4 More on the Nature of Probability Intervals
- 9.1.5 Applications to Rule-bases Systems
- 9.1.6 Bayes vs. Demster-Shafer: A Semantic Clash

- 9.2 Truth Maintenance Systems
- 9.2.1 Naming the Assumptions
- 9.2.2 Uncertainty Management in an ATMS
- 9.2.3 Incidence Calculus

- 9.3 Probabilistic Logic
- 9.4 Bibliographical and Historical Remarks
- Exercises

- 9.1 The Dempster-Shafer Theory
**Chapter 10 Logic and Probability: The Strange Connection**- 10.1 Introduction to Nonmonotonic Reasoning
- 10.1.1 Reiter's Default Logic
- 10.1.2 Problems with Default Logics
- 10.1.3 Empirical vs. Procedural Semantics in Default Reasoning

- 10.2 Probabilistic Semantics for Default Reasoning
- 10.2.1 (-semantics
- 10.2.2 Axiomatic Characterization and a System of Defeasible Inference
- 10.2.3 Relevance-based Conventions
- 10.2.4 Do People Use the Logic of Extreme Probabilities?

- 10.3 Embracing Causality in Default Reasoning
- 10.3.1 How Old Beliefs Were Established Determines Which New Beliefs are Evoked
- 10.3.2 More on the Distrinction Between Causal and Evidential Support
- 10.3.3 The C-E System: A Coarse Logical Abstraction of Causal Directionality
- 10.3.4 Implicit Suppressors and the Need for Finer Abstractions

- 10.4 A Probabilistic Treatment of the Yale Shooting Problem
- 10.4.1 The Problem and its Solution
- 10.4.2 Concluding Dialogue

- 10.5 Bibliographical and Historical Remarks
- Exercises

- 10.1 Introduction to Nonmonotonic Reasoning
- Bibliography
- Author Index
- Subject Index