The Acquisition of Strategic Knowledge

ForewordPrefaceAcknowledgments1 Introduction and Overview 1.1 The Knowledge Acquisition Problem 1.1.1 Why is Knowledge Acquisition Difficult 1.1.2 Automating Knowledge Acquisition 1.2 Strategic Knowledge 1.2.1 What is Strategic Knowledge 1.2.2 Strategic and Substantive Knowledge 1.2.3 Strategic Knowledge and Search Control 1.2.4 Strategic Knowledge and Planning 1.2.5 Strategic Expertise 1.3 The Problem of Acquiring Strategic Knowledge 1.3.1 The Current Approach 1.3.2 Why Acquire Strategic Knowledge 1.4 An Approach to the Acquisition of Strategic Knowledge 1.5 Overview of the Ask Knowledge Acquisition Assistant 1.6 An Example Dialog with Ask 1.7 An Analysis of the Sources of Power in Ask 1.8 Scope of Applicability of Ask 1.9 Assumptions and Limitations 1.10 Major Conclusions2. A Knowledge Acquisition Dialog 2.1 The Domain: A Workup for Chest Pain 2.2 What the System Already Knows 2.3 Running the Application System 2.4 The Knowledge Acquisition Dialog 2.4.1 Eliciting the Expert's Critique 2.4.2 Eliciting Justifications 2.4.3 Generating and Verifying a Strategy Rule 2.4.4 Acquiring a New Term 2.4.5 Generalizing the New Rule 2.4.6 Acquiring Trade-offs Among Features 2.5 Summary of the Dialog Session3. Representing Strategic Knowledge 3.1 Design Criteria for a Representation of Strategic Knowledge 3.1.1 General Principles of the Design of Representations for Knowledge Acquisition 3.1.2 Special Requirements for Representing Strategic Knowledge 3.2 The Architecture: MU 3.2.1 The Structure and Function of MU 3.2.2 The Inference Network 3.2.3 Control Features 3.3 The Language: Strategy Rules 3.3.1 Focus Rules 3.3.2 Filter Rules 3.3.3 Selection Rules 3.3.4 Rule Matching and the Control Cycle 3.3.5 Precedence Among Rules: Shadowing 3.3.6 Shadowing Versus Weighted-sum Preference Models 3.3.7 An Example: Choosing a Diagnostic Question 3.4 Other Representations of Strategy 3.4.1 Procedural Formulations 3.4.2 Control Rules and Metarules 3.4.3 Blackboard-based Control 3.4.4 Strategy as Decision Making 3.4.5 Summary of Alternate Representations4. The Ask Knowledge Acquisition Assistant 4.1 Functional Objectives for the Knowledge Acquisition Assistant 4.2 The Ask Knowledge Acquisition Procedure 4.2.1 Eliciting Feedback on System Performance 4.2.2 Eliciting the User's Preferences 4.2.3 Analyzing the Discrepancy with Existing Strategic Knowledge 4.2.4 Eliciting Justifications for Actions 4.2.5 Formulating New Strategy Rules 4.2.6 Generalizing Strategy Rules 4.2.7 Verifying Strategy Rules 4.3 The Elicitation of Justifications 4.3.1 The Justification Interface 4.3.2 Seeding the Justifications 4.3.3 The New Term Interface 4.4 Learning Strategy Rules 4.4.1 The Learning Task 4.4.2 Credit Assignment 4.4.3 Operationalization: Formulating a New Strategy Rule 4.4.4 Moving Tests into Generators 4.4.5 Generalization 1: Turning Constants into Variables 4.4.6 Generalization 2: Extending Reference 4.4.7 Generalizations: Dropping Conditions 4.5 Ask as a Computer Program 4.5.1 The Software Infrastructure 4.5.2 The User Interface5. Approaches to Knowledge Acquisition 5.1 Preliminaries 5.1.1 The Stages of Knowledge Acquisition 5.1.2 Three Paradigms of Research in Knowledge Acquisition 5.2 The Engineering Approach to Knowledge Acquisition 5.2.1 Improvements in the Analysis Stages of Knowledge Acquisition 5.2.2 Knowledge System Development Tools 5.2.3 Task-specific Architectures 5.3 Model-based Elicitation Tools 5.4 Machine Learning 5.4.1 Similarity-based Learning 5.4.2 Explanation-based Learning 5.4.3 Case-based Learning 5.5 Hybrid Approaches: Learning with Human Guidance 5.5.1 Similarity-based Induction of Control Knowledge with Human Guidance 5.5.2 Inducing Control Plans from Sequences of Actions 5.5.3 Integrating Knowledge Elicitation and Inductive Generalization 5.6 Conclusions Drawn from the Literature6. Experience with Ask 6.1 Acquiring Strategy from Physicians 6.1.1 Sessions with the Experts 6.1.2 The Initial Strategy Rules 6.1.3 Case 1: Building a Rule from Seeded Justifications 6.1.4 Case 2: Success at Defining a New Feature 6.1.5 Case 3: Failure - Producing an Overly-general Filter Rule 6.1.6 Analysis of the Chest Pain Domain 6.1.7 Application of the Chest Pain Strategy to a Plant Pathology Problem 6.2 Reacquiring the Complete MUM Strategy 6.2.1 MUM's Strategic Phase Planner 6.2.2 Representing a General Diagnostic Strategy in Hand-coded Rules 6.2.3 Reacquiring the General Strategy using ASK 6.3 Acquiring a Strategy for Controlling Fire-fighting Agents 6.3.1 Modeling the Reactions of Agents 6.4 Reimplementing the Neomycin Diagnostic Strategy7. Evaluation and Discussion 7.1 A Perspective on Evaluation 7.2 Scope of Applicability: Characteristics of Suitable Tasks 7.2.1 Actions can be Selected One at a Time 7.2.2 Actions can be Modeled as Primitive Steps 7.2.3 Local Action-selection Criteria Avoid Global Pitfalls 7.2.4 An Optimal Decision Among Actions is not Possible or not Necessary 7.2.5 Features can be Measured on Absolute Scales 7.3 Sources of Power in Ask 7.3.1 Interactive Acquisition in the Context of Performance 7.3.2 Model-based Knowledge Acquisition 7.3.3 Explicit, Declarative Representation of Control 7.3.4 Fine-grained Control and Local Decisions 7.3.5 Generalizability of the Representation 7.3.6 Reformulating Strategy as Classification 7.3.7 Integrating Machine Learning and Interactive Knowledge Acquisition 7.4 Assumptions and Limitations of Ask 7.4.1 Strategic Choices Contribute to Expert Performance 7.4.2 Experts'Justifications are a Valid Basis for Strategy 7.4.3 Substantive Knowledge must be Acquired and Correct 7.4.4 Dependence on Knowledge Engineers 7.4.5 Efficiency Concerns 7.4.6 Multiple Experts 7.5 A Summary8. Conclusions and Conjectures 8.1 The Acquirability Trade-off 8.2 Integrating Induction and Elicitation 8.3 Design for Knowledge Acquisition 8.4 Impact of Automated Knowledge Acquisition on Knowledge Engineering Practice 8.5 Explanation as a Medium for Knowledge AcquisitionAppendix 1: A Strategy for Prospective DiagnosisAppendix 2: Acquiring a Strategy from ScratchBibliographyIndex