Entity Information Life Cycle for Big Data

Entity Information Life Cycle for Big Data walks you through the ins and outs of managing entity information so you can successfully achieve master data management (MDM) in the era of big data. This book explains big data’s impact on MDM and the critical role of entity information management system (EIMS) in successful MDM. Expert authors Dr. John R. Talburt and Dr. Yinle Zhou provide a thorough background in the principles of managing the entity information life cycle and provide practical tips and techniques for implementing an EIMS, strategies for exploiting distributed processing to handle big data for EIMS, and examples from real applications. Additional material on the theory of EIIM and methods for assessing and evaluating EIMS performance also make this book appropriate for use as a textbook in courses on entity and identity management, data management, customer relationship management (CRM), and related topics.

• Explains the business value and impact of entity information management system (EIMS) and directly addresses the problem of EIMS design and operation, a critical issue organizations face when implementing MDM systems
• Offers practical guidance to help you design and build an EIM system that will successfully handle big data
• Details how to measure and evaluate entity integrity in MDM systems and explains the principles and processes that comprise EIM
• Provides an understanding of features and functions an EIM system should have that will assist in evaluating commercial EIM systems
• Includes chapter review questions, exercises, tips, and free downloads of demonstrations that use the OYSTER open source EIM system
• Executable code (Java .jar files), control scripts, and synthetic input data illustrate various aspects of CSRUD life cycle such as identity capture, identity update, and assertions

IT managers and software developers as well as graduate and undergraduate students in computer science, information science, business computer information systems, and management information systems

Chapter 1. The Value Proposition for MDM and Big Data

• Definition and Components of MDM
• The Business Case for MDM
• Dimensions of MDM
• The Challenge of Big Data
• MDM and Big Data – The N-Squared Problem
• Concluding Remarks

Chapter 2. Entity Identity Information and the CSRUD Life Cycle Model

• Entities and Entity References
• Managing Entity Identity Information
• Entity Identity Information Life Cycle Management Models
• Concluding Remarks

Chapter 3. A Deep Dive into the Capture Phase

• An Overview of the Capture Phase
• Building the Foundation
• Understanding the Data
• Data Preparation
• Selecting Identity Attributes
• Assessing ER Results
• Data Matching Strategies
• Concluding Remarks

Chapter 4. Store and Share – Entity Identity Structures

• Entity Identity Information Management Strategies
• Dedicated MDM Systems
• The Identity Knowledge Base
• MDM Architectures
• Concluding Remarks

Chapter 5. Update and Dispose Phases – Ongoing Data Stewardship

• Data Stewardship
• The Automated Update Process
• The Manual Update Process
• Asserted Resolution
• EIS Visualization Tools
• Managing Entity Identifiers
• Concluding Remarks

Chapter 6. Resolve and Retrieve Phase – Identity Resolution

• Identity Resolution
• Identity Resolution Access Modes
• Confidence Scores
• Concluding Remarks

Chapter 7. Theoretical Foundations

• The Fellegi-Sunter Theory of Record Linkage
• The Stanford Entity Resolution Framework
• Entity Identity Information Management
• Concluding Remarks

Chapter 8. The Nuts and Bolts of Entity Resolution

• The ER Checklist
• Cluster-to-Cluster Classification
• Selecting an Appropriate Algorithm
• Concluding Remarks

Chapter 9. Blocking

• Blocking
• Blocking by Match Key
• Dynamic Blocking versus Preresolution Blocking
• Blocking Precision and Recall
• Match Key Blocking for Boolean Rules
• Match Key Blocking for Scoring Rules
• Concluding Remarks

Chapter 10. CSRUD for Big Data

• Large-Scale ER for MDM
• The Transitive Closure Problem
• Distributed, Multiple-Index, Record-Based Resolution
• An Iterative, Nonrecursive Algorithm for Transitive Closure
• Iteration Phase: Successive Closure by Reference Identifier
• Deduplication Phase: Final Output of Components
• ER Using the Null Rule
• The Capture Phase and IKB
• The Identity Update Problem
• Persistent Entity Identifiers
• The Large Component and Big Entity Problems
• Identity Capture and Update for Attribute-Based Resolution
• Concluding Remarks

Chapter 11. ISO Data Quality Standards for Master Data

• Background
• Goals and Scope of the ISO 8000-110 Standard
• Four Major Components of the ISO 8000-110 Standard
• Simple and Strong Compliance with ISO 8000-110
• ISO 22745 Industrial Systems and Integration
• Beyond ISO 8000-110
• Concluding Remarks

Appendix A. Some Commonly Used ER Comparators