COVID-19 Update: We are currently shipping orders daily. However, due to transit disruptions in some geographies, deliveries may be delayed. To provide all customers with timely access to content, we are offering 50% off Science and Technology Print & eBook bundle options. Terms & conditions.
Managing Data in Motion - 1st Edition - ISBN: 9780123971678, 9780123977915

Managing Data in Motion

1st Edition

Data Integration Best Practice Techniques and Technologies

Author: April Reeve
Paperback ISBN: 9780123971678
eBook ISBN: 9780123977915
Imprint: Morgan Kaufmann
Published Date: 26th February 2013
Page Count: 204
Sales tax will be calculated at check-out Price includes VAT/GST
Price includes VAT/GST

Institutional Subscription

Secure Checkout

Personal information is secured with SSL technology.

Free Shipping

Free global shipping
No minimum order.

Table of Contents






What this book is about and why it’s necessary

What the reader will learn

Who should read this book

How this book is organized

Part 1: Introduction to data integration

Part 2: Batch data integration

Part 3: Real-time data integration

Part 4: Big data integration

Part 1: Introduction to Data Integration

Chapter 1. The Importance of Data Integration

The natural complexity of data interfaces

The rise of purchased vendor packages

Key enablement of big data and virtualization

Chapter 2. What Is Data Integration?

Data in motion

Integrating into a common format—transforming data

Migrating data from one system to another

Moving data around the organization

Pulling information from unstructured data

Moving process to data

Chapter 3. Types and Complexity of Data Integration

The differences and similarities in managing data in motion and persistent data

Batch data integration

Real-time data integration

Big data integration

Data virtualization

Chapter 4. The Process of Data Integration Development

The data integration development life cycle

Inclusion of business knowledge and expertise

Part 2: Batch Data Integration

Chapter 5. Introduction to Batch Data Integration

What is batch data integration?

Batch data integration life cycle

Chapter 6. Extract, Transform, and Load

What is ETL?




Access layers



Chapter 7. Data Warehousing

What is data warehousing?

Layers in an enterprise data warehouse architecture

Types of data to load in a data warehouse

Chapter 8. Data Conversion

What is data conversion?

Data conversion life cycle

Data conversion analysis

Best practice data loading

Improving source data quality

Mapping to target

Configuration data

Testing and dependencies

Private data



Chapter 9. Data Archiving

What is data archiving?

Selecting data to archive

Can the archived data be retrieved?

Conforming data structures in the archiving environment

Flexible data structures

Chapter 10. Batch Data Integration Architecture and Metadata

What is batch data integration architecture?

Profiling tool

Modeling tool

Metadata repository

Data movement



Part 3: Real Time Data Integration

Chapter 11. Introduction to Real-Time Data Integration

Why real-time data integration?

Why two sets of technologies?

Chapter 12. Data Integration Patterns

Interaction patterns

Loose coupling

Hub and spoke

Synchronous and asynchronous interaction

Request and reply

Publish and subscribe

Two-phase commit

Integrating interaction types

Chapter 13. Core Real-Time Data Integration Technologies

Confusing terminology

Enterprise service bus (ESB)

Service-oriented architecture (SOA)

Extensible markup language (XML)

Data replication and change data capture

Enterprise application integration (EAI)

Enterprise information integration (EII)

Chapter 14. Data Integration Modeling

Canonical modeling

Message modeling

Chapter 15. Master Data Management

Introduction to master data management

Reasons for a master data management solution

Purchased packages and master data

Reference data

Masters and slaves

External data

Master data management functionality

Types of master data management solutions—registry and data hub

Chapter 16. Data Warehousing with Real-Time Updates

Corporate information factory

Operational data store

Master data moving to the data warehouse

Chapter 17. Real-Time Data Integration Architecture and Metadata

What is real-time data integration metadata?



Metadata repository

Enterprise service bus—data transformation and orchestration

Data movement and middleware

External interaction

Part 4: Big, Cloud, Virtual Data

Chapter 18. Introduction to Big Data Integration

Data integration and unstructured data

Big data, cloud data, and data virtualization

Chapter 19. Cloud Architecture and Data Integration

Why is data integration important in the cloud?

Public cloud

Cloud security

Cloud latency

Cloud redundancy

Chapter 20. Data Virtualization

A technology whose time has come

Business uses of data virtualization

Data virtualization architecture

Chapter 21. Big Data Integration

What is big data?

Big data dimension—volume

Big data dimension—variety

Big data dimension—velocity

Traditional big data use cases

More big data use cases

Leveraging the power of big data—real-time decision support

Big data architecture

Chapter 22. Conclusion to Managing Data in Motion

Data integration architecture

Data integration engines

Data integration hubs

Metadata management

The end




Managing Data in Motion describes techniques that have been developed for significantly reducing the complexity of managing system interfaces and enabling scalable architectures. Author April Reeve brings over two decades of experience to present a vendor-neutral approach to moving data between computing environments and systems. Readers will learn the techniques, technologies, and best practices for managing the passage of data between computer systems and integrating disparate data together in an enterprise environment.

The average enterprise's computing environment is comprised of hundreds to thousands computer systems that have been built, purchased, and acquired over time. The data from these various systems needs to be integrated for reporting and analysis, shared for business transaction processing, and converted from one format to another when old systems are replaced and new systems are acquired.

The management of the "data in motion" in organizations is rapidly becoming one of the biggest concerns for business and IT management. Data warehousing and conversion, real-time data integration, and cloud and "big data" applications are just a few of the challenges facing organizations and businesses today. Managing Data in Motion tackles these and other topics in a style easily understood by business and IT managers as well as programmers and architects.

Key Features

  • Presents a vendor-neutral overview of the different technologies and techniques for moving data between computer systems including the emerging solutions for unstructured as well as structured data types
  • Explains, in non-technical terms, the architecture and components required to perform data integration
  • Describes how to reduce the complexity of managing system interfaces and enable a scalable data architecture that can handle the dimensions of "Big Data"


Data Warehouse Professionals; Data Modelers and Architects; Database and Network Administrators; ETL and Application Programmers; Project Managers; IT and Data Center Managers; CIO/CTO


No. of pages:
© Morgan Kaufmann 2013
26th February 2013
Morgan Kaufmann
Paperback ISBN:
eBook ISBN:


"The highlight of the book is that the author is able to present a broad, complicated subject in a coherent, consolidated, and readable manner. This is ideal for busy information technology managers and chief technical officers who have little time for outside reading. The book is especially valuable for those managers who are not familiar with modern data management or who are fluent in general computing technology but are tasked with managing data on a larger scale.", November 4, 2013
"Reeve, an enterprise information consultant with EMC, describes different techniques, technologies, and best practices for managing the transfer of data between computer systems and integrating disparate databases together within a large organization."--Reference and Research Book News, August 2013
"Few if any enterprises have the luxury of a single unified integrated data platform. Yet one of the least considered areas in enterprise information management is how we should treat and manage the growing numbers of interfaces. April Reeve presents a much-needed overview and guide to the challenges of data integration."--John Ladley, Principal of IMCue Solutions, Editor of the Data Strategy Journal

Ratings and Reviews

About the Author

April Reeve

April Reeve

April Reeve has spent the last 25 years working as an enterprise architect and program manager, primarily for large financial services firms. Currently she is working for EMC Consulting as a Business Consultant in the Enterprise Information Management practice. April is an expert in multiple Data Management disciplines including Data Conversion, Data Warehousing, Business Intelligence, Master Data Management, Data Integration, and Data Governance. She is a regular speaker on Data Management topics at Industry conferences.

Affiliations and Expertise

April Reeve is a Business Consultant in the Enterprise Information Management practice at EMC Consulting