Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management introduces information technology professionals to the basic concepts of logging and log management. It provides tools and techniques to analyze log data and detect malicious activity. The book consists of 22 chapters that cover the basics of log data; log data sources; log storage technologies; a case study on how syslog-ng is deployed in a real environment for log collection; covert logging; planning and preparing for the analysis log data; simple analysis techniques; and tools and techniques for reviewing logs for potential problems. The book also discusses statistical analysis; log data mining; visualizing log data; logging laws and logging mistakes; open source and commercial toolsets for log data collection and analysis; log management procedures; and attacks against logging systems. In addition, the book addresses logging for programmers; logging and compliance with regulations and policies; planning for log analysis system deployment; cloud logging; and the future of log standards, logging, and log analysis. This book was written for anyone interested in learning more about logging and log management. These include systems administrators, junior security engineers, application developers, and managers.
- Comprehensive coverage of log management including analysis, visualization, reporting and more
- Includes information on different uses for logs -- from system operations to regulatory compliance
- Features case Studies on syslog-ng and actual real-world situations where logs came in handy in incident response
- Provides practical guidance in the areas of report, log analysis system selection, planning a log analysis system and log data normalization and correlation
Computer Security staff and program managers; system, network, and application administrators; computer security incident response teams; and others who are responsible for performing duties related to computer security log management.
Dr. Anton A. Chuvakin
Kevin J. Schmidt
About the Authors
About the Technical Editor
Organization of the Book
Chapter 5: Case Study: syslog-ng
Chapter 6: Covert logging
Chapter 7: Analysis Goals, Planning and Preparation: What Are We Looking for?
Chapter 8: Simple Analysis Techniques
Chapter 9: Filtering, Matching and Correlation
Chapter 10: Statistical Analysis
Chapter 11: Log Data Mining
Chapter 12: Reporting and Summarization
Chapter 13: Visualizing Log Data
Chapter 14: Logging Laws and Logging Mistakes
Chapter 15: Tools for Log Analysis and Collection
Chapter 16: Log Management Procedures: Escalation, Response
Chapter 17: Attacks Against Logging Systems
Chapter 18: Logging for Programmers
Chapter 19: Logs and Compliance
Chapter 20: Planning Your Own Log Analysis System
Chapter 21: Cloud Logging
Chapter 22: Log Standard and Future Trends
Chapter 1. Logs, Trees, Forest: The Big Picture
Log Data Basics
A Look at Things to Come
Logs Are Underrated
Logs Can Be Useful
People, Process, Technology
Security Information and Event Management (SIEM)
Chapter 2. What is a Log?
Logs? What logs?
Criteria of Good Logging
Chapter 3. Log Data Sources
Log Source Classification
Chapter 4. Log Storage Technologies
Log Retention Policy
Log Storage Formats
Database Storage of Log Data
Hadoop Log Storage
The Cloud and Hadoop
Log Data Retrieval and Archiving
Chapter 5. syslog-ng Case Study
What Is syslog-ngsyslog-ng?
Chapter 6. Covert Logging
Complete Stealthy Log Setup
Logging in Honeypots
Covert Channels for Logging Brief
Chapter 7. Analysis Goals, Planning, and Preparation: What Are We Looking for?
Chapter 8. Simple Analysis Techniques
Line by Line: Road to Despair
Simple Log Viewers
Limitations of Manual Log Review
Responding to the Results of Analysis
Chapter 9. Filtering, Normalization, and Correlation
Common Patterns to Look For
Chapter 10. Statistical Analysis
Combining Statistical Analysis with Rules-based Correlation
Chapter 11. Log Data Mining
Data Mining Intro
Log Mining Intro
Log Mining Requirements
What We Mine For?
Deeper into Interesting
Chapter 12. Reporting and Summarization
Defining the Best Reports
Network Activity Reports
Resource Access Reports
Malware Activity Reports
Critical Errors and Failures Reports
Chapter 13. Visualizing Log Data
Log Data Constellations
Traditional Log Data Graphing
Chapter 14. Logging Laws and Logging Mistakes
Chapter 15. Tools for Log Analysis and Collection
Outsource, Build, or Buy
Basic Tools for Log Analysis
Utilities for Centralizing Log Information
Log Analysis Tools—Beyond the Basics
Chapter 16. Log Management Procedures: Log Review, Response, and Escalation
Assumptions, Requirements, and Precautions
Common Roles and Responsibilities
PCI and Log Data
Review, Response, and Escalation Procedures and Workflows
Validation of Log Review
Logbook—Evidence of Exception of Investigations
PCI Compliance Evidence Package
Periodic Operational Tasks
Chapter 17. Attacks Against Logging Systems
Chapter 18. Logging for Programmers
Roles and Responsibilities
Logging for Programmers
Chapter 19. Logs and Compliance
Chapter 20. Planning Your Own Log Analysis System
Chapter 21. Cloud Logging
Regulatory, Compliance, and Security Issues
Big Data in the Cloud
SIEM in the Cloud
Pros and Cons of Cloud Logging
Cloud Logging Provider Inventory
Chapter 22. Log Standards and Future Trends
Extrapolations of Today to the Future
Log Future and Standards
- No. of pages:
- © Syngress 2013
- 29th November 2012
- eBook ISBN:
- Paperback ISBN:
Dr. Anton Chuvakin is a recognized security expert in the field of log
management and PCI DSS compliance. He is an author of the books "Security Warrior" and "PCI
Compliance" and has contributed to many others, while also publishing dozens of papers on
log management, correlation, data analysis, PCI DSS, and security management. His blog
(http://www.securitywarrior.org) is one of the most popular in the industry.
Additionaly, Anton teaches classes and presents at many security conferences across the world
and he works on emerging security standards and serves on the advisory boards of
several security start-ups. Currently, Anton is developing his security consulting practice,
focusing on logging and PCI DSS compliance for security vendors and Fortune 500 organizations.
Anton earned his Ph.D. from Stony Brook University.
is a recognized security expert in the field of log management and PCI DSS compliance.
Kevin J. Schmidt is a senior manager at Dell SecureWorks, Inc., an industry leading MSSP, which is part of Dell. He is responsible for the design and development of a major part of the company’s SIEM platform. This includes data acquisition, correlation and analysis of log data.
Prior to SecureWorks, Kevin worked for Reflex Security where he worked on an IPS engine and anti-virus software. And prior to this he was a lead developer and architect at GuardedNet, Inc.,which built one of the industry’s first SIEM platforms. Kevin is also a commissioned officer in the United States Navy Reserve (USNR).
Kevin has over 19 years of experience in software development and design, 11 of which have been in the network security space. He holds a B.Sc. in computer science.
is a team lead and senior software developer at SecureWorks, Inc.
Christopher Phillips is a manager and senior software developer at Dell SecureWorks, Inc. He is responsible for the design and development of the company's Threat Intelligence service platform. He also has responsibility for a team involved in integrating log and event information from many third party providers for customers to have their information analyzed by the Dell SecureWorks systems and security professionals. Prior to Dell SecureWorks, Christopher has worked for McKesson and Allscripts where he worked with clients on HIPAA compliance and security and integrating healthcare systems. Christopher has over 18 years of experience in software development and design. He holds a Bachelors of Science in Computer Science and an MBA.
Christopher Phillips is a manager and senior software developer at Dell SecureWorks, Inc.
"The authors provide a way to simplify the complex process of analyzing large quantities of varied logs. The log management and log analysis approaches they recommend are addressed in detail."--Reference and Research Book News, August 2013
"…Anton Chuvakin and his co-authors Kevin Schmidt and Christopher Phillips bring significant real-world experience to the reader and an important book on the topic....For those that want to find the gold in their logs…[it] is a great resource that shows how to maximize the gold that often lays hidden in your large stores of log data."--RSA Conference, December 2012