Group and Crowd Behavior for Computer Vision

Group and Crowd Behavior for Computer Vision provides a multidisciplinary perspective on how to solve the problem of group and crowd analysis and modeling, combining insights from the social sciences with technological ideas in computer vision and pattern recognition.

The book answers many unresolved issues in group and crowd behavior, with Part One providing an introduction to the problems of analyzing groups and crowds that stresses that they should not be considered as completely diverse entities, but as an aggregation of people.

Part Two focuses on features and representations with the aim of recognizing the presence of groups and crowds in image and video data. It discusses low level processing methods to individuate when and where a group or crowd is placed in the scene, spanning from the use of people detectors toward more ad-hoc strategies to individuate group and crowd formations.

Part Three discusses methods for analyzing the behavior of groups and the crowd once they have been detected, showing how to extract semantic information, predicting/tracking the movement of a group, the formation or disaggregation of a group/crowd and the identification of different kinds of groups/crowds depending on their behavior.

The final section focuses on identifying and promoting datasets for group/crowd analysis and modeling, presenting and discussing metrics for evaluating the pros and cons of the various models and methods. This book gives computer vision researcher techniques for segmentation and grouping, tracking and reasoning for solving group and crowd modeling and analysis, as well as more general problems in computer vision and machine learning.

• Presents the first book to cover the topic of modeling and analysis of groups in computer vision
• Discusses the topics of group and crowd modeling from a cross-disciplinary perspective, using social science anthropological theories translated into computer vision algorithms
• Focuses on group and crowd analysis metrics
• Discusses real industrial systems dealing with the problem of analyzing groups and crowds

Computer scientists and electronic researchers in computer vision and pattern recognition; graduate students in these fields

Chapter 1: The Group and Crowd Analysis Interdisciplinary Challenge

• Abstract
• 1.1. The Study of Groups and Crowds
• 1.2. Scope of the Book
• 1.3. Summary of Important Points
• References

Part 1: Features and Representations

Chapter 2: Social Interaction in Temporary Gatherings

• Abstract
• 2.1. Introduction: Group and Crowd Behavior in Context
• 2.2. Social Interaction: A Typology and Some Definitions
• 2.3. Temporary Gatherings: A Taxonomy and Some Examples
• 2.4. Conclusion: Microsociology Applied to Computer Vision
• 2.5. Further Reading
• References

Chapter 3: Group Detection and Tracking Using Sociological Features

• Abstract
• 3.1. Introduction
• 3.2. State-of-the-Art
• 3.3. Sociological Features
• 3.4. Detection Models
• 3.5. Group Tracking
• 3.6. Experiments
• 3.7. Discussion
• 3.8. Conclusions
• References

Chapter 4: Exploring Multitask and Transfer Learning Algorithms for Head Pose Estimation in Dynamic Multiview Scenarios

• Abstract
• 4.1. Introduction
• 4.2. Related Work
• 4.3. TL and MTL for Multiview Head Pose Estimation
• 4.4. Conclusions
• References

Chapter 5: The Analysis of High Density Crowds in Videos

• Abstract
• 5.1. Introduction
• 5.2. Literature Review
• 5.3. Data-Driven Crowd Analysis in Videos
• 5.4. Density-Aware Person Detection and Tracking in Crowds
• 5.5. CrowdNet: Learning a Representation for High Density Crowds in Videos
• 5.6. Conclusions and Directions for Future Research
• References

Chapter 6: Tracking Millions of Humans in Crowded Spaces

• Abstract
• 6.1. Introduction
• 6.2. Related Work
• 6.3. System Overview
• 6.4. Human Detection in 3D
• 6.5. Tracklet Generation
• 6.6. Tracklet Association
• 6.7. Experiments
• 6.8. Conclusions
• References

Chapter 7: Subject-Centric Group Feature for Person Reidentification

• Abstract
• Acknowledgments
• 7.1. Introduction
• 7.2. Related Works
• 7.3. Methodology
• 7.4. Results
• 7.5. Conclusion
• References

Part 2: Group and Crowd Behavior Modeling

Chapter 8: From Groups to Leaders and Back

• Abstract
• 8.1. Introduction
• 8.2. Modeling and Observing Groups and Their Leaders in Literature
• 8.3. Technical Preliminaries and Structured Output Prediction
• 8.4. The Tools of the Trade in Social and Structured Crowd Analysis
• 8.5. Results on Visual Localization of Groups and Leaders
• 8.6. The Predictive Power of Leaders in Social Groups
• 8.7. Conclusion
• References

Chapter 9: Learning to Predict Human Behavior in Crowded Scenes

• Abstract
• 9.1. Introduction
• 9.2. Related Work
• 9.3. Forecasting with Social Forces Model
• 9.4. Forecasting with Recurrent Neural Network
• 9.5. Experiments
• 9.6. Conclusions
• References

Chapter 10: Deep Learning for Scene-Independent Crowd Analysis

• Abstract
• 10.1. Introduction
• 10.2. Large Scale Crowd Datasets
• 10.3. Crowd Counting and Density Estimation
• 10.4. Attributes for Crowded Scene Understanding
• 10.5. Conclusion
• References

Chapter 11: Physics-Inspired Models for Detecting Abnormal Behaviors in Crowded Scenes

• Abstract
• 11.1. Introduction
• 11.2. Crowd Anomaly Detection: A General Review
• 11.3. Physics-Inspired Crowd Models
• 11.4. Violence Detection
• 11.5. Experimental Results
• 11.6. Conclusions
• References

Chapter 12: Activity Forecasting

• Abstract
• 12.1. Introduction
• 12.2. Overview
• 12.3. Activity Forecasting as Optimal Control
• 12.4. Single Agent Trajectory Forecasting in Static Environment
• 12.5. Multiagent Trajectory Forecasting
• 12.6. Dual-Agent Interaction Forecasting
• 12.7. Final Remarks
• References

Part 3: Metrics, Benchmarks and Systems

Chapter 13: Integrating Computer Vision Algorithms and Ontologies for Spectator Crowd Behavior Analysis

• Abstract
• Acknowledgments
• 13.1. Introduction
• 13.2. Computer Vision and Ontology
• 13.3. An Extension of the dolce Ontology for Spectator Crowd
• 13.4. Reasoning on the Temporal Alignment of Stands and Playground
• 13.5. Concluding Remarks
• References

Chapter 14: SALSA: A Multimodal Dataset for the Automated Analysis of Free-Standing Social Interactions

• Abstract
• 14.1. Introduction
• 14.2. Literature Review
• 14.3. Spotting the Research Gap
• 14.4. The SALSA Dataset
• 14.5. Experiments on SALSA
• 14.6. Conclusions and Future Work
• References

Chapter 15: Zero-Shot Crowd Behavior Recognition

• Abstract
• 15.1. Introduction
• 15.2. Related Work
• 15.3. Methodology
• 15.4. Experiments
• 15.5. Further Analysis
• 15.6. Conclusions
• References

Chapter 16: The GRODE Metrics

• Abstract
• 16.1. Introduction
• 16.2. Metrics in the Literature
• 16.3. The GRODE Metrics
• 16.4. Experiments
• 16.5. Conclusions
• References

Chapter 17: Realtime Pedestrian Tracking and Prediction in Dense Crowds

• Abstract
• Acknowledgments
• 17.1. Introduction
• 17.2. Related Work
• 17.3. Pedestrian State
• 17.4. Mixture Motion Model
• 17.5. Realtime Pedestrian Path Prediction
• 17.6. Implementation and Results
• 17.7. Conclusion
• References