Automating Open Source Intelligence

Automating Open Source Intelligence

Algorithms for OSINT

1st Edition - December 3, 2015

Write a review

  • Authors: Robert Layton, Paul Watters
  • Paperback ISBN: 9780128029169
  • eBook ISBN: 9780128029176

Purchase options

Purchase options
DRM-free (PDF, EPub, Mobi)
Sales tax will be calculated at check-out

Institutional Subscription

Free Global Shipping
No minimum order


Algorithms for Automating Open Source Intelligence (OSINT) presents information on the gathering of information and extraction of actionable intelligence from openly available sources, including news broadcasts, public repositories, and more recently, social media. As OSINT has applications in crime fighting, state-based intelligence, and social research, this book provides recent advances in text mining, web crawling, and other algorithms that have led to advances in methods that can largely automate this process. The book is beneficial to both practitioners and academic researchers, with discussions of the latest advances in applications, a coherent set of methods and processes for automating OSINT, and interdisciplinary perspectives on the key problems identified within each discipline. Drawing upon years of practical experience and using numerous examples, editors Robert Layton, Paul Watters, and a distinguished list of contributors discuss Evidence Accumulation Strategies for OSINT, Named Entity Resolution in Social Media, Analyzing Social Media Campaigns for Group Size Estimation, Surveys and qualitative techniques in OSINT, and Geospatial reasoning of open data.

Key Features

  • Presents a coherent set of methods and processes for automating OSINT
  • Focuses on algorithms and applications allowing the practitioner to get up and running quickly
  • Includes fully developed case studies on the digital underground and predicting crime through OSINT
  • Discusses the ethical considerations when using publicly available online data


Researchers in social media and computer security. Computer security professionals, digital investigators in law enforcement.

Table of Contents

    • List of Contributors
    • Chapter 1: The Automating of Open Source Intelligence
      • Abstract
      • The Commercial Angle
      • Algorithms
    • Chapter 2: Named Entity Resolution in Social Media
      • Abstract
      • Introduction
      • Discussion
    • Chapter 3: Relative Cyberattack Attribution
      • Abstract
      • Introduction
      • Basic Attack Structure
      • Anonymization on the Internet
      • Weaknesses in Anonymization
      • Attribution as a Concept
      • Absolute Attribution
      • Relative Attribution
      • Relative Attribution Concepts
      • Inherent Versus Learnt Behaviors
      • Hiding Behavior
      • Consistency of Behavior
      • Relative Attribution Techniques
      • Authorship Analysis
      • Limitations and Issues
      • Research Streams
      • Conclusions
    • Chapter 4: Enhancing Privacy to Defeat Open Source Intelligence
      • Abstract
      • Introduction
      • Requirements and Threats
      • Preliminaries
      • The PIEMCP
      • Formal Security Analysis with CPN
      • Performance Analysis of FSSO-PIEMC
      • Conclusion and Future Work
    • Chapter 5: Preventing Data Exfiltration: Corporate Patterns and Practices
      • Abstract
      • What is Happening Around the World?
      • What is Happening in New Zealand?
      • Specifying the Problem
      • Problems Arising by Implementing Censorship
      • So, What Should be Done?
      • Summary
    • Chapter 6: Gathering Intelligence on High-Risk Advertising and Film Piracy: A Study of the Digital Underground
      • Abstract
      • Introduction
      • Advertising and Risk
      • The Digital Millennium Copyright Act (DMCA)
      • Chilling Effects Database
      • Google Transparency Report
      • Mainstream Advertising and How Piracy is Funded
      • High-Risk Advertising and Their Links to Piracy Websites
      • High-Risk Advertising: Case Studies in Canada
      • High-Risk Advertising: Case Studies in Australia
      • High-Risk Advertising: Case Studies in New Zealand
      • Research Challenges
    • Chapter 7: Graph Creation and Analysis for Linking Actors: Application to Social Data
      • Abstract
      • Introduction
      • The Social Network Model
      • Graph Creation Techniques
      • Graph Analysis for OSINT
      • Twitter Case Study
      • Conclusion
    • Chapter 8: Ethical Considerations When Using Online Datasets for Research Purposes
      • Abstract
      • Introduction
      • Existing Guidelines
      • Interpretation of Existing Guidelines for Online Purposes
      • The Three Proposed Principles Applied to Online Research
      • Autonomy
      • Obtaining Consent
      • Benefits Against Risks
      • Justice
      • Summary
    • Chapter 9: The Limitations of Automating OSINT: Understanding the Question, Not the Answer
      • Abstract
      • Introduction
      • Finding Answers to Questions
      • Credibility and the Quality of Results
      • Relevance
      • The Limitations of Automating Osint
      • Conclusions
    • Chapter 10: Geospatial Reasoning With Open Data
      • Abstract
      • Introduction
      • The Open Geospatial Data Environment
      • Review of Reasoning Methods with Geospatial Data
      • Case Studies in Geospatial Reasoning
      • Conclusions
    • Subject Index

Product details

  • No. of pages: 222
  • Language: English
  • Copyright: © Syngress 2015
  • Published: December 3, 2015
  • Imprint: Syngress
  • Paperback ISBN: 9780128029169
  • eBook ISBN: 9780128029176

About the Authors

Robert Layton

Dr. Robert Layton is a Research Fellow at the Internet Commerce Security Laboratory (ICSL) at Federation University Australia. Dr Layton’s research focuses on attribution technologies on the internet, including automating open source intelligence (OSINT) and attack attribution. Dr Layton’s research has led to improvements in authorship analysis methods for unstructured text, providing indirect methods of linking profiles on social media.

Affiliations and Expertise

Research Fellow at the Internet Commerce Security Laboratory, Federation University Australia

Paul Watters

Paul A. Watters is a Professor of Information Technology at Massey University. He was previously Associate Professor of Information Security at the University of Ballarat, and co-founded the Cybercrime Research Laboratory at Macquarie University. His research interests are human factors in security and open source intelligence, and in measuring the risks associated with cybercrime, especially to children and young people. He is a Fellow of the British Computer Society and his work has been cited 1,249 times He has worked closely with government and industry on many projects, including Westpac, IBM, and the Australian Federal Police (AFP).

Affiliations and Expertise

Professor of Information Technology, Massey University, New Zealand

Ratings and Reviews

Write a review

There are currently no reviews for "Automating Open Source Intelligence"