COVID-19 Update: We are currently shipping orders daily. However, due to transit disruptions in some geographies, deliveries may be delayed. To provide all customers with timely access to content, we are offering 50% off Science and Technology Print & eBook bundle options. Terms & conditions.
Machine Learning and Data Science in the Oil and Gas Industry - 1st Edition - ISBN: 9780128207147

Machine Learning and Data Science in the Oil and Gas Industry

1st Edition

Best Practices, Tools, and Case Studies

Editor: Patrick Bangert
Paperback ISBN: 9780128207147
Imprint: Gulf Professional Publishing
Published Date: 1st March 2021
Page Count: 300
Sales tax will be calculated at check-out Price includes VAT/GST

Institutional Subscription

Secure Checkout

Personal information is secured with SSL technology.

Free Shipping

Free global shipping
No minimum order.


Machine Learning and Data Science in the Oil and Gas Industry explains how machine learning can be specifically tailored to oil and gas use cases. Petroleum engineers will learn when to use machine learning, how it is already used in oil and gas operations, and how to manage the data stream moving forward. Practical in its approach, the book explains all aspects of a data science or machine learning project, including the managerial parts of it that are so often the cause for failure. Several real-life case studies round out the book with topics such as predictive maintenance, soft sensing, and forecasting. Viewed as a guide book, this manual will lead a practitioner through the journey of a data science project in the oil and gas industry circumventing the pitfalls and articulating the business value.

Key Features

  • Chart an overview of the techniques and tools of machine learning including all the non-technological aspects necessary to be successful
  • Gain practical understanding of machine learning used in oil and gas operations through contributed case studies
  • Learn change management skills that will help gain confidence in pursuing the technology
  • Understand the workflow of a full-scale project and where machine learning benefits (and where it does not)


Oil and gas industry expert and practitioner working either in exploration, drilling, completions, engineering, production, maintenance or management

Table of Contents

1. Introduction

 1.1 Who this book is for
 1.2 Preview of the content
 1.3 Oil and gas industry overview
 1.4 Brief history of oil exploration
 1.5 Oil and gas as limited resources
 1.6 Challenges of oil and gas

2. Data Science, Statistics, and Time-Series

 2.1 Measurement, uncertainty, and record keeping
    2.1.1 Uncertainty
    2.1.2 Record keeping
 2.2 Correlation and timescales
 2.3 The idea of a model
 2.4 First principles models
 2.5 The straight line
 2.6 Representation and significance
 2.7 Outlier detection
 2.8 Residuals and statistical distributions
 2.9 Feature engineering
 2.10 Principal component analysis
 2.11 Practical advice

3. Machine Learning
 3.1 Basic ideas of machine learning
 3.2 Bias-variance complexity trade-off
 3.3 Model types
    3.3.1 Deep neural network
    3.3.2 Recurrent neural network or long short-term memory network
    3.3.3 Support vector machines
    3.3.4 Random forest or decision trees
    3.3.5 Self-organizing maps (SOM)
    3.3.6 Bayesian network and ontology
 3.4 Training and assessing a model
 3.5 How good is my model?
 3.6 Role of domain knowledge
 3.7 Optimization using a model
 3.8 Practical advice

4. Introduction to Machine Learning in the Oil and Gas Industry

 4.1 Forecasting
 4.2 Predictive maintenance
 4.3 Production
 4.4 Modeling physical relationships
 4.5 Optimization and advanced process control
 4.6 Other applications

5. Data Management from the DCS to the Historian

 5.1 Introduction
    5.1.1 Convergence of OT and IT
    5.1.2 A maturity model for OT/IT convergence
    5.1.3 Digital Oilfield 2.0 headed to the edge
 5.2 Sensor data
    5.2.1. There are problems with data from sensors:
 data quality challenges
    5.2.2. Validation, estimation, and editing (VEE)
 5.3. Time series data
 5.4. How sensor data is transmitted by field networks
    5.4.1. From Plant to Field: Communications Protocols (HART, Fieldbus, OPC, OPC-UA and Wireless Hart)
    5.4.2. Wireless SCADA radio
    5.4.3. Which protocol is best?
 5.5. How control systems manage data
    5.5.1. Cloud-based SCADA and web-based SCADA
 5.6. Historians and information servers as a data source
    5.6.1. What can you record in a data historian?
 5.7. Data visualization of time series data—HMI (human machine interface)
    5.7.1. Asset performance management systems (APM)
    5.7.2. Key elements of data management for asset performance management
 5.8. Data management for equipment and facilities
    5.8.1. What is a document management system?
 5.9. Simulators, process modeling, and operating training systems
 5.10. How to get data out of the field/plant and to your analytics platform
    5.10.1. Data visualization
    5.10.2. Data analytics
    5.10.3. Three historical stages of industrial analytics
 5.11. Conclusion: do you know if your data is correct?

6. Getting the Most Across the Value Chain

 6.1. Thinking outside the box
 6.2. Costing a project
 6.3. Valuing a project
    6.3.1. How to measure the benefit
    6.3.2. Measuring the benefit
 6.4. The business case
 6.5. Growing markets, optimizing networks
 6.6. Integrated strategy and alignment
 6.7. Case studies: capturing market opportunities
 6.8. Digital platform: partner, acquire, or build?
 6.9. What success looks like

7. Project Management for a Machine Learning Project

 7.1. Classical project management in oil & gas—a (short) primer
 7.2. Agile—the mindset
 7.3. Scrum—the framework
    7.3.1. Roles of scrum
    7.3.2. Events
    7.3.3. Artifacts
    7.3.4. Values
    7.3.5. How it works
 7.4. Project execution—from pilot to product
    7.4.1. Pilot setup
    7.4.2. Product owner
    7.4.3. Development team
    7.4.4. Scrum master
    7.4.5. Stakeholders
 7.5. Management of change and culture
 7.6. Scaling—from pilot to product
    7.6.1. Take advantage of a platform
    7.6.2. Establish a team and involve the assets
    7.6.3. Keep developing
    7.6.4. Involve UX expertise

8. The Business of AI Adoption

 8.1. Defining artificial intelligence
 8.2. AI impacts on oil and gas
    8.2.1. Upstream impacts
    8.2.2. Downstream impacts
    8.2.3. Production and midstream impacts
    8.2.4. New business models
 8.3. The adoption challenge
    8.3.1. The uncertainties of new technology
    8.3.2. AI in the field
 8.4. The problem of trust
    8.4.1 Work is evolving
    8.4.2. Driverless transportation
    8.4.3. Trust and the machine
    8.4.4. The human-smart machine trust gap
    8.4.5. Trusting a smart machine
    8.4.6. Trusting the smart machine developer
    8.4.7. Making it real
    8.4.8. Getting to trust
 8.5. Digital leaders lead
    8.5.1. Finding the digital leader
    8.5.2. Moving beyond trials and pilots
    8.5.3. The role of trials and pilots
    8.5.4. The economics of pilot projects
    8.5.5. Moving to enterprise deployment
 8.6. Overcoming barriers to scaling up
    8.6.1. The scale mismatch
    8.6.2. Supplier consolidation
    8.6.3. The corporate accelerator
    8.6.4. The oil company investor
 8.7. Confronting front line change
    8.7.1. The corporate parallels
    8.7.2. Early warning signs
 8.8. Doing digital change
    8.8.1. A typical change champion
    8.8.2. Organizational reaction to change

9. Global Practice of AI and Big Data in Oil and Gas Industry

 9.1. Introduction
 9.2. Integrate digital rock physics with AI to optimize oil recovery
    9.2.1. The upstream business
    9.2.2. Digital core technology
    9.2.3. Modeling wettability at the pore-scale
 9.3. The molecular level advance planning system for refining
    9.3.1. Prediction of crude oil mixing and molecular properties
    9.3.2. Scheduling optimization at the molecular leve
    9.3.3. Collaborative optimization of the entire industry chain
 9.4. The application of big data in the oil refining process
    9.4.1. Principle and methodology
    9.4.2. A case study of CCR process unit
 9.5. Equipment management based on AI
    9.5.1. Equipment hazard monitoring and warning
    9.5.2. Equipment fault recognition and diagnosis
    9.5.3. Equipment health status, residual life prediction and other management

10. Soft Sensors for NOx Emissions

 10.1. Introduction to soft sensing
 10.2. NOx and SOx emissions
 10.3. Combined heat and power (CHP)
 10.4. Soft sensing and machine learning
 10.5. Setting up a soft sensor
 10.6. Assessing the model
 10.7. Conclusion

11. Detecting Electric Submersible Pump Failures
 11.1. Introduction
 11.2. ESP data analytic
 11.3. Principal Component Analysis
 11.4. PCA diagnostic model
 11.5. Case study: diagnosis of the ESP broken shaft
    11.5.1. Selection of the ESP broken shaft variables
    11.5.2. Score of principle components
    11.5.3. Pump broken shaft identification
 11.6. Conclusions

12. Predictive and Diagnostic Maintenance for Rod Pumps

 12.1. Introduction
    12.1.1. Beam pumps
    12.1.2. Beam pump problems
    12.1.3. Problem statement
 12.2. Feature engineering
    12.2.1. Library-based methods
    12.2.2. Model-based methods
    12.2.3. Segment-based methods
    12.2.4. Other methods
    12.2.5. Selection of features
 12.3. Project method to validate our model
    12.3.1. Data collection
    12.3.2. Generation of training data
    12.3.3. Feature engineering
    12.3.4. Machine learning
    12.3.5. Summary of methodology
 12.4. Results
    12.4.1. Summary and review

13. Forecasting Slugging in Gas Lift Wells

 13.1. Introduction
 13.2. Methodology
 13.3. Focus projects
    13.3.1. Dashboarding landscape/architecture
 13.3.2. Slugging
 13.4. Data structure
 13.5. Outlook
 13.6. Conclusion
 Further reading



No. of pages:
© Gulf Professional Publishing 2021
1st March 2021
Gulf Professional Publishing
Paperback ISBN:

About the Editor

Patrick Bangert

Dr. Patrick Bangert is the Vice President of Artificial Intelligence at Samsung SDS where he leads both the AI software development and AI consulting groups that each provide various offerings to the industry. He is the founder and Board Chair of Algorithmica Technologies, providing real-time process modeling, optimization, and predictive maintenance solutions to the process industry with a focus on chemistry and power generation. His doctorate from UCL specialized in applied mathematics, and his academic positions at NASA’s Jet Propulsion Laboratory and Los Alamos National Laboratory made use of optimization and machine learning for magnetohydrodynamics and particle accelerator experiments. He has published extensively across optimization and machine learning and their relevant applications in the real world.

Affiliations and Expertise

Vice President of Artificial Intelligence at Samsung SDSA, San Jose, CA, United States, and Founder and Board Chair of Algorithmica Technologies GmbH, Bad Nauheim, Germany

Ratings and Reviews