COVID-19 Update: We are currently shipping orders daily. However, due to transit disruptions in some geographies, deliveries may be delayed. To provide all customers with timely access to content, we are offering 50% off Science and Technology Print & eBook bundle options. Terms & conditions.
Machine Learning - 2nd Edition - ISBN: 9780128188033, 9780128188040

Machine Learning

2nd Edition

A Bayesian and Optimization Perspective

Author: Sergios Theodoridis
Hardcover ISBN: 9780128188033
eBook ISBN: 9780128188040
Imprint: Academic Press
Published Date: 6th March 2020
Page Count: 1160
Sales tax will be calculated at check-out Price includes VAT/GST
Price includes VAT/GST

Institutional Subscription

Secure Checkout

Personal information is secured with SSL technology.

Free Shipping

Free global shipping
No minimum order.


Machine Learning: A Bayesian and Optimization Perspective, 2nd edition, gives a unified perspective on machine learning by covering both pillars of supervised learning, namely regression and classification. The book starts with the basics, including mean square, least squares and maximum likelihood methods, ridge regression, Bayesian decision theory classification, logistic regression, and decision trees. It then progresses to more recent techniques, covering sparse modelling methods, learning in reproducing kernel Hilbert spaces and support vector machines, Bayesian inference with a focus on the EM algorithm and its approximate inference variational versions, Monte Carlo methods, probabilistic graphical models focusing on Bayesian networks, hidden Markov models and particle filtering. Dimensionality reduction and latent variables modelling are also considered in depth.

This palette of techniques concludes with an extended chapter on neural networks and deep learning architectures. The book also covers the fundamentals of statistical parameter estimation, Wiener and Kalman filtering, convexity and convex optimization, including a chapter on stochastic approximation and the gradient descent family of algorithms, presenting related online learning techniques as well as concepts and algorithmic versions for distributed optimization.

Focusing on the physical reasoning behind the mathematics, without sacrificing rigor, all the various methods and techniques are explained in depth, supported by examples and problems, giving an invaluable resource to the student and researcher for understanding and applying machine learning concepts. Most of the chapters include typical case studies and computer exercises, both in MATLAB and Python.

The chapters are written to be as self-contained as possible, making the text suitable for different courses: pattern recognition, statistical/adaptive signal processing, statistical/Bayesian learning, as well as courses on sparse modeling, deep learning, and probabilistic graphical models.

New to this edition:

  • Complete re-write of the chapter on Neural Networks and Deep Learning to reflect the latest advances since the 1st edition. The chapter, starting from the basic perceptron and feed-forward neural networks concepts, now presents an in depth treatment of deep networks, including recent optimization algorithms, batch normalization, regularization techniques such as the dropout method, convolutional neural networks, recurrent neural networks, attention mechanisms, adversarial examples and training, capsule networks and generative architectures, such as restricted Boltzman machines (RBMs), variational autoencoders and generative adversarial networks (GANs).
  • Expanded treatment of Bayesian learning to include nonparametric Bayesian methods, with a focus on the Chinese restaurant and the Indian buffet processes.

Key Features

  • Presents the physical reasoning, mathematical modeling and algorithmic implementation of each method
  • Updates on the latest trends, including sparsity, convex analysis and optimization, online distributed algorithms, learning in RKH spaces, Bayesian inference, graphical and hidden Markov models, particle filtering, deep learning, dictionary learning and latent variables modeling
  • Provides case studies on a variety of topics, including protein folding prediction, optical character recognition, text authorship identification, fMRI data analysis, change point detection, hyperspectral image unmixing, target localization, and more


Researchers and graduate students in electronic engineering, mechanical engineering, computer science, applied mathematics, statistics, medical imaging

Table of Contents

1. Introduction
2. Probability and stochastic Processes
3. Learning in parametric Modeling: Basic Concepts and Directions
4. Mean-Square Error Linear Estimation
5. Stochastic Gradient Descent: the LMS Algorithm and its Family
6. The Least-Squares Family
7. Classification: A Tour of the Classics
8. Parameter Learning: A Convex Analytic Path
9. Sparsity-Aware Learning: Concepts and Theoretical Foundations
10. Sparsity-Aware Learning: Algorithms and Applications
11. Learning in Reproducing Kernel Hilbert Spaces
12. Bayesian Learning: Inference and the EM Algorithm
13. Bayesian Learning: Approximate Inference and nonparametric Models
14. Montel Carlo Methods
15. Probabilistic Graphical Models: Part 1
16. Probabilistic Graphical Models: Part 2
17. Particle Filtering
18. Neural Networks and Deep Learning
19. Dimensionality Reduction and Latent Variables Modeling


No. of pages:
© Academic Press 2020
6th March 2020
Academic Press
Hardcover ISBN:
eBook ISBN:

About the Author

Sergios Theodoridis

Sergios Theodoridis

Sergios Theodoridis is professor of machine learning and signal processing with the National and Kapodistrian University of Athens, Athens, Greece and with the Chinese University of Hong Kong, Shenzhen, China. He has received a number of prestigious awards, including the 2014 IEEE Signal Processing Magazine Best Paper Award, the 2009 IEEE Computational Intelligence Society Transactions on Neural Networks Outstanding Paper Award, the 2017 European Association for Signal Processing (EURASIP) Athanasios Papoulis Award, the 2014 IEEE Signal Processing Society Education Award, and the 2014 EURASIP Meritorious Service Award. He has served as president of EURASIP and vice president for the IEEE Signal Processing Society and as Editor-in-Chief IEEE Transactions on Signal processing. He is a Fellow of EURASIP and a Life Fellow of IEEE. He is the coauthor of the best selling book Pattern Recognition, 4th edition, Academic Press, 2009 and of the book Introduction to Pattern Recognition: A MATLAB Approach, Academic Press, 2010.

Affiliations and Expertise

National and Kapodistrian University of Athens, Greece, and Chinese University of Hong Kong, Shenzhen, China.


Reviews of the previous edition:
"Overall, this text is well organized and full of details suitable for advanced graduate and postgraduate courses, as well as scholars..." --Computing Reviews

"Machine Learning: A Bayesian and Optimization Perspective, Academic Press, 2105, by Sergios Theodoridis is a wonderful book, up to date and rich in detail. It covers a broad selection of topics ranging from classical regression and classification techniques to more recent ones including sparse modeling, convex optimization, Bayesian learning, graphical models and neural networks, giving it a very modern feel and making it highly relevant in the deep learning era. While other widely used machine learning textbooks tend to sacrifice clarity for elegance, Professor Theodoridis provides you with enough detail and insights to understand the "fine print". This makes the book indispensable for the active machine learner." --Prof. Lars Kai Hansen, DTU Compute - Dept. Applied Mathematics and Computer Science Technical University of Denmark

"Before the publication of Machine Learning: A Bayesian and Optimization Perspective, I had the opportunity to review one of the chapters in the book (on Monte Carlo methods). I have published actively in this area, and so I was curious how S. Theodoridis would write about it. I was utterly impressed. The chapter presented the material with an optimal mix of theoretical and practical contents in very clear manner and with information for a wide range of readers, from newcomers to more advanced readers. This raised my curiosity to read the rest of the book once it was published. I did it and my original impressions were further reinforced. S. Theodoridis has a great capability to disentangle the important from the unimportant and to make the most of the used space for writing. His text is rich with insights about the addressed topics that are not only helpful for novices but also for seasoned researchers. It goes without saying that my department adopted his book as a textbook in the course on machine learning." --Petar M. Djuric, Ph.D. SUNY Distinguished Professor Department of Electrical and Computer Engineering Stony Brook University, Stony Brook, USA

"As someone who has taught graduate courses in pattern recognition for over 35 years, I have always looked for a rigorous book that is current and appealing to students with widely varying backgrounds. The book on Machine Learning by Sergios Theodoridis has struck the perfect balance in explaining the key (traditional and new) concepts in machine learning in a way that can be appreciated by undergraduate and graduate students as well as practicing engineers and scientists. The chapters have been written in a self-consistent way, which will help instructors to assemble different sections of the book to suit the background of students" --Rama Cellappa, Distinguished University Professor, Minta Martin Professor of Engineering, Chair, Department of Electrical and Computer Engineering, University of Maryland, USA

Ratings and Reviews