Partially Observed Markov Decision Processes: Filtering, Learning and Controlled Sensing

— —

Vikram Krishnamurthy (Cornell University, New York)

Partially Observed Markov Decision Processes

Filtering, Learning and Controlled Sensing

Vikram Krishnamurthy (Cornell University, New York)

$192.95

Hardback

Forthcoming
Pre-Order now

QTY:

English

Cambridge University Press
05 June 2025

Information theory; Probability & statistics; Stochastics; Computer networking & communications; Pattern recognition

Covering formulation, algorithms and structural results and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. In light of major advances in machine learning over the past decade, this edition includes a new Part V on inverse reinforcement learning as well as a new chapter on non-parametric Bayesian inference (for Dirichlet processes and Gaussian processes), variational Bayes and conformal prediction.

By: Vikram Krishnamurthy (Cornell University New York)
Imprint: Cambridge University Press
Country of Publication: United Kingdom
Edition: 2nd Revised edition
ISBN: 9781009449434
ISBN 10: 1009449435
Pages: 651
Publication Date: 05 June 2025
Audience: Professional and scholarly , Undergraduate
Format: Hardback
Publisher's Status: Forthcoming

Vikram Krishnamurthy is Professor of Electrical and Computer Engineering at Cornell University. From 2002 to 2016, he was Professor and Senior Canada Research Chair in Statistical Signal Processing at the University of British Columbia. His research contributions are in statistical signal processing, stochastic optimization and control, with applications in social networks, adaptive radar systems and biological ion channels. He is a Fellow of IEEE and served as Distinguished Lecturer for the IEEE Signal Processing Society and Editor-in-Chief of IEEE Journal of Selected Topics in Signal Processing. He was awarded an honorary doctorate from the Royal Institute of Technology (KTH) Sweden in 2014.

Reviews for Partially Observed Markov Decision Processes: Filtering, Learning and Controlled Sensing

'This book uniquely offers a comprehensive treatment of structural results for Partially Observable Markov Decision Processes (POMDPs), utilizing submodularity and stochastic orders. The new edition expands its scope by introducing essential results in nonparametric Bayes, stochastic optimization, and inverse reinforcement learning, making it an invaluable resource as both a textbook and reference.' Bo Wahlberg, KTH Royal Institute of Technology, Sweden 'This book is a tour-de-force on POMDPs and controlled sensing, featuring insightful treatment of foundational concepts in optimal filtering, stochastic control, and stochastic optimization. The new edition introduces innovative methods for detecting cognitive sensors through inverse reinforcement learning from a microeconomic perspective-critical for radar systems, signal processing, and control researchers.' Muralidhar Rangaswamy, Air Force Research Laboratory, U.S. 'An outstanding advanced graduate-level introduction to the increasingly important topic of partially observed Markov decision processes. The book is a delight to read - comprehensive, clear, up-to-date and insightful while preserving rigor. An essential resource for both researchers seeking to further advance the field and practitioners wishing to implement stochastic control in real engineering systems.' Rob Evans, University of Melbourne