On-line learning is one of the most powerful and commonly used techniques for training large layered networks and has been used successfully in many real-world applications. Traditional analytical methods have been recently complemented by ones from statistical physics and Bayesian statistics. This powerful combination of analytical methods provides more insight and deeper understanding of existing algorithms and leads to novel and principled proposals for their improvement. This book presents a coherent picture of the state-of-the-art in the theoretical analysis of on-line learning. An introduction relates the subject to other developments in neural networks and explains the overall picture. Surveys by leading experts in the field combine new and established material and enable non-experts to learn more about the techniques and methods used. This book, the first in the area, provides a comprehensive view of the subject and will be welcomed by mathematicians, scientists and engineers, whether in industry or academia.
Edited by:
David Saad (Aston University)
Imprint: Cambridge University Press
Country of Publication: United Kingdom
Volume: 17
Dimensions:
Height: 229mm,
Width: 152mm,
Spine: 23mm
Weight: 600g
ISBN: 9780521117913
ISBN 10: 0521117917
Series: Publications of the Newton Institute
Pages: 412
Publication Date: 30 July 2009
Audience:
Professional and scholarly
,
Professional and scholarly
,
Undergraduate
,
Undergraduate
Format: Paperback
Publisher's Status: Active
Foreword C. Bishop; 1. Introduction D. Saad; 2. On-line learning and stochastic approximations Leon Bottou; 3. Exact and perturbative solutions for the ensemble dynamics Todd Leen; 4. A statistical study of on-line learning Noboru Murata; 5. On-line learning in switching and drifting environments Klaus-Robert Mueller, Andreas Ziehe, Noboru Murata and Shun-ichi Amari; 6. Parameter adaptation in stochastic optimization Luis B. Almeida, Thibault Langlois, Jose D. Amaral and Alexander Plakhov; 7. Optimal on-line learning for multilayer neural networks David Saad and Magnus Rattray; 8. Universal asymptotics in committee machines with tree architecture Mauro Copelli and Nestor Caticha; 9. Incorporating curvature information in on-line learning Magnus Rattray and David Saad; 10. Annealed on-line learning in multilayer networks Siegfried Boes and Shun-ichi Amari; 11. On-line learning of prototypes and principal components Michael Biehl, Ansgar Freking, Matthias Hoelzer, Georg Reents and Enno Schloesser; 12. On-line learning with time-correlated patterns Tom Heskes and Wim Wiegerinck; 13. On-line learning from finite training sets David Barber and Peter Sollich; 14. Dynamics of supervised learning with restricted training sets Anthony C. C. Coolen and David Saad; 15. On-line learning of a decision boundary with and without queries Yoshiyuki Kabashima and Shigeru Shinomoto; 16. A Bayesian approach to on-line learning Manfred Opper; 17. Optimal perception learning: an on-line Bayesian approach Sara A. Solla and Ole Winther.
Reviews for On-Line Learning in Neural Networks
Review of the hardback: 'I recommend this book to readers with a theoretical, analytical, or mathematical interest in neural networks, especially online learning.' Computing Reviews I recommend this book to readers with a theoretical, analytical, or mathematical interest in neural networks, especially online learning. Computing Reviews The introduction gives a nice overview of on-line learning in neural networks and relates the subject to other developments in neural networks. The material provides a comprehensive view of the subject and is accessible to mathematicians, statisticians, and engineers in both industry and academia. Journal of the American Statistical Association
