Optimal Estimation of Parameters

Advance praise: 'The minimum description length (MDL) principle is a very universal principle of statistical modeling in estimation, prediction, testing, and coding. Jorma Rissanen, the pioneer of the MDL principle, evolves a new theory to reach the most general and complete notion, which he calls the complete MDL principle. In this book the author derives it by introducing the key notion of maximum capacity. The most fundamental methods of estimation such as maximum likelihood estimation and the MDL estimation are naturally derived as the maximum capacity estimators, and their optimality is justified within a unifying theoretical framework. Through the book, readers can revisit the meaning of estimation from the author's very original viewpoint, and will enjoy the most advanced version of the MDL principle.' Kenji Yamanishi, University of Tokyo 'In this splendid new book, Jorma Rissanen, the originator of the minimum description length (MDL) principle, puts forward a comprehensive theory of estimation which differs in several ways from the standard Bayesian and frequentist approaches. During the development of MDL over the last 30 years, it gradually emerged that MDL could be viewed, informally, as a maximum probability principle that directly extends Fisher's classical maximum likelihood method to allow for estimation of a model's structural properties. Yet providing a formal link between MDL and maximum probability remained elusive until the arrival of this book. By making the connection mathematically precise, Rissanen now ties up the loose ends of MDL theory and at the same time develops a beautiful, unified, entirely original and fully coherent theory of estimation, which includes hypothesis testing as a special case.' Peter Grunwald, Centrum voor Wiskunde en Informatica, The Netherlands