Polopoly will be shut down December 15, 2023. Existing pages will have to be moved or will be removed at that date. Empolyees may read more at FAQ Polopoly Avveckling

MAI0019
Statistical Learning Theory with Concentration Inequalities/
Statistik inlärningsteori med koncentrationsolikheter

Number of credits: 15 hp

Examiner: Timo Koski

Course literature:

• O. Catoni (2004): Statistical Learning Theory and Stochastic Optimization, Lecture Notes in Mathematics 1851, Springer Verlag.
• V.N.Vapnik (1998): Statistical Learning Theory. chapters 14-16. John Wiley & Sons.
• M. Vidyasagar (2003): Learning and Generalization. Springer.
• Material on Support Vector Machines.

Course contents:

• The course investigates tools for analysis of performance of simplified models for prediction, estimation, and classification of complex data.
• The course starts with the theory and algorithms of support vector machines and the Vapnik-Chervonenkis theory.
• The techniques of analysis are rooted in information theory (minimax compression and learning, 'blowing-up lemma'), PAC-Bayesian theorems and concentration inequalities. Tools from probability are Bennett's, Hoeffding's, Chernoff's, Azuma's and McDiarmid's inequalities.
• Oracle inequalities, non-asymptotic bounds on the statistical risk, selfboundedness of Vapnik entropy and concentration inequalities for statistical learning (entropy method, logarithmic Sobolev inequalities) will be presented.

Organisation:

Examination: Presentations by participants. Home assignments.

Prerequisites:

• A graduate course in measure and integration theory (e.g., given by TM/MAI).
• A graduate course in Markov Chain Monte Carlo (e.g., given by mat.stat./MAI).
• A graduate course in probability and stochastic processes (e.g, given by mat.stat./MAI).
• A graduate course in statistical inference. (e.g., given by stat./MAI).
• An undergraduate/graduate course in information theory (e.g. given by Division of Data Transmission/ISY).

Page manager: karin.johansson@liu.se
Last updated: 2014-04-29