MAI0019
Statistical Learning Theory with Concentration Inequalities/
Statistik inlärningsteori med koncentrationsolikheter

Number of credits: 15 hp

Examiner: Timo Koski

Course literature:

• O. Catoni (2004): Statistical Learning Theory and Stochastic Optimization, Lecture Notes in Mathematics 1851, Springer Verlag.
• V.N.Vapnik (1998): Statistical Learning Theory. chapters 14-16. John Wiley & Sons.
• M. Vidyasagar (2003): Learning and Generalization. Springer.
• Material on Support Vector Machines.

Course contents:

• The course investigates tools for analysis of performance of simplified models for prediction, estimation, and classification of complex data.
• The course starts with the theory and algorithms of support vector machines and the Vapnik-Chervonenkis theory.
• The techniques of analysis are rooted in information theory (minimax compression and learning, 'blowing-up lemma'), PAC-Bayesian theorems and concentration inequalities. Tools from probability are Bennett's, Hoeffding's, Chernoff's, Azuma's and McDiarmid's inequalities.
• Oracle inequalities, non-asymptotic bounds on the statistical risk, selfboundedness of Vapnik entropy and concentration inequalities for statistical learning (entropy method, logarithmic Sobolev inequalities) will be presented.

Organisation:

Examination: Presentations by participants. Home assignments.

Prerequisites:

• A graduate course in measure and integration theory (e.g., given by TM/MAI).
• A graduate course in Markov Chain Monte Carlo (e.g., given by mat.stat./MAI).
• A graduate course in probability and stochastic processes (e.g, given by mat.stat./MAI).
• A graduate course in statistical inference. (e.g., given by stat./MAI).
• An undergraduate/graduate course in information theory (e.g. given by Division of Data Transmission/ISY).

Page manager: karin.johansson@liu.se
Last updated: 2014-04-29