Hide menu

MAI0019
Statistical Learning Theory with Concentration Inequalities/
Statistik inlärningsteori med koncentrationsolikheter

Number of credits: 15 hp

Examiner: Timo Koski

Course literature:

  • O. Catoni (2004): Statistical Learning Theory and Stochastic Optimization, Lecture Notes in Mathematics 1851, Springer Verlag.
  • V.N.Vapnik (1998): Statistical Learning Theory. chapters 14-16. John Wiley & Sons.
  • M. Vidyasagar (2003): Learning and Generalization. Springer.
  • Material on Support Vector Machines.

Course contents:

  • The course investigates tools for analysis of performance of simplified models for prediction, estimation, and classification of complex data.
  • The course starts with the theory and algorithms of support vector machines and the Vapnik-Chervonenkis theory.
  • The techniques of analysis are rooted in information theory (minimax compression and learning, 'blowing-up lemma'), PAC-Bayesian theorems and concentration inequalities. Tools from probability are Bennett's, Hoeffding's, Chernoff's, Azuma's and McDiarmid's inequalities.
  • Oracle inequalities, non-asymptotic bounds on the statistical risk, selfboundedness of Vapnik entropy and concentration inequalities for statistical learning (entropy method, logarithmic Sobolev inequalities) will be presented.

Organisation:

Examination: Presentations by participants. Home assignments.

Prerequisites:

  • A graduate course in measure and integration theory (e.g., given by TM/MAI).
  • A graduate course in Markov Chain Monte Carlo (e.g., given by mat.stat./MAI).
  • A graduate course in probability and stochastic processes (e.g, given by mat.stat./MAI).
  • A graduate course in statistical inference. (e.g., given by stat./MAI).
  • An undergraduate/graduate course in information theory (e.g. given by Division of Data Transmission/ISY).

 


Page manager: karin.johansson@liu.se
Last updated: 2014-04-29