Course Websites

ECE 598 PM - Computational Inference and Learning

Last offered Fall 2016

Official Description

Subject offerings of new and developing areas of knowledge in electrical and computer engineering intended to augment the existing curriculum. See Class Schedule or departmental course information for topics and prerequisites. Course Information: May be repeated in the same or separate terms if topics vary.

Section Description

Prerequisites: ECE 490, ECE 534. Computational inference and machine learning have seen a surge of interest in the last 15 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces the mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, sparsity analysis, approximate inference and search, and fast optimization.

Related Faculty

Course Director

Description

Computational inference and machine learning have seen a surge of interest in the last 15 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces the mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, sparsity analysis, approximate inference and search, and fast optimization.

Detailed Description and Outline

Lecture 1: Introduction, information-theoretic functionals.

Lecture 2: Bayes inference, maximum likelihood principle, Maximum A Priori (MAP) estimation, Minimum Mean Squared Error (MMSE) estimation

Lecture 3: Empirical Risk Minimization

Lectures 4, 5: Stochastic Approximation and Stochastic Gradient Descent

Lecture 6: Statistical performance analysis via Monte Carlo methods and importance sampling

Lecture 7: Bootstrap

Lecture 8: Bayesian recursive estimation using Particle Filtering

Lectures 9-11: Parameter estimation via Expectation Maximization (EM) algorithm

Lectures 12, 13: Hidden Markov Models, Viterbi algorithm, Baum-Welch learning

Lectures 14, 15: Linear Dynamical Systems, RTS smoother

Lectures 16-18: Graphical models, belief propagation, approximate inference

Lectures 19-22: Approximate inference using variational inference and mean-field techniques

Lectures 23-25: L1-penalized least squares minimization

Lectures 26-28: Reconstruction of sparse signals using Compressive Sensing

Lecture 29: Dimensionality reduction using random projections; hashing

Grading will be based on homeworks (20%), a midterm exam (40%), and a final project (40%).

Texts

Notes from the instructor and articles from scientific journals.

TitleSectionCRNTypeHoursTimesDaysLocationInstructor
Comput. Inference and LearningPM66385LEC41400 - 1520 T R  3015 Electrical & Computer Eng Bldg Pierre Moulin