ECE6254 Course Syllabus

ECE6254

Statistical Machine Learning (3-0-3)

Prerequisites

ECE 4270

Corequisites

None

Catalog Description

An introduction to the theory of statistical learning and practical machine learning algorithms with applications in signal processing and data analysis.

Textbook(s)

T. Hastie, R. Tibshirani, and J.H. Friedman, The Elements of Statistical Learning (2nd edition), Springer, 2009. ISBN 9780387848570 (required) (comment: Available for download with corrections in the 5th printing with course resources at: http://www-stat.stanford.edu/~tibs/ElemStatLearn/ )

Hayes, Statistical Digital Signal Processing and Modeling, John Wiley and Sons, 1996. ISBN 9780471594314 (required)

Topical Outline

Basic techniques for modeling discrete-time sequences (3 weeks)
     Problem formulation
     The direct (least squares) method
     The Pad\'{e} approximation
     Prony's method
     Shanks' method, iterative prefiltering
     All-pole modeling and linear prediction
     The autocorrelation and covariance methods
     FIR least squares inverse filter design
     Applications and examples
     
Fast algorithms for solving Toeplitz equations (3 weeks)
     The Levinson-Durbin recursion
     Step-up, step-down, inverse Levinson-Durbin recursion
     Minimum phase property of PEF
     Cholesky decomposition of autocorrelation matrix and its consequences
     Lattice filters
     The Levinson recursion
     The Trench algorithm and the Schur recursion - optional
     Split Levinson recursion and line spectral pairs - optional
     Fast covariance algorithm - optional
     Applications and examples 
     
Lattice methods (2 weeks)
     Lattice filters (FIR, all-pole, and pole/zero)
     Forward and backward covariance methods
     The Burg recursion and the modified covariance algorithm
     Examples
     Application - wave propagation in layered material
     
Wiener filtering (2 weeks)
     Review of Discrete-time random processes
     FIR Wiener filters
     Noncausal IIR Wiener filters
     Causal Wiener filters
     Applications - Linear prediction, deconvolution, smoothing

Power spectrum estimation (4 weeks)
     Classical methods 
     The minimum variance method
     The maximum entropy method and relation to minimum variance method
     Parametric spectrum estimation
     Comparison of methods
     Subspace methods
     Applications