ECE6254 Course Syllabus

ECE6254

Statistical Machine Learning (3-0-3)

Technical Interest
Group

Digital Signal Processing

Prerequisites

None

Corequisites

None

Catalog Description

An introduction to the theory of statistical learning and practical machine learning algorithms with applications in signal processing and data analysis.

Textbook(s)

T. Hastie, R. Tibshirani, and J.H. Friedman, The Elements of Statistical Learning (2nd edition), Springer, 2009. ISBN 9780387848570(optional) (comment: Available for download with corrections in the 5th printing with course resources at: http://www-stat.stanford.edu/~tibs/ElemStatLearn/ )

Strategic
Performance
Indicators (SPIs)

SPIs are a subset of the abilities a student will be able to demonstrate upon successfully completing the course.

Topical Outline

1.	Supervised learning
	a)	The Bayes classifier and the likelihood ratio test
	b)	Nearest neighbor classification
	c)	Linear classifiers 
		i.	plugin classifiers  (LDA, logistic regression, Naive Bayes)
	       ii.	the perceptron learning algorithm
	      iii.	maximum margin principle and separating hyperplanes
	d)	Linear regression
	        i.	least-squares linear regression
	       ii.	the LASSO
	e)	Theory of generalization
		i.	overfitting
	       ii.	concentration inequalities
	      iii.	VC dimension and generalization bounds
	       iv.	the bias-variance tradeoff
		v.	regularization
	f)	Nonlinear classifiers 
		i.	nonlinear feature maps 
               ii.	the kernel trick 
	      iii.	SVMs
	       iv.	multi-layer neural networks
	g)	Nonlinear methods in regression 
	h)	Error estimation and validation
2.	Unsupervised learning
	a)	Linear dimensionality reduction and principal component analysis
	b)	Mutltidimensional scaling
	c)	Clustering 
	        i.	K-means
	       ii.	GMMs and the EM algorithm
	      iii.	spectral clustering
	d)	Density estimation
	e)	Feature selection 
	f)	Nonlinear dimensionality reduction (manifold learning)
3.	Other topics (as time permits)
	a)	Matrix factorizations
	b)	Graphical models
	c)	Ensemble methods (boosting, random forests)
	d)	- Deep learning