ECE6260 Course Syllabus

ECE6260

Data Compression and Modeling (3-0-3)

Technical Interest
Group

Digital Signal Processing

Prerequisites

ECE 4270

Corequisites

None

Catalog Description

Theory and algorithms of signal encoding and decoding for data compression. Applications in information systems, digital telephony, digital television, and multimedia Internet.

Textbook(s)

K. Sayood, Introduction to Data Compression (5th edition), Morgan Kaufmann, 2017. ISBN 9780128094747 (required)

A. Gersho, R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Press, 1992. ISBN 9780792391814(optional)

Gibson, Berger, Lookabaugh, Lindbergh and Baker, Digital Compression for Multimedia, Principles and Standards, Morgan Kaufmann, 1998. ISBN 9781558603691(optional)

Strategic
Performance
Indicators (SPIs)

SPIs are a subset of the abilities a student will be able to demonstrate upon successfully completing the course.

Outcome 1 (Students will demonstrate expertise in a subfield of study chosen from the fields of electrical engineering or computer engineering):
1.	Understand the theory and practice of data compression and modeling as well as its applications in digital media and communication systems in everyday use.

Outcome 2 (Students will demonstrate the ability to identify and formulate advanced problems and apply knowledge of mathematics and science to solve those problems):
1.	Understand and implement methods and algorithms for data compression and modeling

Outcome 3 (Students will demonstrate the ability to utilize current knowledge, technology, or techniques within their chosen subfield):
1.	Design algorithms to model a given signal, extract model parameters, and achieve effective data compression on the signal  

Topical Outline

Introduction
 signal compression, lossless and lossy compression
 communication systems and building blocks: sources, channels, and codes
 issues - fixed rate and variable rate, robustness to channel errors, degradation and perceptual effects

Quantization theory
 uniform quantization, distortion and bit rates
 amplitude distribution and high-rate quantization theory
 Bennett approximations and optimal performance, Lloyd's code optimality and algorithm
 elementary distortion-rate theory

Architecture for data compression & introduction to data modeling
 signal models & spectral analysis
 quantization with memory
 fixed-rate vs. variable-rate code
 entropy, estimated entropy, complexity and typical sequence of an ergodic source
 variable rate quantization: lossless codes, prefix code

Lossless Coding Techniques
 Huffman coding, arithmetic coding
 Universal lossless codes, adaptive and predictive lossless coding

Distortion & Similarity Measures
 sample difference, sum of squared deviations and Euclidean distance
 Lp-norm, city-block distance, Mahalanobis distance
 transformation and transformation invariant similarity measures
 spectral distortion measures
 mutual-information, divergence, and Kullback-Liebler number
 perceptual issues

Coding algorithms  scalar quantization
 clustering algorithms for quantizer design
 the Lloyd algorithm and its generalization
 entropy-constrained quantizers

Coding algorithms - vector quantization (VQ)
 sphere packing and optimal uniform lattice quantizers
 Lloyd algorithm - revisited
 progressive vector quantization
 variations of vector quantization
 finite-state VQ and Markov models
 tree and trellis encoding

Applications
 speech and audio coding
 image and video coding

Compression standards and formats
 Historical and evolutional aspects behind development of standards
 Application areas