EE Department Intranet -
Close window CTRL+W

ELEC70070 Information Theory

Lecturer(s): Dr Cong Ling; Dr Javier Barria


This module is designed to introduce you to the main concepts of information theory (i.e., the theory of information and data). Information theory provides you with the metrics (e.g., entropy of a source, mutual information) and with the tools to estimate the best achievable performance of a communication systems. It also allows you to predict the performance of a compression algorithm (eg., image compression algorithm). For this module we require a background knowledge of mathematics and elementary probability.

Learning Outcomes

At the end of the module, you are expected to be able to (1) apply the concept and properties of entropy and mutual information to design communication systems, (2) prove the source coding theorem, (3) deduce the fundamental performance limits of noisy communication channels, (4) predict the performance of a compression algorithm by using the rate-distortion function, (5) apply network information theory to design complex communication systems.


Elements of information theory of discrete systems; information measures, memoryless and memory sources, the noiseless coding theorem; Methods of source coding; Information theory of continuous systems; Shannon's capapcity theorem and its interpretation; Lossy source coding and rate-distortion theory; Network information theory.
Exam Duration: 3:00hrs
Exam contribution: 100%
Coursework contribution: 0%

Term: Spring

Closed or Open Book (end of year exam): Closed

Coursework Requirement:

Oral Exam Required (as final assessment): no

Prerequisite module(s): None required

Course Homepage:

Book List:
Please see Module Reading list