ELEC70070 Information TheoryLecturer(s): Dr Cong Ling; Dr Javier Barria Aims
This module is designed to introduce you to the main concepts of information theory (i.e., the theory of information and data). Information theory provides you with the metrics (e.g., entropy of a source, mutual information) and with the tools to estimate the best achievable performance of a communication systems. It also allows you to predict the performance of a compression algorithm (eg., image compression algorithm). For this module we require a background knowledge of mathematics and elementary probability.
Learning Outcomes
At the end of the module, you are expected to be able to (1) apply the concept and properties of entropy and mutual information to design communication systems, (2) prove the source coding theorem, (3) deduce the fundamental performance limits of noisy communication channels, (4) predict the performance of a compression algorithm by using the ratedistortion function, (5) apply network information theory to design complex communication systems.
Syllabus
Elements of information theory of discrete systems; information measures, memoryless and memory sources, the noiseless coding theorem; Methods of source coding; Information theory of continuous systems; Shannon's capapcity theorem and its interpretation; Lossy source coding and ratedistortion theory; Network information theory.
Exam Duration: 3:00hrs Exam contribution: 100% Coursework contribution: 0% Term: Spring Closed or Open Book (end of year exam): Closed Coursework Requirement: Nil Oral Exam Required (as final assessment): no Prerequisite module(s): None required Course Homepage: http://www.commsp.ee.ic.ac.uk/~cling/IT/InformationTheory.htm Book List: Please see Module Reading list
