EE Department Intranet - intranet.ee.ic.ac.uk
Close window CTRL+W

ELEC60009 Deep Learning


Lecturer(s): Prof Krystian Mikolajczyk

Aims

This is a continuation of the Autumn term course EE3-35 Machine Learning. In contrast to machine learning included in EE3-23, EE3-25 deep learning will focus on deep neural network based learning. It introduces the background and illustrates how deep learning is impacting our understanding of intelligence and contributing to the practical design of intelligent machines.
Deep learning is currently the most active area of research and development and in high demand for experts by hi-tech start-ups, large companies as well as academia. It is the preferred approach for modern AI and machine learning in any domain. Deep learning techniques enable us to automatically extract features from data so as to solve predictive tasks, such as speech recognition, object recognition, machine translation, question-answering, anomaly detection, medical diagnosis and prognosis, automatic algorithm configuration, personalisation, robot control, time series forecasting, and much more.

Upon completion of this module, the student will be able to demonstrate and apply knowledge and understanding of:
- The underlying mathematical and algorithmic principles of deep learning
A wide variety of deep learning algorithms
- The key factors that have made deep learning successful for various applications
How deep learning fits within the context of other ML approaches and what learning tasks it is or isnít suited for
- How to perform evaluation of deep learning algorithms and model selection.
What is involved in learning from data.
- The challenges of deep learning
- The problems that arise when dealing with very small and very big data sets, and how to solve them.

Learning Outcomes

Having successfully completed this module the student will be able to:

Intellectual skills
- formalise a deep learning problem
- propose a deep learning methodology to learn from data under various conditions
- choose and fit models to data
- critically appraise the merits and shortcomings of model architectures on specific problems
- understand neural implementations of attention mechanisms and data representation models
- understand numerical computation, statistics and optimization in the context of deep learning.
- Understand hardware, software and data requirements to perform deep learning

Practical Skills
- apply existing deep learning models to real datasets
- gain first experience in working with deep learning libraries in order to create and evaluate network architectures
- understand and apply the deep learning engineering necessary for designing solutions for new tasks and data.
- approach practical machine learning problems.
- implement basic versions of some of the core deep network algorithms (such as backpropagation)
- implement and evaluate common neural network models for various types of data in language, vision, speech, decision making, and more.
- apply a variety of learning algorithms to data.
- combine modular components to build deep learning systems in a wide range of real-world applications.

Transferable / Key Skills
- apply their knowledge and skills in arbitrary projects and course work where learning from data is essential
- perform a critical appraisal of recent scientific literature in deep learning
- understand tools and techniques used in their design and analysis,
- evaluate the performance of a machine learning algorithm
- design deep learning experiment
- choose appropriate performance evaluation metrics
- conduct a learning experiment and generate results
- present and analyse the results as well as draw appropriate conclusions

Syllabus

- Introduction to deep learning
- Deep Feedforward Networks
- Regularization for Deep Learning
- Optimization for Training Deep Models
- Convolutional Networks
-Sequence Modelling: Recurrent and Recursive Nets
- Autoencoders
- Representation Learning
- Structured Probabilistic Models for Deep Learning
- Deep Generative Models
- Reinforcement learning

Pre-requisites
Machine Learning EE3-25
Maths for Signals and Systems: EE3-10


Textbook
Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT Press, http://www.deeplearningbook.org
Assessment
Exam Duration: N/A
Coursework contribution: 100%

Term: Spring

Closed or Open Book (end of year exam): N/A

Coursework Requirement:
         Coursework only module

Oral Exam Required (as final assessment): N/A

Prerequisite module(s): ELEC60019 - Machine Learning

Course Homepage: Blackboard

Book List:
No.Reference
1.Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT Press, http://www.deeplearningbook.org