ELEC60002 Advanced Signal Processing
Lecturer(s): Prof Danilo Mandic
Robust and reliable estimation of physically meaningful but random data, observed under uncertainty (noise), is a prerequisite to many practical applications, such as those in communications, biomedical engineering, speech or finance.
The aim of this course is to build a foundation for real-world Data Analytics, starting from reliable descriptors of signals and noise, maximum achievable performance bounds of practical estimators for noisy data, model-based and block methods, through to sequential detection and estimation of random signals buried in unknown noise.
The students will be introduced to the fundamentals of statistical signal processing, with particular emphasis upon classical and modern estimation theory, parametric and nonparametric stochastic modelling, time series analysis and forecasting, least squares methods, and basics of adaptive signal processing.
The students will also gain practical experience with random signals, including their own speech and physiological recordings, through the provision of structured coursework assignments based upon using MATLAB.
The students will gain the understanding of advantages of utilising statistical signal processing when performing estimation on real world signals. This includes practical experience related to the probabilistic descriptors of signals and noise, detection of signals in noise, dealing with signal nonstationarity, estimation of parameters of the underlying signal generating models, model-free approaches, real-time approaches to signal tracking, and statistical forecasting.
At the end of the course students should be able to:
Perform robust estimation of random data in noisy environment,
Analyse discrete time random signals based upon their statistical properties, such as their probability density functions, degree of statistical stationarity and ergodicity, degree of correlation and coupling.
Calculate the first, second, and higher statistical moments of discrete time random signals, namely their mean, correlation and covariance functions, and use these for analysis of real world signals.
Familiarise themselves with the concept of an estimator of an unknown general scalar or vector parameter.
Based upon the concept of goodness of an estimator, produce reliable estimates of unknown random variables.
Understand the importance of assessing the bias and variance of an estimator, and their implications in processing real-world data.
Grasp the concepts of a minimum variance estimator and consistent estimator.
Become familiar with parametric stochastic models for random data, such as the Autoregressive (AR), moving average (MA), and their combination (ARMA).
Learn about the Yule-Walker (Normal) equations and the importance of first and second order moments in statistical modelling of time series.
Derive stability and invertibility conditions for linear stochastic models.
Learn about optimal model selection, and employ those concepts to process real world signals using ARMA model theory, such as financial time series, in particular in the prediction setting.
Understand how to assess the performance of statistical estimators, and how to employ the bias-variance dilemma in practical problems to obtain optimal performance.
Use Cramer Rao theory to obtain the Minimum Variance Unbiased Estimator (MVUE), that is, a theoretic bound on the performance of any estimator, and evaluate its variance via the Fisher information matrix.
Based on the Cramer Rao theory, derive more practical special cases such as the Best Linear Unbiased Estimator (BLUE).
Understand how the BLUE and other estimators can be considered as constrained optimisation problems.
Formulate the maximum likelihood and Bayesian estimator and compare their performance with standard estimators.
Formulate the Least-Squares estimation problem and apply it to a range of real world applications and gain experience with optimal model selection.
Derive the sequential form of the least squares estimator and understand its convergence properties.
Learn about the principle of orthogonality within least squares estimation and its implications on practical estimators.
Become familiarised with the concept of optimal Wiener filtering, and understand the difference between block and sequential adaptive supervised models.
Learn basics of adaptive supervised learning, and gain understanding of its applications in real-time adaptive prediction, system identification, noise cancellation, and channel identification.
Become familiar with the concept of steepest descent and learn about the benefits and drawbacks of iterative learning strategies.
Derive the Least Mean Square (LMS) adaptive filtering algorithm for prediction and identification of real world signals.
Understand the operation of LMS when applied to real-world nonstationary signals and signals with large dynamical range.
Become familiar with the concept of artificial neuron and its interpretation as a nonlinear adaptive filter.
Apply statistical signal processing in noise cancellation and signal enhancement applications such as in finding features in your own speech, latent components in your own electrocardiogram, acoustic echo control and separation of maternal/foetal heartbeats from their mixture.
Discrete random signals; statistical stationarity, strict sense and wide sense. Averages; mean, correlations and covariances. Bias-Variance dilemma. Curse of dimensionality. Linear stochastic models. ARMA modelling. Stability of linear stochastic models. Introduction to statistical estimation theory. Properties of estimators; bias and variance. Role of Cramer Rao lower bound. Minimum variance unbiased estimator. Best linear unbiased estimator (BLUE). Maximum likelihood estimator. Bayesian estimation. Least squares estimation: orthogonality principle, block and sequential forms. Wiener filtering, adaptive filtering and signal modelling. Concept of an artificial neuron. Applications: time series modelling (financial, biomedical), acoustic echo cancellation and signal enhancement, inverse system modelling and denoising (communications), estimation of directional processes (such as those in oil exploration).
Exam Duration: N/A
Coursework contribution: 100%
Closed or Open Book (end of year exam): N/A
To be announced
Oral Exam Required (as final assessment): no
Prerequisite module(s): None required
Course Homepage: unavailable