Unit SIGNAL PROCESSING AND OPTIMIZATION FOR BIG-DATA
- Course
- Computer engineering and robotics
- Study-unit Code
- A001256
- Curriculum
- Data science
- Teacher
- Paolo Banelli
- Teachers
-
- Paolo Banelli
- Hours
- 72 ore - Paolo Banelli
- CFU
- 9
- Course Regulation
- Coorte 2021
- Offered
- 2022/23
- Learning activities
- Affine/integrativa
- Area
- Attività formative affini o integrative
- Academic discipline
- ING-INF/03
- Type of study-unit
- Obbligatorio (Required)
- Type of learning activities
- Attività formativa monodisciplinare
- Language of instruction
- Italian
- Contents
- - FUNDAMENTALS of STATISTICAL SIGNAL PROCESSING- FUNDAMENTALS of CONVEX OPTIMIZATION -BIG-DATA REDUCTION and SAMPLING - GRAPH-BASED SIGNAL/DATA PROCESSING- DISTRIBUTED OPTIMIZATION AND SIGNAL PROCESSING for LEARNING over NETWORKS
- Reference texts
- Most of the class content will be inspired to some chapters and paragraphs of these books:- S.Kay, Fundamentals of Statistical Signal Processing, Vol. I & II, Prentice Hall, 1993-1998;
- S. Theodoridis, Machine Learning: A Bayesian and optimization perspective.- T. Hastie, et. al., The Elements of Statistical Learning: data Mining, Inference, and Prediction
- M. E. J Newman, Networks an Introduction- S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004;
- S. Boyd et al., Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Foundations and Trends in Machine Learning, 3(1):1–122, 2011- Furthermore some notes of the teacher will be available. - Educational objectives
- Understanding the basics of statistical inference and convex optimization as fundamental tools in (big)-data analytics. Understanding the concept of data-reduction/sampling and conditions under which statistical inference and reconstruction of the information does not suffer too much by reduction/sampling. Extend the knowledge of classical signal processing to signals defined over a graph, which is a natural representation of big-data either dependent on their distribution over a network, or on their statistical similarity, or both. Understand the methodological tools to distribute complex statistical inference on parallel and distributed agents (computers, etc.) as a way to empower statistical inference on big-data, possibly geographically or logically distributed over a network.
- Prerequisites
- Mandatory: Calculus, Linear Algebra, Random Variables and Stochastic processes, Fourier Analysis, Digital signal processing.Suggested: Machine Learning and Data Mining. Useful: Estimation and Detection Theory (Statistical Inference)
- Teaching methods
- The class will be given face-to-face by the lecturer with the aid of computer-slides. Furthermore some of the algorithms will be also implements by PC-based simulations, interactively with the students.
- Other information
- Learning verification modality
- 1) Short Thesis on a topic related to the class content, with computer aided simulations. To be given 1 week before the oral exam.
2) Oral Exam: Discussion of the Thesis plus typically 2 questions. - Extended program
- - Part I: FUNDAMENTALS OF STATISTICAL SIGNAL PROCESSING (14 hours) Minimum variance unbiased estimation; Cramer-Rao lower bound;Sufficient statistics; maximum likelihood estimation, Linear estimation, least squares; Bayesian estimation: MMSE estimation, linear estimation. Adaptive estimation theory: Least mean squares estimation, recursive least squares estimation; Kalman filtering. STATISTICAL DECISION THEORY: Neyman-Pearson, Minimum Probability of Error, Bayes Risk, Multiple Hypothesis Testing;
- Part II: FUNDAMENTALS OF CONVEX OPTIMIZATION (10 hours ) Basics of convex optimization: Convex sets, convex functions, convex optimization problems;Duality theory: Lagrange dual problem, Slater's constraint qualifications, KKT conditions; Optimization algorithms: Primal methods (steepest descent, gradient projection, Newton method), primal-dual methods (dual ascent, alternating direction method of multipliers);Examples of applications: Approximation and fitting, statistical estimation and detection, adaptive filtering, supervised and unsupervised learning from data;
- Part III: BIG-DATA REDUCTION (8 hours)Compressed Sampling/Sensing and reconstructionStatistical Inference by Sparse Sensing, Classification by Principal Component Analysis.
- Part IV: GRAPH-BASED SIGNAL PROCESSING (18 hours) Signals on graph: motivating examples; algebraic graph theory, graph features; signal processing on graphs: Fourier Transform, smoothing, sampling, and data compression on graph;
- Part V: DISTRIBUTED OPTIMIZATION, SIGNAL PROCESSING, and LEARNING over NETWORKS (22 hours)Average consensus: Theory and algorithms; Distributed optimization: Consensus and sharing; Distributed optimization: Primal and primal-dual methods; Distributed signal processing: Estimation and detection; Distributed signal processing: LMS, RLS and Kalman Filtering on Graphs. Distributed supervised learning: Regression and data classification; Distributed unsupervised learning: Dictionary learning and data clustering;