cs-570 statistical signal processing

CS-570Statistical Signal Processing

Spring Semester 2019

Grigorios Tsagkatakis

Lecture 1: Introduction to CS-570

Spring Semester 2017-2018CS-541 Wireless Sensor Networks

University of Crete, Computer Science Department1

Today’s Objectives

• CS-570 Overview

• Introduction to Statistical Signal Processing

CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department

2Spring Semester 2017-2018

About CS-570

• Lectures

Monday 12:00-14:00, H208

Wednesday 14:00-16:00, H204

• Office Hours: 1 Hour after each class

• Prerequisites: Digital Signal Processing (CS-370), Applied Mathematics for Engineers (CS-215), Probabilities (CS-217)

CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department

3Spring Semester 2017-2018

About CS-570

Practical Information

• 2 individual homeworks on the material taught (30% of your final grade)• Exercises on MATLAB/python• 1st assignment will be handed out at the beginning of March• 2 weeks time to complete each assignment (hard deadline).

• Standalone project (max for 2 students) (50% of your final grade)• Research topic• Experimental work / analysis on experimental data• Submission of a project report in a technical paper form (motivation, related work, problem

formulation, adopted methodology, results, conclusions & outlook)• Duration: mid of April - End of semester (~mid of June)

• Written Exam (20% of your final grade)

All above are compulsory for getting a grade at the end of the exam

CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department

4Spring Semester 2017-2018

Topics

CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department

5

Week 1: Introduction to Statistical Signal Processing & Review

Week 2: Introduction to optimization

Week 3: Signal sensing and reconstruction

Week 4: Computational imaging

Week 5: Deterministic signal processing

Week 6: High and low dimensional signal processing

Week 7: Statistical signal models

Week 8: Time-series modeling

Week 9: Distributed signal processing

Week 10: Machine learning for signal processing

Week 11: Applications in remote sensing

Week 12: Applications in Internet-of-Things

Week 13: Review and presentations

Spring Semester 2017-2018

6Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Signal Processing fundamentals

• Acquisition

• Processing

• Analysis

• Storage

• Transmission

7Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Temperature& Humidity

Image Sound

Pressure Vibration,Motion

Glucose (&biometrics)

Signal Processing fundamentals

• Acquisition

• Processing

• Analysis

• Storage

• Transmission

8Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Signal Processing fundamentals

• Acquisition

• Processing

• Analysis

• Storage

• Transmission

9Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Signal Processing fundamentals

• Acquisition

• Processing

• Analysis

• Storage

• Transmission

10Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Signal Processing fundamentals

• Acquisition

• Processing

• Analysis

• Storage

• Transmission

11Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

12Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

13Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

14Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

15Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

Variety

16Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

Variety

Veracity

17Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Big Data

The 5VsVolume

Velocity

Variety

Veracity

Value

18Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Earth Observation

19Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Astrophysics

20Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Sky Survey Project Volume Velocity

Sloan Digital Sky Survey (SDSS)

50 TB 200 GB per day

Large Synoptic Survey Telescope (LSST )

~ 200 PB 10 TB per day

Square Kilometer Array (SKA )

~ 4.6 EB 150 TBper day

Internet-of-Things

21Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Biomedical signals

22Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Neuroscience

23Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Imaging technologies

24Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

25Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Fundamental Signal Processing

• Signal Sensing: Compressed Sensing

• Inverse problems: Signal Denoising, Enhancement

• Filtering

• Time-series modeling and prediction

• Modeling: dimensionality reduction

26Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

• -

27Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

28Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

power unit(battery based – limited lifetime!)

sensors(transducer, measuring a physical phenomenon e.g.

heat, light, motion, vibration, and sound)

Memory /storage

(data acquisition, and preprocessing, buffers

handling)

transceiver(connection to the outer-world, e.g. other sensor nodes, or data

collectors --sinks)

• Sensor Node

• Basic unit in sensor network

• Contains on-board sensors, processor, memory, transceiver, and power supply

CS-541 Wireless Sensor NetworksUniversity of Crete, Computer Science Department

29

Sensing node paradigm

microProcessor(communication with sensors &

transceivers , preprocessing, buffers handling, etc)

Spring Semester 2017-2018

Review of basic concepts

30Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Vectors

31Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

• A column vector where

• A row vector where

denotes the transpose operation

Vectors

• Vectors can represent an offset in 2D or 3D space

• Points are just vectors from the origin

• Data (pixels, gradients at an image keypoint, etc) can also be treated as a vector

• Such vectors don’t have a geometric interpretation, but calculations like “distance” can still have value

32Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Matrix

33Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

• A matrix is an array of numbers with size 𝑚 ↓ by 𝑛 →, i.e. m rows and n columns.

• If , we say that is square.

=

Basic Matrix Operations

34Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

• Addition

• Can only add a matrix with matching dimensions, or a scalar.

• Scaling

Matrix Operations

• Inner product (dot product) of vectors• Multiply corresponding entries of two vectors and add up the result

• x·y is also |x||y|cos( the angle between x and y )

• If B is a unit vector, then A·B gives the length of A which lies in the direction of B

35Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Matrix Operations

Matrix Multiplication

• Properties

• Powers• We can refer to the matrix product AA as A2, and AAA as A3, etc.

• Only square matrices can be multiplied that way

36Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Matrix Operations

• Transpose

• Determinant• returns a scalar

• Represents area of the parallelogram described by the vectors in the rows of the matrix

• For

• Properties:

37Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Matrix Operations

• Trace

• Invariant to a lot of transformations

• Properties:

38Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Special matrices

• Identity matrix I

• Diagonal matrix

• Symmetric matrix

• Skew-symmetric matrix

39Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Matrix Inverse

• Given a matrix A, its inverse A-1 is a matrix such that AA-1 = A-1A = I

• Inverse does not always exist. If A-1 exists, A is invertible or non-singular. Otherwise, it’s singular.

• For matrices that are invertible

40Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

System of linear equations

41Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Solution of system

• Inverse of a matrix

•Solution of systems of linear equations

• Provided A-1 exists

• If both x and y are solutions thenis also a solution for any real α

42Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

z = αx + (1 − α)y

Linear combinations

• Linear combination of some set of vectors {v(1), . . . , v(n)} is given by multiplying each vector v(i) by a corresponding scalar coefficient and adding the results:

•The span of a set of vectors is the set of all points obtainable by linear combination of the original vectors.

43Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Norms

44Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Norms

• L2 norm, also known as Euclidean norm

• L21 norm

• Infinite norm (or max norm)

• Frobenius norm (Matrix norm)

45Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Linear independence

46Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

• Suppose we have a set of vectors v1, …, vn

• If we can express v1 as a linear combination of the other vectors v2…vn, then v1 is linearly dependent on the other vectors. • The direction v1 can be expressed as a combination of the directions

v2…vn. (E.g. v1 = .7 v2 -.7 v4)

• If no vector is linearly dependent on the rest of the set, the set is linearly independent.• Common case: a set of vectors v1, …, vn is always linearly independent if

each vector is perpendicular to every other vector (and non-zero)

Linear independence

47Spring Semester 2019CS-570 Statistical Signal Processing

University of Crete, Computer Science Department

Not linearly independentLinearly independent set