elec-e8740 course overview and introduction to sensor fusion

ELEC-E8740 — Course Overview andIntroduction to Sensor FusionSimo SärkkäAalto University

September 9, 2019

Course Overview and IntroductionSimo Särkkä

2 / 29

Contents

1 Course Organization

2 Overview of Sensor Fusion

3 Example: Sensor Fusion in Drone and Autonomous Car

4 Summary


4 / 29

The Team

Simo SärkkäMain LecturerF305, Rakentajanaukio [email protected]

Muhammad EmzirProject/Exercise Teacher,Co-LecturerF307, Rakentajanaukio [email protected]

mailto:[email protected]

mailto:[email protected]


5 / 29

Your Turn. . .

My name is. . .My background is in. . .I am particularly interested in. . .In this course, I hope to. . .The best thing about. . .You should also know that. . .


6 / 29

Intended Learning OutcomesAfter successfully completing this course, you are able to:

explain the principles and components of sensor fusionsystems,identify and explain the differences between linear andnonlinear models and their implications on sensor fusion,construct models of multi-sensor systems and useleast-squares algorithms for sensor fusion,construct continuous and discrete time state-space modelsbased on ordinary differential equations, differenceequations, and physical sensor models,develop and compare state-space models and Kalman aswell as particle filtering algorithms for solving sensor fusionproblems.


7 / 29

Schedule (1)13 lectures

Wednesdays, 14:15 - 16:00 in F175b, Health TechnologyHouse, Otakaari 3.Except for the first lecture on Monday, Sep 9, 2019, 14:15 -16:00Detailed preliminary schedule (incl. topics) on MyCourses

12 exercise sessionsExercise sessions are held on Mondays, 14:15 - 16.00 inF175b, Health Technology House, Otakaari 3Exercises start on Monday, Sep 16, 2019.

No lecture and exercise session during exam week (Oct21, 2019–Oct 25, 2019)Exam on Monday December 9, 2019, 14.00 - 17.00 inF175b, Health Technology House, Otakaari 3.

Check MyCourses regularly for updates!


8 / 29

Course Literature

Lecture notes and slides are the main course literatureRewards for finding mathematical errors (not for typos andsuch, but report anyway)

Lecture notes are available on the course homepage inmycourses.Slides will be made available in mycourses just beforeeach lecture.Optional textbook:

Gustafsson, F. (2018). Statistical Sensor Fusion.Studentlitteratur, 3rd edition.Very complete (and more extensive) treatment of thesubjectNot required to pass the courseGood reference for the future


9 / 29

Assessment and GradingAssessmentTo pass this course, you need to:

pass the written exam,pass the project.

Written ExamPen & paper exam,Allowed aids: One (1) hand-written A4-paper with notes,pens, calculator. No lecture notes, books, etc.!

GradingThe grading scale is 0–5,The final grade is the average of the written exam and theproject.


10 / 29

Exercise Sessions

Exercise sessions are held on Mondays, 14:15 - 16.00 inF175b, Health Technology House, Otakaari 3, starting onMonday, Sep 16, 2019.In the exercise sessions, the teacher shows you hands onhow to solve the exercises.You also have the chance to solve the exercises yourself.Pen & paper and computer exercises

Matlab or Octave recommended, Python is possibleBring your own computer

Exercise sessions are not mandatory but highlyrecommended, the exam questions may be related to theexercises


11 / 29

Project — General InformationTrack an autonomous robot using multiple sensorsStarts in Oct 16, 2019, ends on Dec 20, 2019

Project guide will be available on MyCourses beforehandGroup work, groups of 2-3Experimental work

Experiments are done in our lab in F321, Otakaari 3Lab time needs to be booked through MyCourses

The project is a significant amount of the individual work inthis course, start early!


12 / 29

Project — Assessment

Two written hand-ins:1 Intermediate report, deadline: Nov 15, 20192 Final report, deadline: Dec 20, 2019

Both hand-ins have to be submitted as PDF documents onMyCourses before the deadlineThe final evaluation criteria will be available on MyCoursestogether with the project guide


13 / 29

Schedule (2)

Week

37 38 39 40 41 42 43 44 45 46 47 48 49 50 51

Static Fusion

Dynamic Models

Robot Platform

Kalman Filtering

Particle Filtering

Project

Intermediate Report

Final Exam

Final Report


14 / 29

Presemo Questionnaire

We are using presemo on this course.Please use your computer or mobile phone and go to:

http://presemo.aalto.fi/fusion

http://presemo.aalto.fi/fusion


16 / 29

Definition of Sensor Fusion

One possible definition of sensor fusion:"computational methodology which aims at combin-

ing the measurements from multiple sensors such thatthey jointly give more information on the measured sys-tem than any of the sensors alone."

The important aspects are:It is computational methodology.Uses measurements from multiple sensors.Attempts to use the information from all the sensors jointly.


17 / 29

Sensor Fusion Applications: Drones


18 / 29

Sensor Fusion Applications: Autonomous Cars


19 / 29

Sensor Fusion Applications: Smartphones

Accelerometer)Gyroscope)

Wi0Fi)

Bluetooth)

Magnetometer)

GPS/GNSS)

3G/4G/5G)

Barometer)


20 / 29

The Components of Sensor Fusion

Sensor(s)EstimationAlgorithm

Variable ofInterest

Model(s)


22 / 29

Model of a Drone (1)

We measure y1 with e.g.radar or ultrasound.We measure y2 with e.g.radar or barometer.We wish to "fuse" thesensor measurements toget the location (px ,py ).The model in this case is

y1 = px + r1,

y2 = py + r2. (r1 and r2 here denote measurement noises)

Sensor fusion amounts tojust px ≈ y1 and py ≈ y2.

y1

px

y2 py


23 / 29

Model of a Drone (2)

We could also measurethe distance y3 to anadditional tilted wall.The model now becomes

y1 = px + r1,

y2 = py + r2,

y3 =1√2(px − x0) +

1√2

py + r3.

In vector form:

y = G x + b + r.

Linear least squaresmethod gives x = (px ,py ).

y1

px

y3y2 py


24 / 29

Model of an Autonomous Car (1)We measure relative positionsof M landmarks.We get 2M measurements(M = 4 here):

y1 = sx1 − px + r1,

y2 = sy1 − py + r2,

...y2M−1 = sx

M − px + r2M−1,

y2M = syM − py + r2M .

Again leads to form

y = G x + b + r.


25 / 29

Model of an Autonomous Car (2)

We only measure the range toeach landmark.In that case we have

yR1 =

√(sx

1 − px)2 + (sy1 − py )2 + rR

1 ,

...

yRM =

√(sx

M − px)2 + (syM − py )2 + rR

M .

This is a non-linear model

y = g(x) + r

Non-linear least squaresmethod is needed.


26 / 29

Dynamic Models

The object of interest might also be moving.We can model time-continuity with a dynamic model.For example, we might have

xn = xn−1 + qn (here qn is a noise process)

More generally we get state-space models of the form

xn = f(xn−1) + qn,

yn = g(xn) + rn.

Can be coped with Kalman filters, extended/unscentedKalman filters, and particle filters.


28 / 29

Technical Contents of the Course

Formulation of sensor fusion as aleast squares problem.Solution methods for linear leastsquares problems.Solution methods for non-linearleast squares problems.Solution methods for dynamicleast squares (state-estimation)problems.Implementation of themethodology to robot platform.


29 / 29

SummaryLectures are on Wednesdays in 14:15-16:00Exercises on Mondays in 14:15-16:00Teaching materials are lecture notes and slides inmycourses.Project work starts in mid November and it is about sensorfusion in a mobile robot.Exam is in early December and project work deadline justbefore Christmas.Sensor fusion is methodology for intelligent processing ofmeasurements from multiple sensors.In practice, linear/non-linear least squares methods andKalman/particle filtering methods.We use presemo on the course. . .

elec-e8740 course overview and introduction to sensor fusion

Documents