1© 2015 The MathWorks, Inc.

Connecting MATLAB and ROS

for Autonomous Driving

Carlos Santacruz-Rosero, Ph.D.Senior Application Engineer – Robotics

Harshita BhuratRobotics System Toolbox Product Manager

2

Agenda

Automated Driving at BMW

ROS for Automated Driving

Robotics Workflow at the MathWorks

3

4

Automated Driving with ROS at BMW

** MICHAEL AEBERHARD, BMW, ROSCON 2015

5

Automated Driving with ROS at BMW

Using Robotics System Toolbox

6

What is ROS (Robot Operating System)?

• An architecture for distributed inter-

process communication

• Packages for common algorithms

and drivers

• Multilanguage interface (C++,

Python, Lua, Java and MATLAB)

7

ROS Trends in Robotics Development

http://rosindustrial.org/ric-americas/

• #1 middleware for robotics

applications development

• Popular in research and

gaining great momentum in

industry

8

Benefits of ROS for Automated Driving

• Sensor data acquisition

• Reliable distributed architecture

• Lots of “off the shelf” algorithms for

autonomy • Path planning

• Collision avoidance

• Sensor processing

• Simplified component compatibility

through standalone interfaces

• Integration with simulation environments

9

Autonomous Car as an Advanced Robotics System

LIDAR RADAR GPS/IMUCamera

Steering Actuator

Actuator ECUs

GAS/Brake Actuator

Motion Controllers

Planning

Localization Obstacle

avoidance

Global Map

Motion controlNODE

NODE

NODE

NODE

NODE

NODE

ROS: communication framework and stack of libraries

10

Robotics Development Workflow with Hand Code

Implementation

- Traceability to requirements

- Analyzing logged data

- Designing algorithm

- Translating to C/C++ , Python

- Visualizing and debugging

- Changing open-source code

- Integration into production

Product

Requirements

Idea

Design, Simulation, Prototype

C++ Python

11

Robotics Development Workflow with MATLAB and MBD

Implementation

Product

Requirements

Idea

Design and

Simulation

- System Level Simulation

- Leverage built-in algorithms and

Apps

- Co-simulation with ROS-Enabled

Systems

- C/C++ automatic code generation

- SIL with ROS Enabled Systems

- Debug C/C++ with MATLAB Engine

Prototype

RAPID ITERATIVE PROCESS

Design independent of

communication

framework!

12

Simulate & Prototype

Robotics Development Workflow with MATLAB and ROS

Visualize

Experiment

Test

13

Let’s solve a real problem: Sign Detection System

Clearpath Husky Robot

- ROS Enabled

- Kinect (RGB and Point cloud)

- Hokuyo 2D Lidar

Track and Park

Reduce Speed

Collision Avoidance

Simulation model in Gazebo

14

System Level Design ROS as CommunicationFramework

Object ClassifierObstacle Avoidance

State Machine

15

Sign Recognition with Collision Avoidance

16

Robotics Development Workflow with MATLAB and ROS

- Import image data from ROS bags

- Threshold and detect features in images

- Train a classifier

- Test algorithm

Implementation

Product

Requirements

Idea

Design and

Simulation

Prototype

RAPID ITERATIVE PROCESS

17

Importing Simulation and Experimental Data

Robotics System Toolbox™

Import ROS Data

Filter your logged field

data by topic or time

intervalExperimental data or

simulation Data

Data ready to

design algorithms

18

Visualize, Analyze, and Process Data: Classifier

OutputInput

stop

membrane

speedlimit

neg

Training data Preprocessing Feature Extraction Training

Classifier

% Detect red regions

BW = createMask(videoFrame);

% Fill image regions

BW = imfill(BW,'holes');

% Get bounding boxes

stats = regionprops('table',BW,'BoundingBox','Area');

% Filter based on area size

targetIndex = stats.Area > 500;

% Get bounding boxes from detected regions

testFeatures(k,:) = extractHOGFeatures(Icr);

19

Visualize, Analyze, and Process Data: Classifier

Statistics and Machine Learning Toolbox™Computer Vision System Toolbox™

OutputInput

stop

membrane

speedlimit

neg

Training data Preprocessing Feature Extraction Training

ClassifierClassifier

Preprocessing Feature Extraction

20

Design and Test Algorithm

21

Design and Test Algorithm

22

Robotics Development Workflow with MATLAB and ROS

- Test algorithm with an external simulator

- Tune your algorithm

- Integrate with other algorithms

Implementation

Product

Requirements

Idea

Design and

Simulation

Prototype

RAPID ITERATIVE PROCESS

23

MATLAB and Simulink connect to the ROS network

- Multiple master support

- ROS publishers/subscribers

- ROS services

- ROS TF tree

- ROS Parameter server

?

Vehicle

24

Co-simulation with ROS

25

Co-simulation with ROS

26

Leverage Built-in Algorithms in MATLAB

0.2 𝑠 0.2 𝑠 0.2 𝑠

time

% Sensor fusion and pose estimation

>> pf = robotics.ParticleFilter;

% Robot localization

>> mcl = robotics.MonterCarloLocalization;

% Loops at system time or ROS time

>> r = robotics.Rate(5);

% Obstacle Avoidance

>> vfh = robotics.VectorFieldHistogram;

27

Optional Products for Robotics Applications Development

Image Processing

Toolbox™

Contrast adjustment

Geometric transformations

Various filters

Segmentation

Object analysis

Computer Vision System Toolbox™

High-speed video I/O

Point Cloud processing

Tracking

Stereovision

Image Acquisition Toolbox™

Image capture from standard H/W

Analog, Camera Link, DCAM,

GigE Vision, USB camera, etc

Microsoft Kinect Support

Statistics and Machine Learning

Toolbox™

Multivariate statistics

Probability distribution

Machine learning

Experimental design

Statistical process control

28

Optional Products for Robotics Applications Development

Control System Toolbox™

Linear analysis

Classical

control design

Modern

control design

Simulink Design Optimization™

Model parameter estimation from

test data

Optimization of

parameters

Response

optimization

Simulink Control Design™

Automatic tuning of PID

Controller blocks

Linearization of

Simulink models

Continuous-

Discrete time

conversions

Robust Control Toolbox™

Robust control design

Automatic tuning

of gain-scheduled

controllers

29

Final Design

30

Robotics Development Workflow with MATLAB and ROS

System level design to target a different

middleware or framework

Implementation

Product

Requirements

Idea

Design and

Simulation

Prototype

RAPID ITERATIVE PROCESS

31

Deployment

Generate ROS

Node with

Simulink and

Embedded

Coder™

Create a

Stand Alone

Executable

with MATLAB

Compiler™

Generate a

shared library

with MATLAB

Coder™

Determine deployment methods based on application

32

Implementation

34

Takeaways

1. Robotics development workflow with MATLAB and Simulink for autonomous driving with ROS.

2. Rapid iterative process based on powerful visualization tools, interactive apps, built-in algorithms,

debugging capabilities.

3. Design is independent of communication framework.

Call to Action! Check Demo

35

% Thank you