robotics development workflow with matlab and simulink€¦ · robotics development workflow with...
TRANSCRIPT
![Page 1: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Robotics Development Workflow
with MATLAB and Simulink
Carlos Santacruz-Rosero, Ph.D
Sr Application Engineer - Robotics
![Page 2: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/2.jpg)
2
Agenda
• Introduction
• Advanced Robotics Systems
• Robotics Development Workflow with MATLAB and Simulink
• Takeaways
![Page 3: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/3.jpg)
3
Car as an Advanced Robotics System
LIDAR RADAR GPS/IMUCamera
Steering Actuator
Actuator ECUs
GAS/Brake Actuator
Motion Controllers
Planning
Localization Obstacle
avoidance
Global Map
Motion control
SENSE
CONTROL
PERCEIVE PLAN
![Page 4: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/4.jpg)
4
Collaborative Robot as an Advanced Robotics System
Torque TemperatureCamera RGB-D
Impedance Control
Actuator Control
Safety Guards
Motion Controllers
Motion Planning
Obstacle Avoidance Object Detection
World
Optimization
SENSE
CONTROL
PERCEIVE PLAN
![Page 5: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/5.jpg)
5
Architecture - Advanced Robotics System
Motion Planning
Obstacle Avoidance Object Detection
World
Optimization
Impedance Control
Actuator Control
Safety Guards
Motion Controllers
Middleware
Data synchronization
CPU/GPU Cluster, Server, Cloud
Real-time requirements
Microcontroller, FPGA, Real-time PC
![Page 6: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/6.jpg)
6
Technology Solutions for Autonomous SystemsP
ER
CE
IVE
SE
NS
EC
ON
NE
CT
PL
AN
CO
NT
RO
L
Computer Vision Phased Array Statistics & Machine Learning
Control System Tbx
StateflowRobotics System Tbx
Data Acquisition TbxHW Support Packages
Simscape Toolboxes
Communications Tbx WLAN System Toolbox
Simulink Real-Time
Robotics System Tbx
![Page 7: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/7.jpg)
7
Success Stories
![Page 8: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/8.jpg)
8
Technology Solutions for Autonomous SystemsP
ER
CE
IVE
SE
NS
EC
ON
NE
CT
PL
AN
CO
NT
RO
L
Computer Vision Phased Array Statistics & Machine Learning
Control System Tbx
StateflowRobotics System Tbx
Data Acquisition TbxHW Support Packages
Simscape Toolboxes
Communications Tbx WLAN System Toolbox
Simulink Real-Time
Robotics System Tbx
![Page 9: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/9.jpg)
9
Robotics System Toolbox
Environment for prototyping, simulating, and deploying robotics applications
Interface and
Deployment to
ROS
Algorithms for
Manipulators and
Humanoids
Algorithms for
Mobile
Robotics
![Page 10: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/10.jpg)
10
Robotics Applications with Robotics System Toolbox
![Page 11: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/11.jpg)
11
Workflow Convergence is Needed
Implementation
- Trace to requirements
- Design/debug algorithm
- System integration
- Testing on physical system
- Manual Coding
- Verification and Validation
System
Requirements
Idea
![Page 12: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/12.jpg)
12
Robotics Development Workflow with MATLAB and Simulink
Implementation
System
Requirements
Idea
Design and
Simulation
- Built-in algorithms and apps
- System-Level Simulation MBD
- Co-simulation
- C/C++ automatic code generation
- Processor-in-the-loop (PIL)
- Debug C/C++ with MATLAB Engine
Prototype
RAPID ITERATIVE PROCESS
Design independent of
hardware
implementation!
![Page 13: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/13.jpg)
13
Let’s solve a real problem: Sign Detection System
Clearpath Husky Robot
- ROS Enabled
- Kinect (RGB and Point cloud)
- Hokuyo 2D Lidar
Track and Park
Reduce Speed
Collision Avoidance
Simulation model in Gazebo
![Page 14: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/14.jpg)
14
System Level Design ROS as CommunicationFramework
Object ClassifierObstacle Avoidance
State Machine
![Page 15: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/15.jpg)
15
Sign Recognition with Collision Avoidance
![Page 16: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/16.jpg)
16
Robotics Development Workflow with MATLAB and ROS
- Import logged data
- Train a classifier
- Test component
Implementation
Product
Requirements
Idea
Design and
Simulation
Prototype
RAPID ITERATIVE PROCESS
![Page 17: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/17.jpg)
17
Importing Simulation and Experimental Data
Robotics System Toolbox™
Import ROS Data
Filter your logged field
data by topic or time
intervalExperimental data or
simulation Data
Data ready to
design algorithms
![Page 18: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/18.jpg)
18
Visualize, Analyze, and Process Data: Classifier
OutputInput
stop
membrane
speedlimit
neg
Training data Preprocessing Feature Extraction Training
Classifier
% Detect red regions
BW = createMask(videoFrame);
% Fill image regions
BW = imfill(BW,'holes');
% Get bounding boxes
stats = regionprops('table',BW,'BoundingBox','Area');
% Filter based on area size
targetIndex = stats.Area > 500;
% Get bounding boxes from detected regions
testFeatures(k,:) = extractHOGFeatures(Icr);
![Page 19: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/19.jpg)
19
Visualize, Analyze, and Process Data: Classifier
Statistics and Machine Learning Toolbox™Computer Vision System Toolbox™
OutputInput
stop
membrane
speedlimit
neg
Training data Preprocessing Feature Extraction Training
ClassifierClassifier
Preprocessing Feature Extraction
![Page 20: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/20.jpg)
20
Design and Test Algorithm
![Page 21: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/21.jpg)
21
Design and Test Algorithm
![Page 22: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/22.jpg)
22
Robotics Development Workflow with MATLAB and ROS
- Test algorithm with an external simulator
- Tune your algorithm
- Integrate with other components
Implementation
Product
Requirements
Idea
Design and
Simulation
Prototype
RAPID ITERATIVE PROCESS
![Page 23: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/23.jpg)
23
MATLAB and Simulink connect to the ROS network
- Multiple master support
- ROS publishers/subscribers
- ROS services
- ROS TF tree
- ROS Parameter server
?
Vehicle
![Page 24: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/24.jpg)
24
Co-simulation with ROS
![Page 25: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/25.jpg)
25
Co-simulation with ROS
![Page 26: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/26.jpg)
26
Robotics Development Workflow with MATLAB and ROS
System level design to target a different middleware or framework
Implementation
Product
Requirements
Idea
Design and
Simulation
Prototype
RAPID ITERATIVE PROCESS
![Page 27: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/27.jpg)
27
Deployment
Generate ROS
Node with
Simulink and
Embedded
Coder™
Create a
Stand Alone
Executable
with MATLAB
Compiler™
Generate a
shared library
with MATLAB
Coder™
Determine deployment methods based on application
![Page 28: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/28.jpg)
28
Implementation
![Page 29: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/29.jpg)
29
Part1: Object Detection and Position Estimation
Object Detection and
3D Object position
estimation
![Page 30: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/30.jpg)
30
Part2: Trajectory Planning
Optimization-based
Trajectory Planning
![Page 31: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/31.jpg)
31
Part3: System Integration
System Integration and Control
Logic Design
![Page 32: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/32.jpg)
32
Advanced Robotics Application Requires Multiple Technologies
Trajectory Planning
System Integration
Detect and Estimate Position
of ObjectsComputer Vision
Machine Learning
Control Logic
Co-simulation
Code generation
Robotics
Optimization
MATLAB and Simulink: very powerful tools to design advanced robotics applications
![Page 33: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/33.jpg)
33
Trajectory Planning with RGB-D Sensor
![Page 34: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/34.jpg)
34
System Level Design
![Page 35: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/35.jpg)
35
Robotics Development Workflow with MATLAB and Simulink
Visualize
Experiment
Prototype
Simulate
Deploy
Middleware
![Page 36: Robotics Development Workflow with MATLAB and Simulink€¦ · Robotics Development Workflow with MATLAB and Simulink Carlos Santacruz-Rosero, Ph.D Sr Application Engineer - Robotics](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4a29aa8ec9721b1b36db03/html5/thumbnails/36.jpg)
36
Questions
% Thank you