designing a pick and place robotics application using matlab … · 26 today: design pick and place...
TRANSCRIPT
![Page 1: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/1.jpg)
1© 2017 The MathWorks, Inc.
Designing a Pick and Place Robotics
Application Using MATLAB and Simulink
Carlos Santacruz-Rosero, PhD
Sr Application Engineer – Robotics
Pulkit Kapur
Sr Industry Marketing Manager– Robotics
![Page 2: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/2.jpg)
2
Key Takeaway of this Talk
Success in developing an autonomous robotics system requires:
– Multi-domain simulation
– Great tools which make complex workflows easy and
integrate with other tools
– Model-based design
![Page 3: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/3.jpg)
3
Challenges with Autonomous Robotics Systems
Applying Multidomain Expertise
Technical Depth and System Stability
IP Protection
End-to-End workflows
Complexity of Algorithms
![Page 4: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/4.jpg)
4
What does success look like?
![Page 5: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/5.jpg)
5
![Page 6: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/6.jpg)
6
![Page 7: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/7.jpg)
7
![Page 8: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/8.jpg)
8
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 9: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/9.jpg)
9
Another Example: Self-Driving Cars
Planning
Localization Obstacle
avoidance
Global Map
Following
Steering
Actuator ECUs
Accelerator
Braking
Motion Controllers
Control
LIDAR
RADAR
GPS/IMU
Camera Platform
Sense
Perceive Communication Plan
Deep
learning
![Page 10: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/10.jpg)
10
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 11: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/11.jpg)
11
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 12: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/12.jpg)
12
Platform Design
How to create a model of my system that suits my needs?
ActuatorsMechanics Environment
![Page 13: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/13.jpg)
13
SolidWorks Model Simscape Multibody Model
Mechanics: Import models from common CAD Tools
![Page 14: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/14.jpg)
14
Mechanics: One line import from URDF
%% Import robot from URDF
smimport('j2n6s300_standalone_stl.urdf');
![Page 15: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/15.jpg)
15
Rigid Body Tree Dynamics
▪ Specify rigid body inertial properties
▪ Compute for the rigid body tree
– Forward dynamics
– Inverse dynamics
– Mass matrix
– Velocity product
– Gravity torque
– Center of mass position and Jacobian
Compute rigid body tree dynamics quantities
» load exampleRobots.mat
» lbr.DataFormat = 'column';
» q = lbr.randomConfiguration;
» tau = inverseDynamics(lbr, q);
![Page 16: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/16.jpg)
17
Actuators: Connect Motors
![Page 17: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/17.jpg)
18
Actuators: Model other domains
![Page 18: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/18.jpg)
21
Environment: Connect to an external robotics simulator
![Page 19: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/19.jpg)
22
Synthetic
data
Logged
multi-sensor
data
Algorithm
Create new scenario or refine sensor model
Collect lab data
Refine algorithm
yes
no
Expected
Behavior
C Code
Generate code
Integrate
with embedded
environment
Environment: Connect to an external robotics simulator
![Page 20: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/20.jpg)
23
Environment: Connect MATLAB and Simulink with ROS
MATLAB Code
SM Models
Built-in
algorithms
Robot
ROS node
Simulation
environment
Networking
Code Generation
ROS Bag import
![Page 21: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/21.jpg)
25
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 22: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/22.jpg)
26
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
PlatformSupport for Common Sensors
Cameras, Laser Scanners, Optical Encoders, IMU, GPS.
Image analysis, including segmentation, morphology,
statistics, and measurement
Apps for image region analysis, image batch processing,
and image registration
Image enhancement, filtering, geometric transformations,
and deblurring algorithms
Intensity-based and non-rigid image registration.
Visualizing Point Clouds
To visualize a point cloud in MATLAB, use the
showPointCloud and scatter3 command.
![Page 23: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/23.jpg)
27
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 24: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/24.jpg)
28
Sign Detector and Classifier
Sign Classifier
Images Labels
![Page 25: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/25.jpg)
29
MATLAB makes machine learning easy and accessible
Traditional Machine Learning approach
Machine
Learning
ClassificationTraditional Feature Extraction
Boy
Dog ✓
Bicycle
Deep Learning approach
…
𝟗𝟓%𝟑%
𝟐%
Boy
Dog ✓
Bicycle
Convolutional Neural Network (CNN)
Learned features
End-to-end learning
Feature learning + Classification
![Page 26: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/26.jpg)
30
Complex workflows made easy with MATLAB
stop
membrane
speedlimit
neg
Training data Preprocessing Feature Extraction Training
Classifier
% Detect red regions
BW = createMask(videoFrame);
% Fill image regions
BW = imfill(BW,'holes');
% Get bounding boxes
stats = regionprops('table',BW,'BoundingBox','Area');
% Filter based on area size
targetIndex = stats.Area > 500;
% Get bounding boxes from detected regions
testFeatures(k,:) = extractHOGFeatures(Icr);
![Page 27: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/27.jpg)
31
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 28: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/28.jpg)
32
Planning
[𝑥𝑏 𝑦𝑏 𝜃𝑏]
Path Planner
MapInitial PoseFinal Pose
Path
[𝑥𝑎 𝑦𝑎 𝜃𝑎]
![Page 29: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/29.jpg)
33
Explore Built In Functions: PRM Planner
%% Create PRM path planner
planner = robotics.PRM;
planner.Map = og;
planner.NumNodes = 600;
planner.ConnectionDistance = 5;
show(originalOg); hold on;
show(planner, 'Map', 'off');
%% Create map as Binary Occupancy Grid
robotics.BinaryOccupancyGrid(binaryImage,20);
og.GridLocationInWorld = [-10.5, -10.5];
figure;
show(og);
![Page 30: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/30.jpg)
34
Today: Design Pick and Place Application
Sense
Perceive
Plan &
Decide
Control
Platform
![Page 31: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/31.jpg)
35
Control of Manipulator Arms
![Page 32: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/32.jpg)
36
Model-Based Design for Motion Control
▪ Requirements traceability
▪ Early verification of requirements
▪ Automatic code generation.
▪ Automatic report generation
▪ Test automation
![Page 33: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/33.jpg)
37
Key Takeaway of this Talk
Success in developing an autonomous robotics system requires:
– Multi-domain simulation
– Great tools which make complex workflows easy and
integrate with other tools
– Model-based design
![Page 34: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/34.jpg)
38
ChallengeDevelop control systems for a two-armed mobile
humanoid robot with 53 degrees of freedom
SolutionUse Model-Based Design with MATLAB and Simulink
to model the controllers and plant, generate code for
HIL testing and real-time operation, optimize
trajectories, and automate sensor calibration
Results▪ Programming defects eliminated
▪ Complex functionality implemented in hours
▪ Advanced control development by students
enabled
German Aerospace Center (DLR) Robotics and
Mechatronics Center Develops Autonomous
Humanoid Robot with Model-Based Design
DLR’s humanoid robot Agile
Justin autonomously performing
a complex construction task.
Link to user story
“Model-Based Design and automatic
code generation enable us to cope
with the complexity of Agile Justin’s
53 degrees of freedom. Without
Model-Based Design it would have
been impossible to build the
controllers for such a complex
robotic system with hard real-time
performance.”
Berthold Bäuml
DLR
![Page 35: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/35.jpg)
39
Festo Develops Innovative Robotic Arm
Using Model-Based Design
ChallengeDesign and implement a control system for a pneumatic
robotic arm
SolutionUse Simulink and Simulink PLC Coder to model,
simulate, optimize, and implement the controller on a
programmable logic controller
Results▪ Complex PLC implementation automated
▪ Technology and innovation award won
▪ New business opportunities opened
“Using Simulink for Model-Based
Design enables us to develop the
sophisticated pneumatic controls
required for the Bionic Handling
Assistant and other mechatronic
designs. With Simulink PLC Coder, it
is now much easier to get from a
design to a product.”
Dr. Rüdiger Neumann
Festo
Link to user story
The Festo Bionic Handling
Assistant. Image © Festo AG.
![Page 36: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/36.jpg)
40
Scania Develops Advanced Emergency
Braking Systems with Model-Based
Design
Challenge
Develop an advanced emergency braking system to
reduce rear-end collisions
Solution
Use Model-Based Design to develop sensor fusion
algorithms, simulate and verify designs, and generate
code for implementation on a production ECU
Results
▪ 1.5 million kilometers of recorded sensor data
simulated in 12 hours
▪ Design changes quickly implemented
▪ Optimized production C code generated
Link to article
A controlled road test of Scania’s advanced
emergency braking system software.
“To deploy the sensor fusion
system to the ECU, we generated
C code from our Simulink model
with Embedded Coder. With code
generation, we were able to get to
an implementation quickly, as well
as avoid coding errors.”
Jonny Andersson
Scania
![Page 37: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/37.jpg)
41Mobileye chips are used in over 5.2 million vehicles
![Page 38: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/38.jpg)
42
User Story: Bipedal Robot
“When other researchers see
that we’ve gone directly from
controllers developed in
MATLAB and Simulink to a real-
time implementation with
Simulink Real-Time, they get
pretty excited. The approach we
took is now being used in other
departments at the University of
Michigan and by robotics
researchers at other universities,
including MIT and Oregon State
University.”
- Prof. Jesse Grizzle,
University of Michigan
![Page 39: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/39.jpg)
43
Preceyes
“Simulink and Simulink Real-Time enabled controls algorithm design and implementation of these
algorithms on the device,” Gerrit Naus, COO and co-founder at Preceyes.
![Page 40: Designing a Pick and Place Robotics Application Using MATLAB … · 26 Today: Design Pick and Place Application Sense Perceive Plan & Decide Control Platform Support for Common Sensors](https://reader035.vdocument.in/reader035/viewer/2022071105/5fdecd4e5b891d26ea3c33e3/html5/thumbnails/40.jpg)
46
% Thank you