aerial robot swarms - dtcvijay kumar is the ups founda-tion professor and the deputy dean for...
TRANSCRIPT
1!1
Vijay Kumar!UPS Foundation Professor!
University of Pennsylvania!
Aerial Robot Swarms!
University of Minnesota!April 16, 2014
2!
Unmanned Aerial Vehicles!
Mass!
0! 1! 10! 100! 1,000! 10,000! 100,000!
Northrop-Grumman Global Hawk (32,200 lbs)!
Boeing X-45A UCAV (12,000 lbs)!
Northrop-Grumman
Boeing X-45A UCAV (12,000 lbs)lbs)
Bell Eagle Eye (2,250 lbs) !
AscTec!Pelican (3.5 lbs)!
Bell Eagle Eye (2,250 lbs)
Hummingbird (1 lb)!Hummingbird (1 lb)Hummingbird (1 lb)
KMel kNanoQuad (0.12 lb)!
Hummingbird (1 lb)Hummingbird (1 lb)
KMelKMel kNanoQuadkNanoQuad(0.12 lb)(0.12 lb)
Gen. Atomics MQ-9 Reaper (10,000 lbs)!
Gen. Atomics Predator (7,000 lbs)!Gen. Atomics Predator (2,250
lbs)!
Images from www.af.mil
Boeing Scaneagle (40 lbs)!
3!
Unmanned Aerial Vehicles!
!!
!> $10B industry!
!" Military: Surveillance, force protection, warfare (> 75 countries)!
!" Civilian commercial: Transportation, agriculture, mining, security!
!" Civilian private: DIY Drones!
!!
!
!!
!!
Increasingly smaller, lower power and inexpensive inertial sensing, GPS, and computing
1!
10!
100!
1000!
10000!
1980! 1990! 2000! 2010!
Number of UAVs worldwide!
4!
Search and Rescue!
Michael, et al, J. Field Robotics, 2012!
5!
Security and First Response!
6!
Security and First Response!Flight through building!Flight through building
7!
Real Time Agrylitics!
8
Apple Orchards, Pennsylvania State University Biglerville, PA
9!
10
Quadrotor
11
Pico 11 cm 20 g, 6.5 Watts Max speed 6m/s
12
$10M
$1M
$100
$10
$1
Global Hawk
Predator, Reaper
Scan Eagle, Raven
Hobby Kits
Toys
Chips
$100K
$1K-10K? Autonomous, agile micro aerial vehicles
our focus
GPS, airbag sensors, processors
DIY Drones, Kick starter projects
10s
100s
1,000,000s
100,000s
10,000s
13!
Length Scales!
13
!"
!#$"
!#%"
!#&"
!#'"
!#("
!#)"
!#*"
!#+"
!" !#!%" !#!'" !#!)" !#!+" !#$" !#$%" !#$'"
,-./"012345"
64078/"92:"
;<3<344"
=4>?.-0"
@122?073?5A"
B-0<"=>1C"
B-0<"
==;""
=?.<"
3-D printing
Folding (w MIT)
Pop-up book MEMS (Wood) 3-D printing
v ⇠p
L
14!
[Kushleyev, Mellinger and Kumar, RSS 2012]!
The KMel Nano Quadrotor!
15!
16!
Size does Matter!
a � 1R
, � � 1R2a � 1
R, � � 1
R2
111linear acceleration
11angular acceleration
mass M inertia I
r
R
thrust f
17!17
1 Dynamics, Control and Planning!2 State Estimation (Autonomy)!
3 Swarm technology!
Research Challenges!
18
Quadrotor
19!
x
X
Y
Z
r !1"
!2"
!3"
!4"
y
#, yaw!
$, pitch!%, roll!
x, y, z position!
Underactuated
z Control!
#, yaw
$, pitch%, roll
R
20!
x
X
Y
Z
r !1"
!2"
!3"
!4"
y
#, yaw!
$, pitch!%, roll!
x, y, z position!
Underactuated
z Control!
#, yaw
$, pitch%, roll
R
21!
Differential Flatness!
[x, y, z, #]!
A
B
A B
B
B
Minimum snap trajectory!
[Mellinger and Kumar, 2011]!
2
6666666666666666664
x
y
z
✓
�
x
y
z
✓
�
3
7777777777777777775
min�(t)
Z T
0↵k....r (t)k2 + � (t)2dt
SE(3)
22!
Control on SE(3)!Specified trajectory!
�(t) : [0, T ]⇥ R3 � SO(2)
x
y
z
t t
Rdese3 =t�t�
� = �des
Rdes
#des"
M = � ⇥ I� + I(�kReR � k�e�)
eR(Rdes, R)
k
f = m (r + Kver + Kper + g). k
[Wen and Kreutz-Delgado, 1991; Lee et al, 2011; Mellinger and Kumar, 2011]!
23!
Software Architecture!
[Kraft 2003; Mellinger, Michael, and Kumar 2010; Mellinger and Kumar 2011]!
Rigid body dynamics
Motor controller Attitude
controller
Position controller
Trajectory Planner
Rigid body dynamics controller Attitude controller
Rdes
&des
R, '!r, r
f
M Attitude
controller controller RRRdesdes
R, , 'r, r
M M M
Basin of attraction almost all of SO(3) !
~0.001 s!
~0.01 s!
~0. 1 s!
24!
Roll and Pitch!
25!
Robust, nonlinear controller!
tr[I � (Rdes)T R] < 2 ⇤e�(0)⇤2 ⇥ 2�min(I)
kR
⇤1� 1
2tr
�I � (Rdes)T R
⇥⌅
26!
Translation!
27!
[Mellinger and Kumar, ICRA 2011]!
Minimum Snap Trajectories!
A
C
B
28!
Minimum Snap Trajectories!
[Mellinger and Kumar, ICRA 2011]!
29![Thomas et al, ICRA 2014]!
Aerial Grasping and Manipulation !
30
motion capture cameras
reflective markers
Sensing
31 Photograph by Joe McNally
32!32
Unreliable GPS!
33!33
No GPS!
34!34
1 Dynamics, Control and Planning!2 State Estimation (Autonomy) !
3 Swarm technology!
35!
Microsoft Kinect!
Hokuyo!Laser Scanner!
Operation in Unstructured Environments!
36!
�x
d1d2
d3
d03
d02
d01
Pillars!
Robot!
(x1, y1) (x3, y3)(x2, y2)
Concurrently estimate !!Locations of pillars (6)!!Displacement of the robot (2)!
Simultaneous Localization !And !Mapping!
New robot position!
Simultaneous Localization !And !Mapping!
37
• 1.8 GHz Core i3 processor, 8 GB RAM
• u- blox LEA-6T GPS module • Hokuyo UTM-30LX LiDAR • 2 mvBlueFOX-MLC200w grayscale
HDR cameras • (fisheye lenses, 752 × 480, 25 Hz) • IMU 100 Hz
[Shen, Mulgaonkar, Michael, and Kumar 2014]
38
Heterogeneous Sensors “Proprioceptive” Sensors
● Inertial Measurement Unit
“Absolute” Sensors
● GPS, Magnetometer
● Altimeter
“Relative” Sensors ● Laser scanner
● Cameras
zt = h(xt) + nt
zt = h(xt, . . . ,xt�k) + nt
xt = f(xt�1,ut�1) + nt
39!
f2!
f1! f0!f0f0f
Features in environment!
x0!
x1!
f3! f4!
x2!
x3!
f6!
f3f3fz00
z10
z20 z2
1z11
z32
z33
z23
z43
z63
x1 = f(x0, u01,�t01)
x2 = f(x1, u12,�t12) x3 = f(x2, u23,�t23)
Robot!Pose!
Simultaneous !Localization !And !Mapping!
Simultaneous !Localization !And !Mapping!
40!
Map Refine (20 Hz)
Pose Graph SLAM
position
Multi-Sensor Unscented
Kalman Filter (100 Hz)
velocity
Estimation and Control Architecture!
Trajectory Generator (20 Hz)
Planner (20 Hz)
User Interface
Velocity estimator
Visual odometry
Laser odometry
Altitude estimator
Downward Camera (30Hz)
Stereo Camera (25 Hz)
IMU (100 Hz)
Pressure Altimeter (20 Hz)
Laser Scanner (20 Hz)
GPS (10Hz)
Controller (100 Hz)
41!
Onboard State Estimation!
[Shen, Michael, and Kumar 2011]!
IMU, Laser scanner, and camera!
Auton Robot
Lee, T. (2011). Geometric tracking control of the attitude dynamics ofa rigid body on SO� 3� . In Proc. of the Amer. control conf., SanFrancisco, CA.
Lee, T., Leok, M., & McClamroch, N. H. (2010). Geometric trackingcontrol of a quadrotor UAV on SE � 3� . In Proc. of the IEEE conf.on decision and control, Atlanta, GA.
Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory genera-tion and control for quadrotors. In Proc. of the IEEE intl. conf. onrobot. and autom., Shanghai, China.
Mellinger, D., Michael, N., & Kumar, V. (2010). Trajectory generationand control for precise aggressive maneuvers with quadrotors. InProc. of the intl. sym. on exp. robot., Delhi, India.
Mesbahi, M. (2005). On state-dependent dynamic graphs and theircontrollability properties. IEEE Transactions on Automatic Con-trol, 50(3), 387–392.
Michael, N., Mellinger, D., Lindsey, Q., & Kumar, V. (2010). TheGRASP multiple micro UAV testbed. IEEE Robotics & Automa-tion Magazine, 17(3), 56–65.
Nieuwstadt, M. J. V., & Murray, R. M. (1998). Real-time trajectorygeneration for differentially flat systems. International Journal ofRobust and Nonlinear Control, 8(11), 995–1020.
Ogren, P., Fiorelli, E., & Leonard, N. (2002). Formations with a mis-sion: stable coordination of vehicle group maneuvers. In Proc.of intl. sym. on mathematical theory networks and syst., NotreDame, IN.
Olfati-Saber, R., & Murray, R. M. (2002). Distributed cooperative con-trol of multiple vehicle formations using structural potential func-tions. In Proc. of the IFAC world congress, Barcelona, Spain.
Olfati-Saber, R., & Murray, R. M. (2004). Consensus problems in net-works of agents with switching topology and time-delays. IEEETransactions on Automatic Control, 49(9), 1520–1533.
Shim, D., Kim, H., & Sastry, S. (2003). Decentralized nonlinear modelpredictive control of multiple flying robots. In Decision andcontrol, 2003. Proceedings. 42nd IEEE conference on (Vol. 4,pp. 3621–3626). New York: IEEE.
Tabuada, P., Pappas, G. J., & Lima, P. (2001). Feasible formations ofmulti-agent systems. In Proc. of the Amer. control conf., Arling-ton, VA (pp. 56–61).
Tanner, H., Pappas, G. J., & Kumar, V. (2002). Input-to-state stabilityon formation graphs. In Proc. of the IEEE intl. conf. on robot. andautom., Las Vegas, NV (pp. 2439–2444).
Turpin, M., Michael, N., & Kumar, V. (2011). Trajectory design andcontrol for aggressive formation flight with quadrotors. In Proc.of the intl. sym. of robotics research, Flagstaff, AZ.
Vicsek, T., Czirók, A., Ben-Jacob, E., Cohen, I., & Shochet, O. (1995).Novel type of phase transition in a system of self-driven particles.Physical Review Letters, 75(6), 1226–1229.
Matthew Turpin is a Ph.D. candi-date in the Department of Mechan-ical Engineering and Applied Me-chanics at the University of Penn-sylvania. He works on formation de-sign and control of micro-aerial ve-hicles.
Nathan Michael is a Research As-sistant Professor in the Departmentof Mechanical Engineering and Ap-plied Mechanics at the University ofPennsylvania. He received his Ph.D.in Mechanical Engineering from theUniversity of Pennsylvania in 2008.He works in the areas of dynamics,estimation, and control for groundand aerial robot systems.
Vijay Kumar is the UPS Founda-tion Professor and the Deputy Deanfor Education in the School of En-gineering and Applied Science atthe University of Pennsylvania. Hereceived his Ph.D. in MechanicalEngineering from The Ohio StateUniversity in 1987. He has beenon the Faculty in the Departmentof Mechanical Engineering and Ap-plied Mechanics with a secondaryappointment in the Department ofComputer and Information Scienceat the University of Pennsylvaniasince 1987. He is a Fellow of the
American Society of Mechanical Engineers (ASME) and the Institu-tion of Electrical and Electronic Engineers (IEEE).
42!
Autonomous Exploration!IMU, Laser scanner, and RGBD camera!
43
44!
Lithium Polymer Batteries!
0!
200!
400!
600!
800!
1000!
1200!
1400!
0! 50! 100! 150! 200! 250! 300! 350! 400! 450! 500!
Spec
ific
Pow
er (W
/kg)!
Specific Energy (Whr/kg)!
Quadrotors with 10-20 min endurance!
[B. Morgan, ARL and Y. Mulgaonkar, Penn]!
Bolt (10 secs)!
Armstrong (20 mins)!
350
Specific Energy (Whr/kg)[B. Morgan, ARL and Y.
350
Specific Energy (Whr/kg)[B. Morgan, ARL and Y.
1000
Spec
ific
Pow
er (W
/kg)
1000
Spec
ific
Pow
er (W
/kg) Loose
weight!!
45!
Power!
0!
200!
400!
600!
800!
1000!
1200!
1400!
0! 50! 100! 150! 200! 250! 300! 350! 400! 450! 500!
Spec
ific
Pow
er (W
/kg)!
Specific Energy (Whr/kg)!
10,000 Whr/kg
Lithium Polymer Batteries!
Quadrotors with 10-20 min endurance!
[B. Morgan, ARL and Y. Mulgaonkar, Penn]!
Bolt (10 secs)!
Armstrong (20 mins)!
350
Specific Energy (Whr/kg)[B. Morgan, ARL and Y.
350
Specific Energy (Whr/kg)[B. Morgan, ARL and Y.
1000
Spec
ific
Pow
er (W
/kg)
1000
Spec
ific
Pow
er (W
/kg) Loose
weight!!
46
Reducing the Payload CPU: Intel Atom Processor, 1.6 GHz, 1 GB Ram Sensing: 2 grayscale Matrix Vision cameras, 376x240 + IMU
Weight: 740gram
Power: ~120 W
2012
[Shen, Mulgaonkar, Michael, and Kumar 2013]
47
Vision + IMU State Estimation ● Estimate rotations by tracking features ● Assuming features are known, estimate position ● Estimate scale with second camera
● Assuming pose is known, map features
48!
Vision + IMU State Estimation!
49![Shen, Mulgaonkar, Michael, and Kumar, 2013] ç !
Fast (4 m/s), Indoor Outdoor, foliage
Indoor, 3-D Indoor/outdoor, visual SLAM
50!
51!51
1 Dynamics, Control and Planning!2 State Estimation !
3 Swarm technology!
52!
Collaboration in Small Teams!
[Lindsey, Mellinger, and Kumar, 2012]
53!
1 Act independently
2 Require only local information
3 Anonymous behavior
54!
robot i
robot j
g(t) 2 SO(2)⇥R3
si,j(t) = xj(t)� xi(t)
Leader-Follower Networks!
[Desai, Ostrowski, and Kumar, 1999; Turpin, Michael and Kumar, 2011]
57!
Control of Formation Shape and Group Motion!
(Turpin, Michael, and Kumar, 2013)!
58!
[Kushleyev, Mellinger and Kumar 2012]
Control of Formation Shape and Group Motion!
59!
Search and Rescue!
[Michael et al, 2012]!
N. Michael, S. Shen, K. Mohta, Y. Mulgaonkar, V. Kumar, K. Nagatani, Y. Okada, S. Kiribayashi, K. Otake, K. Yoshida, K. Ohno, E. Takeuchi, and S. Tadokoro, “Collaborative mapping of an earthquake-damaged building via ground and aerial robots,” J. Field Robotics, vol. 29, no. 5, pp. 832–841, 2012.!
60!7th, 8th, and 9th floors!
61!
Thank you! The contributions and collaborations of the fo l low ing co l leagues are g race fu l l y acknowledged: A. Wolisz, K. Pister, R. Brodersen, E. Alon, G. Kelson, A. Niknejad, B. Nikolic, J. Wawrzynek, P. Wright, D. Tse, M. Maharbiz, J. Carmena, R. Knight, L. Van de Perre, and B. Gyselinckx, R. Muller, M. Mark, D. Chen, A. Parsha, S. Gambini, and all my current and past graduate students. Contributions by the BWRC member companies and the MuSyC and GSRC consortia are truly appreciated. Many thanks to V. Vinge, N. Stephenson, I. Asimov, and C. Stross for providing the true visions!