a city driving robotic car named “tommy jr.” · a city driving robotic car named “tommy...
Post on 11-Aug-2020
6 Views
Preview:
TRANSCRIPT
A City Driving Robotic Car Named “Tommy Jr.”
Paul J. PerroneCEO – Perrone RoboticsTeam Lead – Team Jefferson
TS-5925
2008 JavaOneSM Conference | java.sun.com/javaone | 2
Learn about a robotic car in a historic city driving race and the Java™ technology that got him there
2008 JavaOneSM Conference | java.sun.com/javaone | 3
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 4
The DARPA Grand Challenge and Tommy
175+ mile race through desertNo driver onboard or in remote control• Completely Autonomous Ground Vehicles (AGVs)
$2M prizeApplications Video Site Visit NQEs/DGC“Tommy” is born and advances100% Java technology based bot• Java SE platform for navigation & obstacle avoidance• Java ME platform for controls
2008 JavaOneSM Conference | java.sun.com/javaone | 5
The DARPA Urban Challenge and Tommy Jr.
60+ mile through a cityAGV city driving• Navigation & obstacle avoidance• Plus all rules involved with city driving
$2M/$1M/$500K prizes$1M grants (“Track A”) & Indies (“Track B”)Applications Video Site Visit NQEs/UCE“Tommy Jr.” is born and advances – Track B100% Java technology based bot• Java SE technology & Java ME technology again• Java technology RTS for real-time nav controls• SunSPOT safety controller
2008 JavaOneSM Conference | java.sun.com/javaone | 6
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 7
Automotive Platform: Affordable & Practical
Scion xB• Used 2004 = “affordable”• Small footprint = “smaller collision surface”• Navigable = “more responsive collision avoidance”• Boxy = “plentiful/easier external mounting”
+ “plentiful/easier internal mounting”Actuation integration & drop-in• Automatic/shifter actuator• Cruise control drop-in• Brake actuator• Steering motor
Tapped• Throttle position• Speedometer• Turn signals
2008 JavaOneSM Conference | java.sun.com/javaone | 8
Hardware: COTS & Java technology capableEmbedded x86• All of TJr’s decision making
and sensor fusion• Java SE technology &
Java RTS technology • MAX Standard & Real-Time
Desktop-grade x86• Front-end UI to TJr• Stereo camera processing• Java SE technology;
MAX StandardJStamp micro-controllers• Feedback control of motors• Java ME technology;
MAX MicroSunSPOT• Safety controller• Java ME technology; MAX Micro
Misc: redundant I/O controller, motor drivers, etc…
LAN
RADAR
FeedbackEncoders
VehicleController
Motor Drivers
Motor Motor Motor Motor
Steering Shifting Throttle Brake
INSSensing
GPSPositioning
SideLADARsLADARs
Camera
VehicleState
Sensors
VehicleState
Sensors
Electronics Interface
VehicleController
WatchdogController
E-stop Electronics
IgnitionEbrake
AuxProcessor
ESTOPReceiver
MainProcessor
UserInterface
2008 JavaOneSM Conference | java.sun.com/javaone | 9
Sensors: COTS & Precision
GPS – precision positioning• Standard and differential GPS• Hi-precision GPS – 2 to 4 inches
INS – Inertial Navigation System – precision orientation• Heading, pitch, roll• GPS backup
LADAR – laser range sensors – obstacle detection• SICK LMS sensors – one of our strongest capabilities• Detection ahead, on sides, and behind• Virtual “curb feeler”• Low hanging obstacle detection – last minute
Stereo camera – lane detection• Rapid integration• Incomplete usage
RADAR – long range warnings• Integrated, minimal utility
LAN
RADAR
FeedbackEncoders
VehicleController
Motor Drivers
Motor Motor Motor Motor
Steering Shifting Throttle Brake
INSSensing
GPSPositioning
SideLADARsLADARs
Camera
VehicleState
Sensors
VehicleState
Sensors
Electronics Interface
VehicleController
WatchdogController
E-stop Electronics
IgnitionEbrake
AuxProcessor
ESTOPReceiver
MainProcessor
UserInterface
2008 JavaOneSM Conference | java.sun.com/javaone | 10
Software: Layered Platform
Java platform, Java platform, Java platform• Java RTS API, Java SE API, Java ME API, Java SunSPOT API, Java
Comm APIMAX Robotics Platform• Real-Time, Standard, and Micro profiles• General purpose rapid integration platform• Many available drivers and libraries
MAX Sensor Drivers• GPS, INS, LADAR, Camera drivers
MAX Actuation Drivers & Frameworks• Motor & feedback controls• Analog/digital I/O
MAX-UGV• UGV framework• Extensible/configurable navigation maneuvers• Extensible/configurable obstacle sensation
Microprocessor/Micro-controller
OS/Runtime
Java
MAX
MAX-Drivers
MAX-UGV
“Tommy Jr.”
2008 JavaOneSM Conference | java.sun.com/javaone | 11
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 12
Main Processing Loops
Route planning (asynch)GPS & INS position & orientation sensing (asynch)LADAR sensing and obstacle detection (asynch)Camera sensing and lane detection (asynch)Navigation maneuvers (synch)Navigation controls (synch)Feedback controls (synch)Safety controls (synch)
GPS
INS
LADAR
Camera
ObstacleDetection
LaneDetection
NavigationControls
NavigationManeuvers
FeedbackControls
SafetyControls
RoutePlanning
2008 JavaOneSM Conference | java.sun.com/javaone | 13
Navigation Maneuvers
More driving rules and complexity• More rules for city driving (i.e. more maneuvers)• More complex steps for maneuvers (i.e. more maneuver actions)• Added navigation maneuver framework to MAX-UGV
Detection of maneuvers• Look-ahead for imminence• Activate/de-activate, pause/resume
Maneuver prioritization & exclusion• Maneuver precedence/priority
Maneuver actions• Configurable actions• Sequence of steps
Navigation maneuvers• Intersections, passing, merging, parking, zones, …
2008 JavaOneSM Conference | java.sun.com/javaone | 14
Parking Maneuver Sample
public class Parking extends ManeuverGeneric {
/** Detect imminence of maneuver and enter as appropriate **/ public boolean detectIfImminent() { … Course course = super.getMovementPlanner().getCourse(); WaypointProximity nextSpot = course.getWaypoint(NextParkingSpotArrivalPointID); startSpot = nextSpot.getWaypoint();
if(nextSpot.getDistance() < minDistanceForSpotApproach){ super.getMovementPlanner().setSpeed(spotApproachSpeed); if(!parking){ if(nextSpot.getDistance() < minDistanceForStartParking){ arrivalSpot = startSpot; initParking(); // Starts parking maneuver (parking = true) } } … }
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 15
Parking Maneuver Sample – Cont’d
// Configurable parking actions// Start approach to spotApproachSpot// Start slow down as approach spot (pump brakes, timeout, slow speed)BrakePumpAction// When within x meters of checkpoint, stopApproachCheckpoint // Once speed below certain mph and certain timeout, start applying reverseWaitForCheckpointHalt// Wait for reverse shift, then give a little gas, invert steeringWaitForReverse// Invert steering, backup toward staging point until reachedBackup// Wait, until stop. WaitForBackupHalt // Clear out those spot points that are part of route (i.e. make sure considered visited)// Then proceed out of maneuver (release steering override)WaitForDrive
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 16
Parking Maneuver Sample – Cont’d
public class ApproachCheckpoint extends ParkingAction {
public boolean execute() { if(getParking().reachedCheckpoint()){ super.getMovementPlanner().overrideSteering(); super.getMovementPlanner().setSteeringAngle(0.0); super.getMovementPlanner().setSpeed(0.0); reachedCheckpoint = true; return true; }else{ reachedCheckpoint = false; super.getMovementPlanner().setSpeed(pullInApproachSpeed); return false; } }
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 17
Obstacle Detection Maneuver
More perspectives, obstacles, and degrees of freedom• More sensor perspectives (front, sides, rear)• Moving and static obstacles• Multiple lanes and routes• Embodied obstacle reactions as MAX-UGV maneuvers
Obstacle detection• Enhancements to LADAR-based detection
Obstacle resolution• Moving obstacle additions• Had size, position, and confidence• Add speed and direction
Obstacle avoidance maneuvers• Lane collision, zone collision, passing, backup, parking…
2008 JavaOneSM Conference | java.sun.com/javaone | 18
Passing Another Vehicle Sample
public class Passing extends ManeuverGeneric {… public boolean detectIfActive() { boolean pass = passing || stoppedFollowingDuringPass || (getMovementPlanner().getClosestUnpassableDistance() <= closestDistanceForStopWhenUnpassable) || isReroutingDuringPass(); return pass; }
public boolean detectIfImminent() { boolean immin =
(getMovementPlanner().getClosestUnpassableDistance() <= closestDistanceForImminentPass)|| isReroutingDuringPass();
return immin; }}
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 19
Passing Another Vehicle Sample – Cont’d
// Initial passing impasse detection and beginning to slow downInitialImpasseStopAction// Wait for car to slow down (may exit out of maneuver in meantime)WaitForPassingSlowDownAction// Wait for car to come to complete halt (may exit out of maneuver)WaitForPassingStopAction// Wait for an opening, timeout, and begin passingWaitForOpeningAndPass// As pass, continue to check passage way, merging as needed, exit on completionHandleExtendedMerge
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 20
Feedback Controls
Navigation controls• Derived steering, speed, and shift commands
Feedback controls• Steering position control• Braking level & position control• Throttle level & position control• Shift position control
Closed-loop PID controls (actuator positioning)Speed controls (combined brake/throttle levels)
2008 JavaOneSM Conference | java.sun.com/javaone | 21
Steering Control Sample
public class SteeringControlPlan extends PlanGeneric{
public void init(){ commandSensor = (CommandSensor) super.getSensor(COMMAND_SENSOR_ID); steerControl = (FeedbackControl) super.getComponent(STEER_FEEDBACK_COMPONENT_ID); steeringFeedbackSensor = (Sensor) super.getSensor(STEERING_FEEDBACK_SENSOR_ID); steeringMotor = (Actuator) super.getActuator(STEERING_MOTOR_ID); steerControl.registerSensor(steeringFeedbackSensor); steerControl.registerActuator(steeringMotor); }
public void trigger(){ // Get commanded steering value SensorState state = commandSensor.getSensorState(); double steerValue = state.getDouble(DESIRED_STEERING_VALUE_ID);
// Set desired steering value steerControl.setDesiredPosition(steerValue); }}
Copyright 2001-2008. Perrone Robotics, Inc. All rights reserved.
2008 JavaOneSM Conference | java.sun.com/javaone | 22
Fail-Safety
Safety Risk = high energy + grand challenge + little time + little money
Back Story• Tommy Sr. smarting• E-stop of little use• Safety-critical systems knowledge
Focus on speed controls• Redundant speed sensing (2 speed inputs, 2 GPS speeds)• Redundant throttle position sensing (2 inputs)• Redundant speed calculations• Fail-safe controls
Watchdog safety• Monitor processing• Monitor sensors• Monitor other controllers• Cut throttle, stop car if issue
2008 JavaOneSM Conference | java.sun.com/javaone | 23
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 24
Tommy
Desert-ready robotic dune buggy$60K – 10 man-months software – 1 PC (Dec ’04 to Sept ’05)100% Java technology bot – decisions AND micro-controlMAX for rapid integrationMAX-UGV for navigation & obstacle avoidanceBuilding upon this software platform
2008 JavaOneSM Conference | java.sun.com/javaone | 25
Drop-In: Car + Actuators + Hardware + Sensors
Four man-month (one actual month) drop-in (Jan ’07)Power electronics: Alternator, batteries, isolation, convertersActuators: braking, steering, throttle, shifting, E-brakeActuator electronics: Motor drivers, relays, switchesFeedback: signals, encoders, electronicsE-stop electronics: Commercial E-stop, government E-stopSensors: GPS, INS, LADAR, Stereo Camera, RADARRapid after market drop-in approach
2008 JavaOneSM Conference | java.sun.com/javaone | 26
Base Software Platform Configuration
One week to fully autonomous vehicle (Feb ’07)Java technology & MAX installsBasic MAX parameter configurationsCalibration & tuningAutonomous navigation & obstacle avoidanceProven generic UGV framework with rapid configuration
2008 JavaOneSM Conference | java.sun.com/javaone | 27
Rules Creation and Evolution
Route planning: course maps (RNDF) + mission checkpoints (MDF)Passing vehiclesStopping at intersections & obeying precedence orderQueuing and following behind vehicles3-point U-TurnsProper turn signalingMergingOpen zone/lot navigationParking in parking spotsLane collision avoidanceGround obstacle avoidance (ditches, curbs)Proximity collision avoidanceRoute re-planningGPS outage handlingLane detection & road followingLow hanging obstacle avoidanceEight months of rule creation and testing (Mar ’07 to Oct ’07)
2008 JavaOneSM Conference | java.sun.com/javaone | 28
Loose Ends
Dashboard UI: manage & test TJrSimulation: test new rulesCourse maker: plot and load test coursesSafety: fail-safe, don’t rely on E-stop (controlled by DARPA)People: engineering & operationsParts: sensors, car, hardwareFacilities: basic testing, demos, large-scale tests, CA testsTravel: TJr legal to human drive in VA!, trailer to/from CAFinances/Marketing: sponsorship, logos, demos, TV, news
2008 JavaOneSM Conference | java.sun.com/javaone | 29
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 30
Applications, Sponsorships, & Kool-Aid
March 2006: Announcement• $2M/$1M/$500K prizes; $50K to all qualifiers, $100K to all finalists
April 2006: Team Jefferson back• Slam Dunk Plan ($$$): $800K• Winning Chance Plan ($$): $400K• Austerity Plan ($): $90K
Summer 2006: DARPA Kick-Off• No prize money• Vehicles required to have stock chassis – can’t reuse Tommy Sr.
Oct 2006: Track A awards• $1M up front to 11 teams; mostly large teams
Oct 2006: Track B applicants• 89 teams total (11 Track As, 78 Track Bs)• Interest down from 2004 & 2005
Jan 2007: Prize adjustments• Prize money reinstated; qualifier money not reinstated
Jan-Feb 2007: Team Jefferson Accelerates• Sponsorship support• Mechanical/electrical engineering accelerates
2008 JavaOneSM Conference | java.sun.com/javaone | 31
Video Submission & Site Visit
March 2007: Video Submission• Demo safety, self-navigation, and passing• 1 week SW configuration + few weeks new passing rules testing• 53 teams advance to site visit (11 Track As, 42 Track Bs)• Tommy Jr. advances• Targeting austerity plan
June 2007: Site Visit Demonstration• Demo safety, self-nav, passing, intersections, route planning,
queuing, U-turns, turn signaling• Demo in front of 3 DARPA officials• 35 teams advance to qualifications (10 Track As, 25 Track Bs)• Tommy Jr. aces & advances – projected among top ranked visits• Austerity plan goals close
2008 JavaOneSM Conference | java.sun.com/javaone | 32
Leaving for the Race
October 2007: Leaving Virginia for California• Tommy Jr in trailer, RV, and 3 team members• 3 day – 2 night trip fueled by caffeine & Java technology • Austerity goals met, partial way to winning chance target• More testing required:
• Lane detection, route re-planning, ground obstacles• Low-hanging obstacles?
• Completed:• Self-navigation, route planning, lane collision avoidance, intersections,
merging, queuing, passing, zones, parking, U-turns, signaling, proximity and zone collision avoidance, GPS outages, fail-safety
• Development & test enhancements• Operations & marketing enhancements
2008 JavaOneSM Conference | java.sun.com/javaone | 33
The National Qualifiers
Day 1: Teams given course maps on USB stix w/their team nameDay 2: E-stop test• Passed, DARPA presses wrong buttons
Day 3: Circular loop with intersections and re-routing• TJr. navigates intersections and U-turns, re-routes• TJr clips low-hanging railway crossing arm, no E-stop issued• Other teams w/issue, railway crossing arm ruled illegal for course
Day 4: Circular loop with intersections and merging• TJr. navigates intersections and obeys precedence and merging
rules• TJr. goes wide on a sharp turn in course map file• Other teams w/issue, we re-plan for course
2008 JavaOneSM Conference | java.sun.com/javaone | 34
The National Qualifiers - Continued
Day 5: Large course representative of city• TJr. navigates well initially• TJr. considers re-routing, course map points clip K-barriers• Tjr. restart, begins re-route, clips barrier, sensor damage, no E-stop
Day 6: Second run of loop w/intersections & re-routing• Team prepared for low-hanging obstacle, but crossing arm removed• Sensor damage causes TJr to detect ground and re-route• Team goes off to prep for Day 7• Gets call back early to run another course, team inquiry• Team gets call from DARPA director that is disqualified, team appeal
Day 7: Second run of loop w/intersections & merging• Team prepared to complete its second runs• DARPA director denies appeal• Urban Challenge Program Manager tries to help, decision is final
2008 JavaOneSM Conference | java.sun.com/javaone | 35
The Final Event
11 teams advance to final event (7 Track As, 4 Track Bs)DARPA director selects the teams, claims safety basisSix teams complete the course (5 Track As, 1 Track B)Prize Winners:• 1st Place: Carnegie Mellon – General Motors team (Track A)• 2nd Place: Stanford Univ – Volkswagen team (Track A)• 3rd Place: Virginia Tech – CAT team (Track A)
2008 JavaOneSM Conference | java.sun.com/javaone | 36
Tommy Jr. VideoPerrone Robotics, Inc.
2008 JavaOneSM Conference | java.sun.com/javaone | 37
Agenda
The challengeTommy Jr’s partsTommy Jr’s brainsTommy Jr’s journeyTommy Jr’s advancementConclusions
2008 JavaOneSM Conference | java.sun.com/javaone | 38
Reflections on the Journey
What worked well• Extraordinarily large function base generated per unit time• Building on Java technology and a framework definitely way to go
What didn’t work well• Gaps in lane detection & route re-planning – time runs out
Issues with race operations• Rules incomplete and not-specified• Decisions made by director vs committee• E-stop safety not employed
Lessons learned from race• Design for generalized vs specific rules ($$$)• Design for dynamic vs specified rules ($$$)• Design for more fail-safety scenarios paramount ($$)
2008 JavaOneSM Conference | java.sun.com/javaone | 39
Reaping the Rewards
Designs for use beyond challenge• High profile race & technology• Have generalized Java technology based robotics & automation
platform• Have generalized Java technology based UGV platform
Awareness generated• TV, news, trade• Hi-profile demonstrations• Sponsors benefit before, during, & after race
Applying to the real-world• Distributed sensor applications for automation• Platform and framework licensing• Commercial UGV applications• Vendor/sponsor technology used and licensed
2008 JavaOneSM Conference | java.sun.com/javaone | 40
Tommy Junior’s Sponsors
Sun Microsystems – Java technology, advisors, financialPerrone Robotics – MAX, software/systems/operations, financial
University of Virginia – mechanical/electrical
Northrop Grumman - financialAutoTrader - financial
WowWee Robotics - financial, toy botsBDMetrics – financialSICK - lasers45Fix - financialTDFund – financial
2008 JavaOneSM Conference | java.sun.com/javaone | 41
Paul J. PerroneCEO – Perrone Robotics www.perronerobotics.comTeam Lead – Team Jefferson www.teamjefferson.com
TS-5925
top related