lecture control and perception in networked and automated

Part

Patrick Scheffe, M. Sc. | Dr.-Ing. Bassam Alrifaee

Winter Semester 2020/2021

Lecture

Control and Perception in Networked and Automated Vehicles

6

Software Architectures and Testing Concepts

Control and Perception in Networked and Automated VehiclesPart 6: Software Architectures and Testing Concepts | Patrick Scheffe, M. Sc.; Dr.-Ing. Bassam Alrifaee

Course contents

► Dynamic vehicle models

► Control and optimization

► Network and distribution

► Machine perception

► Software architectures and testing concepts

► Case study

2


CPM Lab architecture

3

Setup/InfrastructureSense

PlanAct

Control

Map

Static map

Dynamic map

Preview map

Scenario server

Simulation of passive traffic participants

Perception Decision-MakingCoordination

Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model

Scenario, static map, mission plan

States, plan

Environment model

Sensor data Control inputs

Environment model


Further literature (1)

► B. Alrifaee. Networked Model Predictive Control for Vehicle Collision Avoidance. PhD thesis, RWTH Aachen University, 2017.

► K. Beck. Extreme Programming Explained. Addison-Wesley, 1999.

► E. Gamma, R. Helm, R. Johnson, J. Vlissides. Design Patterns. Elements of Reusable Object-Oriented Software. Addison-Wesley, 1994.(= „Gang of Four“, „GoF“)

4


Further literature (2)

► M. Kloock, J. Maczijewski, P. Scheffe, A. Kampmann, A. Mokhtarian, S. Kowalewski, and B. Alrifaee. Cyber-Physical Mobility Lab – An Open-Source Platform for Networked and Autonomous Vehicles, 2020.

► P. Scheffe, J Maczijewski, M. Kloock, A. Kampmann, A. Derks, S. Kowalewski, and B. Alrifaee. Networked and Autonomous Model-scale Vehicles for Experiments in Research and Education, 2020.

► M. Kloock, P. Scheffe, I. Tülleners, J. Maczijewski, S. Kowalewski, and B. Alrifaee. Vision-Based Real-Time Indoor Positioning System for Multiple Vehicles, 2020.

5


CPM Lab motivation

6

Lab: rapid functional

prototyping, cost and time efficient

Simulation: abstracts from

real-world behavior

Real-world: expensive and

time-consuming


CPM Lab components

7

High-level

controller (HLC)

Indoor positioning

system (IPS)

Mid-level

controller (MLC)


CPM Lab room

8

Driving area𝑥

𝑦

Door

Win

do

ws

𝑦max = 4 m

𝑥max = 4,5 𝑚

Lab camera

IPS


CPM Lab architecture: communication

9


PlanAct

Control

Map

Static map

Dynamic map

Preview map

Scenario server



Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model


States, plan

Environment model


Environment model


Domain

Data Distribution Service

10

Sub

Sub

Sub

Sub

Pub

Pub

Pub

Topic

Topic

Topic


Domain 21:network

Domain 1:local

Data Distribution Service – CPM Lab

11

Vehiclen

Vehicle1

IPS

HLC1

Middle-ware

Command

State

Pose

Command

State

PoseHLC

n


Data Distribution Service – message format

► Message datatype on topic defined with Interactive Data Language (IDL)

► Examples Pose ID, 𝑥 𝑡 , 𝑦 𝑡 , 𝜓 𝑡 Trajectory ID, 𝑥 𝑡 , 𝑦 𝑡 , 𝑣𝑥 𝑡 , 𝑣𝑦 𝑡

State ID, 𝑥 𝑡 , 𝑦 𝑡 , 𝜓 𝑡 , 𝑠 𝑡 , 𝑣 𝑡 , 𝑎 𝑡 , …

12


CPM Lab architecture: indoor positioning system

13


PlanAct

Control

Map

Static map

Dynamic map

Preview map

Scenario server



Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model


States, plan

Environment model


Environment model


pose

posepose

ID

Indoor positioning system

► Camera mounted on the ceiling Top-down view on the field Streams images with 50 Hz

► Vehicles are equipped with LEDs 3 LEDs to encode pose

• Non-equilateral triangle

1 LED to encode ID• Frequency

14


Indoor positioning system

► Low exposure time leads to high contrast of LEDs to background

► IPS pipeline:1. Extract LED points of camera image 2. Transform image coordinates to world coordinates3. Find vehicles4. Extract position, orientation, and ID

15


Indoor positioning system – interface

16

PoseImage IPS

Vehicle

HLC

Cam


CPM Lab architecture: vehicle

17


PlanAct

Map

Static map

Dynamic map

Preview map

Scenario server



Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model


States, plan

Control

Environment model


Environment model


Vehicle – µCar

► Mechanical platform XRAY M18 PRO LiPo

► Sensors IMU (DeboSens BNO055) Odometer (3 Hall-effect sensors + magnet)

► Actuators Servo motor (Hitec D89MW) for steering Brushless DC Motor (Pololu VNH5019) for propulsion

► Computation Mid-level controller (Raspberry Pi Zero W) for WLAN, clock synchronization,

sensor fusion, trajectory following control and path tracking Low-level controller (ATmega2560) for reading sensor data and actuation

18

Control and Perception in Networked and Automated VehiclesPart 6: Software Architectures and Testing Concepts | Patrick Scheffe, M. Sc.; Dr.-Ing. Bassam Alrifaee19

Servo

Motor

Battery


Raspberry Pi

Zero WATmega2560

Motor Driver

Inertial

Measurement

Unit


Pose LEDPose LEDsID LED


Hall Effect Sensors

and Magnet


Vehicle – operating modes

► Operating modes with different commands Motor and steering Speed and curvature Speed and path Trajectory

23


Vehicle – operating modes

► Path tracking: Stanley controller [1]

► Trajectory following Model predictive controller on Raspberry Pi Zero W Simplified bicycle model [2]

24

Servo

Voltage

x, y, φ

δ

v

Vehicle

model

Motor

[1] Hoffmann, Gabriel M., et al. "Autonomous automobile trajectory tracking for off-road driving: Controller design, experimental validation and racing." 2007 American control conference[2] https://arxiv.org/abs/2004.08364

https://arxiv.org/abs/2004.08364


State

Vehicle – interface

25

Pose

Vehicle

CommandHLC

IPS

HLC


CPM Lab architecture: high-level controller

26


PlanAct

Control

Map

Static map

Dynamic map

Preview map

Scenario server



Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model


States, plan

Environment model


Environment model


High-level controller

► Commands the vehicle

► Executes algorithms to be evaluated on scenario

► HLC in any programming language,only interface to DDS necessary

27

CommandHLC

Pose

StateVehicle

IPS

Vehicle


CPM Lab architecture: Scenario definition

28


PlanAct

Control

Map

Static map

Dynamic map

Preview map

Scenario server



Sensors Actuators

Verification

Routing

Behavior

Trajectory

Object data

Error, noise

Coupling graph

States, plan

Mission planning

Environment model


States, plan

Environment model


Environment model


Scenario definition: CommonRoad [1]

► Lanelets Left and right bounds Connections

► Static obstacles Dimensions

► Dynamic obstacles Dimensions Trajectory

► Planning problem Initial state Goal state

29

[1] Althoff, Matthias, Markus Koschi, and Stefanie Manzinger. "CommonRoad: Composable benchmarks for motion planning on roads." 2017 IEEE Intelligent Vehicles Symposium (IV)


Experimental concept

30

Development Computer• Trajectory planning• Middleware

Raspberry Pi• Trajectory following• Time synchronization

ATmega• Actuation• Measurement reading

Vehicle

DDS(UDP, WLAN)

SPI

Trajectory

Controlinputs

Estimatedstates

Measurementdata

Triggered

Real-time

Real-time

HLC

MLC

LLC

Operating Mode Hierarchy


Experimental concept

31


Rapid functional prototyping

32

► Simulation Plant (vehicle [+ IPS]) is simulated, HLC runs on desktop PC

► Experiment Software tested in simulation can be used through sharing of interfaces

Plant

Controller

XiL:

► Plant: model on desktop PC, model on real-time HW, real

► Controller: model on desktop PC, code on desktop PC, code on evaluation HW, code on protoyping HW, code on target HW


Process model – eXtreme Programming

► Evolutionary development in small increments

► Most important features first

► Test-first, automated testing

► Have working code

► Pair programming

► No explicit design, documentation, review suitable for small projects

33


Software design

► Design software for reusability, transparency, ease of maintenance, ...Guide: design patterns (GOF)

► Subsystems Working stand-alone Defined interfaces Consisting of components

► Components

► In MATLAB Structures Classes Functions

34


Git commands

1. git cloneTo copy a git repository from remote source

2. git statusTo check the status of files you’ve changed in your working directory

3. git addAdds changes to stage/index in your working directory

4. git commitCommits your changes and sets it to new commit object for your remote

5. git pushPush your changes to remote

6. git pullPull changes from the remote

35

You can find a more elaborateintroduction to git in the moodle room.


Simulation

36

lecture control and perception in networked and automated

Documents