P1.4

Light has to go where it is needed: Future Light Based Driver Assistance Systems Thomas Könning¹, Christian Amsel¹, Ingo Hoffmann² ¹ Hella KGaA Hueck & Co., Lippstadt, Germany ² Hella-Aglaia Mobile Vision GmbH, Berlin, Germany Keywords: Driver assistance systems, Lighting systems, Light based driver assistance

systems, camera technology, Image processing technology

1 Abstract To approach the vision of an optimum sight at night, the first light based driver assistance systems have been developed and introduced into the market. Examples are infrared night vision systems, high beam assists and full adaptive front lighting systems (AFS), like the ILS (Intelligent Light System) from Daimler Chrysler's E-Class. Future light based driver assistance systems do not only adapt the light distribution automatically to the street and weather conditions, furthermore, those systems adapt its lighting distributions to the actual traffic situation. Those systems base upon the interaction of CMOS image sensors with an image processing unit and state-of-the-art lighting technology. The Adaptive Cut-Off Line System® from Hella will be the first camera- and light-based driver assistance system. It controls the vehicle lighting with the goal of the maximum possible range and from there the optimum driver sight. This will be achieved with the adaptation of the low beam to oncoming and preceding traffic participants. The result is, that the low beam doesn't end at approximately 65 meters like today but increases up to a range of a couple of hundred meters in best cases – without any dazzle of other traffic participants, because the beam ends always at their vehicles. The light distribution will be generated by light electronic components and variable light projection type modules like the Hella VarioX® system, presently the strongest Xenon light in the automotive market.

P1.4

Figure 1: Light based driver assistance: an interaction of camera, image processing and lighting technology

In future, further more light based driver assistance systems like glare-free high beam or marking light will introduce into the market. For these applications LED-arrays are the most promising technology. Backed by image processing they are able to address particular segments, with the result of inhibiting any glare to other traffic-participants (glare-free high beam) or, inverse, a spotlight aimed to pedestrians and other dark but relevant objects (marking light).

2 Camera-controlled headlamps revolutionize lighting technology

2.1 Adaptive Cut-off line system Optimum road-space illumination for the driver, while avoiding non-permitted dazzling of other road-users, is the task of light-based driver assistance systems.

P1.4

Optimum customer benefit is achieved when the car driver can take in as much information as possible without being burdened with additional tasks. This includes adaptive and assistive light control on the basis of camera-based pedestrian and object detection. The first camera-controlled headlamp systems will be launched by Hella in 2009. The further development of vehicle frontlighting from static to dynamic systems is now a reality. The first such system, namely dynamic bend lighting, was launched in 2003. In bends, this almost doubles the range of the low beam, and therefore the visual range of the driver. The next step, which is already in series production, is the Advanced Frontlighting System (AFS), based on Hella's VarioX headlamp module. Above all, the headlamp light illuminates the road space according to speed, weather and the road. This means that it is a true driver assistance system and, on the basis of the current legal AFS regulation, represents the pinnacle of today’s headlamp technology. Light-based driver assistance systems of the future go even further: They automatically adapt their light distribution not only to the road and weather conditions, but also to the respective traffic situation. They are based on the interaction of image-producing sensors, powerful software for image processing, and state-of-the-art lighting technology. The first light-based driver assistance system to be controlled by image data will be the adaptive cut-off line. The system always sets the range of the AFS headlamps such that the driver has optimum visibility with the longest range possible. This is achieved through the adaptation of the headlamp range to preceding or oncoming motor vehicles. This means that the low beam does not stop, as is usually the case today, at around 65 meters on the oncoming lane, but rather that it can, in extreme cases, be increased to several hundred meters. In addition to the maximum possible visual range for the driver, dazzling of other road-users is, at the same time, impossible, as the headlamp cone always ends at their vehicles [1]. If the image-processing system does not detect any road-users, the system can provide the driver with light up to the high-beam level. As soon as the camera

P1.4

detects other road-users at up to a distance of approximately 800 meters, the range of the headlamps is adapted accordingly within milliseconds.

Figure 2: Benefit of an Adaptive Cut-off line System®

2.2 With the VarioX module toward glare-free high beam The lighting technology basis of AFS is the VarioX projection module from Hella. Between the light source and the projection lens there is a drum which is rotatable on bearings and which is contoured according to the AFS light distributions. Using a stepper motor, it is rotated to the respectively required position within milliseconds. Glare-free high beam follows the premise that the car driver travels almost constantly with high beam switched on. If road-users who are in danger of being dazzled appear in the road space, those portions of the high-beam light distribution that could cause a problem to others are automatically faded out.

P1.4

This can be realized technically through special contouring of the rotatable drum in the VarioX projection module. This means that areas in the high beam can be omitted. On the basis of the image data and through intelligent adjustment of the VarioX modules, oncoming traffic is thus faded out. For the driver, however, the high-beam light distribution is almost maintained, while the visual range is considerably increased in comparison with current systems.

Figure 3: VarioX®-Module

2.3 Marking light illuminates people and points of danger in a specific manner

The marking light is a further innovation in the field of light-based driver assistance system and behaves in exactly the opposite manner to glare-free high beam: Using an AFS light distribution as a basis, people and points of danger are

P1.4

illuminated in a specific manner. The driver is able not only to discern them considerably earlier but also consciously detects them and can adapt his or her driving behavior accordingly in time. This innovation, which is currently under development, is realized on the basis of so-called LED arrays. While, to date, the strengths of LED headlamps have been above all in the areas of light color and styling, the technical potential of the next generation is in LED segments which can be triggered individually.

2.4 The future belongs to multi-function cameras Currently, a market is coming into being for light-based driver assistance systems which rely on cameras and which, in terms of function, stand out considerably from systems already known (such as lane recognition or night vision on the basis of simply one sensor). Hella is already working on multi-function camera systems with which several functions can be combined, such as lane recognition or traffic-sign recognition and adaptive cut-off line. Further added value is created through the fusion of the data of camera systems and other vehicle sensors. Thus, Hella is pursuing concepts in order to realize an ACC stop & go system (deceleration to a standstill and moving-off initiated by the driver) with the aid of the ACC lidar sensor (lidar = light detection and ranging, based on infrared technology) and a camera. Today, such a system is realized with the aid of up to three radar sensors. In order to meet this demanding requirement it makes sense, that both camera approaches, monocular and stereo, pursued. By exploiting depth information, stereo vision opens up spatial vision and therefore makes a significant contribution to object classification. Hella achieves increased availability of assistance systems at night with the aid of modern infrared technology, which, even today, is the basis of active night-vision systems. Infrared technology allows a high-beam distribution to be generated which is invisible to humans but which is useable for corresponding cameras. In addition to night-vision systems, other assistance systems, from lane

P1.4

or traffic-sign recognition to assistive light (glare-free high beam and marking light), can thus also be supported in an optimum manner.

2.5 Image processing as a key technology for driver assistance At the present time, radar systems are used in luxury cars to recognize surrounding vehicles and obstacles. Comfort systems like ACC or safety-relevant systems like pre-crash sensors or emergency brakes have been installed and are already in use. These systems are based on new sensor technology providing information not only about scenes inside the car but also about the periphery surrounding and ahead. Using this information, the car is able to warn the driver, to suggest alternatives in critical situations, to prepare its security systems for a crash or to take over the control to actively avoid collisions. To reliably perform these tasks, it is necessary to get as much information as possible about the course of the road and all relevant obstacles, i.e. vehicles, pedestrians, animals, trees, buildings and others. All these obstacles have to be detected and tracked up to a distance determined by the assistance system, providing information about their position, size, speed and thread potential. When the classification is confident about these parameters, it is possible to deduce actions and to provide this information to special driver assistance systems. Since traffic infrastructure is designed for visual perception, some tasks like traffic sign recognition or lane detection need a camera system to be fulfilled. Furthermore, object detection and classification tasks can be solved with a camera as well, since it provides very dense spatial and temporal information. Here the visual system is superior to sampling technologies like radar supplying just a few distance measurements with poor lateral accuracy. Another point is an increase of covering range. Here the active radar system has to boost the radiation to be emitted, where the image processing algorithms have to become more sophisticated and the camera has to be featured with higher resolution and sensitivity. But no radiation at all is emitted. This way interference of cars equipped with this assistance system is excluded. Last but not least, the enhancement of any hardware used benefits from the booming customer mass market regarding imager and processing hardware, since no separate technology is used.

P1.4

To provide an Adaptive cut-off line assistant it is necessary to detect any traffic participant at night up to a distance of 800 meters. In a first step the image processing detects any light source in the field of view. In parallel the egomotion of the subject vehicle is recovered. Subsequent modeling steps extract a set of attributes of the observed lights. This way a detailed model is established and tracked over time providing enough information to perform an in-depth classification. As a result it is possible to differentiate between headlamps, rear lamps, street lamps, advertising, reflectors and so on. The last step selects the adequate light distribution providing as much light as possible regarding the current traffic situation. This distribution gets signaled to the vehicle’s headlamps without any interaction with the driver. A high beam distribution is chosen when no relevant object is detected.

Figure 4: Image taken by a front camera at night time (a), after the classification of objects (b) and the internal modelling of the traffic situation (c)

P1.4

3 Vision: Halving of road fatalities by 2010 The continuous further development of light-based driver assistance systems is accompanied by the European Union‘s eSafety action program for road safety, with the objective of halving the number of road fatalities by 2010. Driver assistance systems from Hella based on the recording and analysis of the vehicle surroundings will make an important contribution to this. According to studies by the German ministry of transport, such systems would have a preventive effect on more than 50 percent of all accidents. Reference [1] Kesseler, W., Könning, T., Amsel, C.: From camera based night vision

systems to lighting driver assistant systems; VDI-Tagung Optische Technologien in der Fahrzeugtechnik, VDI-Verlag, 2006