a road sign recognition system based on a dynamic visual model

National Taiwan University

[email protected] 1

A Road Sign Recognition System Based on a Dynamic

Visual Model

C. Y. Fang Department of Information and Computer Education

National Taiwan Normal University, Taipei, Taiwan, R. O. C.

C. S. Fuh Department of Computer Science and Information Engineering

National Taiwan University, Taipei, Taiwan, R. O. C.

S. W. Chen Department of Computer Science and Information Engineering


P. S. Yen Department of Information and Computer Education


National Taiwan University [email protected] 2

Outline

Introduction Dynamic visual model (DVM) Neural modules Road sign recognition system Experimental Results Conclusions


Introduction -- DAS Driver assistance systems (DAS)

The method to improve driving safety

Passive methods: seat-belts, airbags, anti-lock braking systems, and so on.

Active methods: DAS

Driving is a sophisticated process The better the environmental information a

driver receives, the more appropriate his/her expectations will be.


Introduction -- VDAS Vision-based driver assistance systems (VDAS) Advantages:

High resolution Rich information Road border detection or lane marking detection Road sign recognition

Difficulties of VDAS Weather and illumination Daytime and nighttime Vehicle motion and camera vibration


Subsystems of VDAS Road sign recognition system System to detect changes in driving

environments System to detect motion of nearby vehicles Lane marking detection Obstacle recognition Drowsy driver detection ……


Introduction -- DVM

DVM: dynamic visual model A computational model for visual analysis

using video sequence as input data

Two ways to develop a visual model Biological principles Engineering principles

Artificial neural networks

Dynamic Visual Model

Conceptualcomponent

Perceptualcomponent

Sensorycomponent

Information acquisition

CART neural module

STA neural module

Yes

No

Video images

Focuses of attention

Spatialtemporal information

Categorical features

Category

Feature detection

Pattern extraction

CHAM neural module

Patterns

Data transduction

Action

Epi

sodi

c M

emor

y


Human Visual Process

Transducer

Sensory analyzer

Class of input stimuli

Perceptual analyzer

Conceptual analyzer

Physical stimuli

Data compression

Low-level feature extraction

High-level feature extraction

Classification and recognition


Neural Modules Spatial-temporal attention (STA)

neural module Configurable adaptive resonance the

ory (CART) neural module Configurable heteroassociative memo

ry (CHAM) neural module


STA Neural Network (1)ak

Output layer(Attention layer)

nj

Inhibitory connection

Excitatory connection

Input layer

wij

ai

xj

nk

ni


STA Neural Network (2) The input to attention neuron ni due to input st

imuli x:

The linking strengths between the input and the attention layers

corresponding neurons

wkj

ni

nj

nk

Input neuron

Attention layer

rk

Gaussian function G

m

jjiji

vi xwI

1

xw


STA Neural Network (3) The input to attention neuron ni due to later

al interaction:

Lateral distance

“Mexican-hat” function of lateral interaction

Interaction

+

ikNk

kikikli

i

aMuI,

rr


STA Neural Network (4) The net input to attention neuron ni :

: a threshold to limit the effects of noise

where 1< d <0

)),(( iii netqBpaAa

li

vii IInet

, 0 if

0 if )(

xdx

xxxA

, 0 if 0

0 if )(

x

xxxB


STA Neural Network (5)

tt

p

1

pd

1

The activation of an attention neuron in response to a stimulus.

stimulus

activation


ART2 Neural Network (1) CART

r

p

u

w

v

x

q

y

Input vector i

Input representation field F1

Attentional subsystemOrienting subsystem

G

G

G

G

G

Category representation field F2

Reset signal

＋

＋

＋

＋

＋

＋

＋

＋＋

＋＋

＋＋

＋

＋

＋

＋

＋

－

－－－

－

Signal generatorS


ART2 Neural Network (2) The activities on each of the six sublayers on F 1:

where I is an input pattern

where

where the J th node on F 2 is the winner

iii auIw

we

wx i

i

)()( iii qbfxfv

uv

e vii

iJii dzup

qp

e pii

x0 0)(

xxxf


ART2 Neural Network (3) Initial weights:

Top-down weights:

Bottom-up weights: Parameters:

0)0( ijz

Mdji

)1(

1)0(z

0, ba

10 d

11

d

cd

10

10

1e


HAM Neural Network (1) CHAM

j

Output layer(Competitive

layer)Excitatoryconnection

Input layer

wij

xj

i

viv1 v2 vn


HAM Neural Network (2) The input to neuron ni due to input stimuli x:

nc: the winner after the competition

m

jjijii xwnet

1

xw

)).max(arg( ii

c netn

otherwise. 0

if 1 ci

ni v


Road Sign Recognition System Objective

Get information about road Warn drivers Enhance traffic safety Support other subsystems


Problems

contrary light

side by side shaking occlusion


Information Acquisition

Color information Example: Red color

Shape information Example: Red color edge


Results of STA Neural Module— Adding Pre-attention


Locate Road Signs — Connected Component


Categorical Feature Extraction

Normalization: 50X50 pixels Remove the background pixels Features:

Red color horizontal projection: 50 elements Green color horizontal projection: 50 elements Blue color horizontal projection: 50 elements Orange color horizontal projection: 50 elements White and black color horizontal projection: 50

elements Total: 250 elements in a feature vector


Conceptual Component— Classification results of the CART

Training Set

Test Set


Conceptual Component— Training and Test Patterns for the CHAM


Conceptual Component— Another Training Patterns for the CHAM


Experimental Results of the CHAM


Experimental Results


Other Examples


Discussion

Vehicle and camcorder vibration Incorrect recognitions

Input patterns

Recognition results

Correct patterns


Conclusions (1) Test data: 21 sequences

Detection rate (CART): 99% Misdetection: 1% (11 frames) Recognition rate (CHAM): 85% of detected

road signs Since our system only outputs a result for

each input sequence, this ratio is enough for our system to recognize road signs correctly.


Conclusions (2)

A neural-based dynamic visual model Three major components: sensory, perceptual

and conceptual component Future Researches

Potential applications Improvement of the DVM structure DVM implementation

a road sign recognition system based on a dynamic visual model

Documents

d sta neural network

attention neuron ni

net input

input pattern

visual analysis

visual process

input stimuli x

fang department of information