Eye-RIS

Vision System

sense – process - control autonomous mode Program stora

Outline

hardwaresystem overviewAnaFocus, SIS Q-Eye chipDiCopALTERA NIOS II RISC uP, MLP

softwaredevelopment environment

System overview

Early-processingExtract information from the image flowOutput: reduced set of dataPixel-wise simultaneous processing

Not a loop!

Post-processing Inputs are abstract entitiesOutput complex decisions

System overview

Two main parts:Smart Image Sensor (SIS) – Q-EYE chipRISC processor – Altera Nios

Q-Eye chip

176 x 144 (QCIF) resolution Each cell:

Multi-mode optical sensorsPixel memoriesLinear and non-linear analogue processorsBinary processors Interconnected with 8 neighbours

Q-Eye chip

„This means that each pixel can both sense the corresponding spatial sample of the image and process this data in close interaction and cooperation with other pixels.”

„Smart Image Sensor Q-Eye is also called Focal-Plane Processor (FPP) because it processes images at the same physical layer where they are sensed.”

Conventional Vision System

Be aware of the data size of F!

Eye-RIS Vision System

In this case: f << F

Architecture of the Q-Eye chip

The cell array

176x144 interconnected sensing-processing cells. Each of these cells contains mixed-signal circuitry capable of: Sensing images; including adaptive linear and high-dynamic range

acquisition Storing up to 7 grey scale images and up to 4 binary images Performing grey-scale image additions, subtraction and multiplication by

a constant Performing spatio-temporal image filtering (low-pass, high pass, band

pass and stop band) with programmable bandwidth. Performing grey scale image convolutions Performing grey scale to binary, and binary to grey scale conversions Performing generic combinational and sequential binary operations Performing Hit-and-Miss binary operations Real-time loading and downloading of images Ultra fast calculation and input/output of certain image features such as

centroids, mean values, pixel coordinates…

Block Diagram of a Q-Eye cell

Altera Nios II

General-purpose RISC processor core. Harvard architecture. Full 32-bit instruction set, data path, and address space. 32 general-purpose registers. 32 external interrupt sources. Single-instruction 32 x 32 multiply and divide producing a 32-bit result. Single-instruction barrel shifter. Access to a variety of on-chip peripherals, and interfaces to off-chip memories

and peripherals through Avalon Bus. Hardware-assisted JTAG debug module enabling processor start, stop, step and

trace. Instruction and Data Cache Memory. JTAG Debug Module. Exception and interrupt controller.

Summary of Q-Eye chip

Chip: Q-Eye Technology: 0,18 um Power Supply: Dual (1.8 V, 3.3 V) # of Processor Elements array: 176×144 Accuracy: ~8bit Density of Processor Elements: 422 PE × mm2

Die Size: 60 mm2

Package size: 64×67×63 mm Power Consumption of system: 700mW Maximum frame rate: > 10 000 I/O type: Binary or grayscale type images

Software

Eye-RIS ADKNios II: ANSI C/C++Q-EYE: CFPP

Programming

Programming Q-Eye chip

The CFPP Image Processing Library (IPL) Spatio-temporal filters Arithmetic operation between images Thresholding Morphological and logic binary operations

The Extended Image Processing Library (EIPL) Blob management Classifying functions Linear and non-linear digital processing of grey-level images Geometrical transformations

Programming Altera Nios II

C,C++, Assembly The Eye-RIS Basic Library (EBL):

Control the execution of the CFPP code I/O management Sending and/or receiving images to/from the PC

either to be displayed or saved to disk Printing error or information messages to a console Timer management

Programming Eye-RIS (sync)

Programming Eye-RIS (async)

CFPP global memories

// Declarationextern fpp_int value;extern fpp_bool flag; extern fpp_time texp; // Access void foo() {

value = 123; int foo = value; if(flag) {

FPPTime_write(texp, 500); int readTExp = FPPTime_read(texp);

} }

DEMO