At the same time, user needs to look 1) at the cursor - to know open a menu, and 2) image showing the result of sensing (face tracking) – to know how to move his face.

Rest position = Zero position for Nouse in Mouse and Joystick modes (X0,Y0)

Range of face motion = is used, along with Nouse Zero position, to map nose position in image to cursor position on a screen.

Detected nose position. If it is not detected, the range box turns red AND NouseCursor (active cursor) turns red too

CENTER = REST = Zero position for Nouse in Mouse and Joystick modes (X0,Y0)

Range of face motion (rectangle ABCD) = is used, along with Nouse Zero position, to a) map nose position in image to cursor position on a screen in Mouse mode, and b) to map nose position in image to a letter in typing mode (using on-screen NouseBoard key layout and protocol

Most stretched left, right, up, down positions a user can exhibit (NB: they are not necessarily on the same row or column, neither at the same distance from the CENTER position.

Lines showing the division of the entire space of user’s motion onto 5x3 block, as obtained from calibration procedure and used in letter typing mode using NouseBoard (Invention 2005b).

Detected nose position. It is mapped to a letter using the protocol and layout of NouseBoard. Here (2,2) – for left image or (1,2) – for right image, position will be used.

C

A

D

A

B

Typical maximal range of motion for a) physically adapt user, and b) disabled user

S -1:

State 22 (or 0): not activated icon. Nouse is running, but no face is detected.

When a face is detected, FR module starts.

0.0. After N secs (M good snapshots)

0.1. ID consistent (it thinks it recognized a face)

0.2. ID inconsistent (it does not recognize a face)

0.1 - “a) If it’s you Dmitry, you can now proceed to showing your “rest” (0,0) position. b) Otherwise, please move your head, so I have a better look of you”

Otherwise, please move your head a bit head, so I can memorize you better”

b) would you like to enroll

0.1.0. if clicked. if face is still (or/and FD.x/y the same), store RangeZero, retrieve RangeXY & go to State 1.1 or 1.6 or 2.0

0.1.1. otherwise go to 0.0

(or

1ba) I does not seem to know you

1

image

550: Done

505: New setup?

Face tracked to set s(with colour,motion,corners computed)

520: Preprocess video image

525: Define search window

530: Find X-spotwith pixel accuracy

using correlation

535: Find X-spotwith sub-pixel accuracy

using convolution

Motion,colour,edges,Haar-wavelets nose search box: x,y,width,height

540: Report X-spot location to USER

545: Continue tracking?YES

YES

NO

NO

USER

User recognized

Convex-shape template matching nose tip detection: I,J (pixel precision)

Integration over continuous intensity X,Y (sub-pixel pixel precision)

Nouse connected

550: Done

505: New setup?

Visual pattern analyzed

Nouse zero position (0,0) setFace position converted to (X,Y)

User’s face detected

540: Report X-spot location to USER

545: Continue tracking?YES

YES

NO

NO

USER

User recognizedNouse initialization

and calibration User’s motion range obtained