a perceptual interface using integral projections
DESCRIPTION
A PERCEPTUAL INTERFACE USING INTEGRAL PROJECTIONS. Ginés García Mateos [email protected] Sergio Fructuoso Muñoz [email protected] Dept. de Informática y Sistemas University of Murcia - SPAIN. System overview. Face detector and tracker. Camera. User of the per-ceptual interface. - PowerPoint PPT PresentationTRANSCRIPT
UNIVERSITY OF MURCIA (SPAIN)
ARTIFICIAL PERCEPTIONAND PATTERN RECOGNITION
GROUP
A PERCEPTUAL INTERFACE USING INTEGRAL PROJECTIONS
A PERCEPTUAL INTERFACE USING INTEGRAL PROJECTIONS
Ginés García [email protected]
Sergio Fructuoso Muñ[email protected]
Dept. de Informática y SistemasUniversity of Murcia - SPAIN
2
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
System overview System overview
User of the per-ceptual interface
Camera Face detector and tracker
FACE LOCATION:X: 6 Y: 0 Z: 3
FACE ORIENTATION:Roll: 7.7 Pitch: 0 Yaw: 3
FACE MOVEMENT:ΔX: -2 ΔY: 1 ΔZ: 0
3D movement and pose estimation
Control signals (virtual world movement)
Virtual world rendering
MOVEFORWARD/BACKWARD
LEFT/RIGHTROTATE
LEFT/RIGHTLOOK
UP/DOWN
3
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Face integral projectionsFace integral projections• Definition. Let i(x,y) be an image, and R(i) a region in it:
Vertical Integral Projection Horizontal Integral Projection
PVR : {ymin, ..., ymax} R PHR : {xmin, ..., xmax} R
PVR(y) = i(x,y); (x,y) R(i) PHR(x) = i(x,y); (x,y) R(i) • When applied to human faces, typical patterns of
projection appear.• This is used to design a face detector and tracker.
Pv(y): Vertical I.P. of the face
Ph1(y): Horizon-tal I.P. of eyes’ region
Ph2(y): Horizon-tal I.P. of mouth’s region
4
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Face tracking with I.P.Face tracking with I.P.
Green line: Projection model Red line: Projection instance
PVFACE(y) Align PVFACE
y y
EYES PHEYES(x) Align PHEYES
x x
EYE1, EYE2 PVEYE1,PVEYE2 Align PVEYEi
yy
STEP 1
STEP 2
STEP 3FACE Face tracking is a
3-step process.
1. Vertical alignment: compute move-ment and scale in vertical direction
2. Horizontal alignment: compute move-ment and scale in horizontal direct.
3. Orientation alignment: compute orienta-tion of the face
5
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Pose estimation (i)Pose estimation (i)• Depth estimation: inversely proportional to the
size of the head in the image. Approximated with the eye-to-eye distance.
0.674 0.947 1.184 1.636Relative
distances
• Roll estimation: approximated with the perceived angle of the eyes.
-33.4º -13.0º 13.5º 20.2º
Roll angles
6
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Pose estimation (ii)Pose estimation (ii)• Pitch estimation: a heuristic measure is defined,
using the vertical I.P. of the face
Estimated pitch
• Yaw estimation: another heuristic, using the horizontal I.P. of the eyes.
-7 0 11
Estimated yaw
19 -2 -9
7
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Virtual environmentVirtual environment• The perceptual interface is used to control the
movement in a virtual 3D world.• We have used DirectX 9 and OpenCV 3.
Virtual 3D world
8
A PERCEPTUAL INTERFACE
USING INTEGRAL
PROJECTIONS
Ginés G.M.
Sergio F.M.
PRIA-7-2004St. PetersburgOCT, 2004
Results and conclusionResults and conclusion
Conclusions• Depth and roll estimation is very reliable.• Pitch and yaw are less stable.
• Sample videos available at:http://dis.um.es/~ginesgm/fip/percint.html
.. ,10/2004