rick parent - cis681 motion analysis – human figure processing video to extract information of...

Rick Parent - CIS681

Motion Analysis – Human Figure

Processing video to extract information of objects

Motion tracking

Pose reconstruction

Motion and subject recognition.

3 categories of objectives

Rick Parent - CIS681

Motion Tracking

Identification and location of subject characteristics

Based on features or regionsOptical flowsilhouettes

ApplicationsPedestrian flowtraffic flow.

Rick Parent - CIS681

Pose Reconstruction

2D or 3D tracking of body parts

Usually with geometric model• DoFs?• Silhouette• Color, Textures

What?• Entire pose• Hand & arm gestures• Facial expression & lips

Applications• Medical - gait analysis• Sports• Entertainment

With instrumentation – MoCap.

Rick Parent - CIS681

Recognition

Extract discriminatorsNeural NetsPCA

Abstract out the “what”high level semantics

Can be based on tracking or pose reconstruction

What?Gestures - by arm motionPerson - by his/her gaitAmerican Sign Language (ASL) - by finger position.

Rick Parent - CIS681

Problems.

Complex, varying environment

Occlusion of body parts.

Segmentation issues

Rick Parent - CIS681

Possible simplifying assumptions on environment

Static and/or uniform background

Restrict number of objects

Restriction on movement

Restrict complexity of objects.

Rick Parent - CIS681

Segmentation problems

Image quality

Low contrast images

Motion blur

Strong shadows, reflections

Loose fitting clothes - use tight fit, markers.

Rick Parent - CIS681

Occlusion problems

3D model, 2D image

Underconstrained problems

Use multiple cameras.

Rick Parent - CIS681

Taxonomy

Static v. dynamic processing e.g. use nOptical flow

2D v. 3D Model based or featuresEdgesRegions: colors, textures

Single camera v. multi-cameraLocate points uniquelySingle: underconstrained

Rick Parent - CIS681

Processing Variations

Background subtraction• Contract enhancement• Shadow removal• Edge detection• Region filling• Clustering• Silhouette formation

Dynamic update of background• Changing illumination

• Shadows, brightness• Update of static fixtures

Body part matching• Find appendages, joints• Toe - heel curvature• Head

SpeedReal-time

only previous framesInteractiveBatch

Need previous examples?Need initialization?

Single frame or temporal info.

Rick Parent - CIS681

Cambridge Research lab

Rick Parent - CIS681

Cambridge Research lab

Rick Parent - CIS681

Cambridge Research lab

Rick Parent - CIS681

3D Model OptimizationLive action

Synthetic scene adjust compare

project

Rick Parent - CIS681

3D Model Optimization

Matching features of synthetic image with live image

Temporal coherenceAnatomyArea matchingBoundary matching

Rick Parent - CIS681

Gavrila

Multiple camerasDetect edges - remove static ones

Rick Parent - CIS681

Gavrila

Rick Parent - CIS681

Gavrila

Rick Parent - CIS681

MERL

Capture motion snippets Baysian formulation

Rick Parent - CIS681

Sminchisescu

Rick Parent - CIS681

Sminchisescu

Rick Parent - CIS681

Sminchisescu

Rick Parent - CIS681

Sminchisescu