Upload File

rick parent - cis681 motion analysis – human figure processing video to extract information of...

  • Home
  • Documents
  • Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject
23
Rick Parent - CIS68 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject recognition. 3 categories of objectives

Upload: elfreda-freeman

Post on 20-Jan-2016

215 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Motion Analysis – Human Figure

Processing video to extract information of objects

Motion tracking

Pose reconstruction

Motion and subject recognition.

3 categories of objectives

Page 2: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Motion Tracking

Identification and location of subject characteristics

Based on features or regionsOptical flowsilhouettes

ApplicationsPedestrian flowtraffic flow.

Page 3: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Pose Reconstruction

2D or 3D tracking of body parts

Usually with geometric model• DoFs?• Silhouette• Color, Textures

What?• Entire pose• Hand & arm gestures• Facial expression & lips

Applications• Medical - gait analysis• Sports• Entertainment

With instrumentation – MoCap.

Page 4: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Recognition

Extract discriminatorsNeural NetsPCA

Abstract out the “what”high level semantics

Can be based on tracking or pose reconstruction

What?Gestures - by arm motionPerson - by his/her gaitAmerican Sign Language (ASL) - by finger position.

Page 5: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Problems.

Complex, varying environment

Occlusion of body parts.

Segmentation issues

Page 6: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Possible simplifying assumptions on environment

Static and/or uniform background

Restrict number of objects

Restriction on movement

Restrict complexity of objects.

Page 7: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Segmentation problems

Image quality

Low contrast images

Motion blur

Strong shadows, reflections

Loose fitting clothes - use tight fit, markers.

Page 8: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Occlusion problems

3D model, 2D image

Underconstrained problems

Use multiple cameras.

Page 9: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Taxonomy

Static v. dynamic processing e.g. use nOptical flow

2D v. 3D Model based or featuresEdgesRegions: colors, textures

Single camera v. multi-cameraLocate points uniquelySingle: underconstrained

Page 10: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Processing Variations

Background subtraction• Contract enhancement• Shadow removal• Edge detection• Region filling• Clustering• Silhouette formation

Dynamic update of background• Changing illumination

• Shadows, brightness• Update of static fixtures

Body part matching• Find appendages, joints• Toe - heel curvature• Head

SpeedReal-time

only previous framesInteractiveBatch

Need previous examples?Need initialization?

Single frame or temporal info.

Page 11: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Cambridge Research lab

Page 12: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Cambridge Research lab

Page 13: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Cambridge Research lab

Page 14: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

3D Model OptimizationLive action

Synthetic scene adjust compare

project

Page 15: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

3D Model Optimization

Matching features of synthetic image with live image

Temporal coherenceAnatomyArea matchingBoundary matching

Page 16: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Gavrila

Multiple camerasDetect edges - remove static ones

Page 17: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Gavrila

Page 18: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Gavrila

Page 19: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

MERL

Capture motion snippets Baysian formulation

Page 20: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Sminchisescu

Page 21: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Sminchisescu

Page 22: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Sminchisescu

Page 23: Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject

Rick Parent - CIS681

Sminchisescu


Fast Human Pose Estimation using Appearance and Motion via …fmdb.cs.ucla.edu/Treports/050046.pdf · 2007. 5. 14. · Fast Human Pose Estimation using Appearance and Motion via Multi-Dimensional

Stable Pose Tracking from a Planar Target with an Analytical Motion …media.ee.ntu.edu.tw/personal/pcwu/research/mmsp2014_spe/... · 2018-10-24 · Stable Pose Tracking from a Planar

Face recognition across non uniform motion blur, illumination, and pose

Self-organizing neural integration of pose-motion …Parisi et al. Self-organizing integration of pose-motion features and Pinto, 1993). Behavioral testing has shown that young children

Patient MoCap: Human Pose Estimation under Blanket ... · Keywords: pose estimation, motion capture, occlusion, CNN, RNN, random forest 1 Introduction Human motion analysis in the

Learning Human Pose and Motion Models for Animation Aaron Hertzmann University of Toronto

Human activity recognition using multidimensional indexing ... · of human motion capture, namely, initialization, tracking, pose estimation, and recognition. Human motion recogni-tion

Line-Based Relative Pose Estimationelgammal/pub/CVPR11_Elqursh.pdf · Relative pose estimation is the problem of computing the relative motion between two images and is a key ingredient

Human Pose Estimation using Motion Exemplarsmori/research/papers/fathi_motion... · Human Pose Estimation using Motion Exemplars Alireza Fathi and Greg Mori School of Computing Science

Personal Care Robot: Mechanism, Motion Control, and Pose

An evaluation of 3D motion flow and 3D pose estimation for ...robotics.dei.unipd.it/images/Papers/Conferences/Munaro2013_RSS... · An evaluation of 3D motion flow and 3D pose estimation

Pose and Motion Estimation of Free-Flying Objects

Rick Parent - CIS681 ORIENTATION Use Quaternions Interpolating rotations is difficult

Pose Features based on Motion arXiv:1609.05420v1 [cs.CV ... · arXiv:1609.05420v1 [cs.CV] 18 Sep 2016. 2 Senthil Purushwalkam, Abhinav Gupta Appearance/Pose Motion Appearance/Pose

PoTion: Pose MoTion Representation for Action Recognition

Learning 3D Human Pose from Structure and Motion arXiv:1711.09250v2 [cs.CV… · 2018-07-05 · Learning 3D Human Pose from Structure and Motion Rishabh Dabral 1, Anurag Mundhada

Rolling Shutter Pose and Ego-motion Estimation using Shape … · 2020-05-23 · Rolling Shutter Pose and Ego-motion Estimation using Shape-from-Template 3 Fig.1. An overview of the

Learning Character-Agnostic Motion for Motion …Learning Character-Agnostic Motion for Motion Retargeting in 2D • 75:3 shape, pose and lighting codes. Peng et al. [2017] disentangle

Rick Parent - CIS681 Interpolation of a time-space curve Interpolation (of a curve in space) Speed control…

Learning gaze biases with head motion for head pose …arts.buaa.edu.cn/papers/Learning gaze biases with head...Learning gaze biases with head motion for head pose-free gaze estimation

Chained Multi-Stream Networks Exploiting Pose, …openaccess.thecvf.com/content_ICCV_2017/papers/...Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action

Performance Evaluation of Markerless 3D Skeleton Pose ... · 15-4-2020  · Performance Evaluation of Markerless 3D Skeleton Pose Estimates with Pop Dance Motion Sequence Rollyn T

Real-Time Motion Control Using Pose Space Probability ... · Real-Time Motion Control Using Pose Space Probability Density Estimation Dumebi Okwechime, Eng-Jon Ong, and Richard Bowden

Learning 3D Human Pose from Structure and Motion · Learning 3D Human Pose from Structure and Motion Rishabh Dabral 1, Anurag Mundhada , Uday Kusupati , Safeer Afaque1, Abhishek Sharma2,

3D Pose from Motion for Cross-view Action Recognition via ...julm/papers/cvpr14.pdf · 3D Pose from Motion for Cross-view Action Recognition via Non-linear Circulant Temporal Encoding

Fast Human Pose Estimation using Appearance and Motion …faculty.ucmerced.edu/mhyang/papers/cvpr07a.pdf · Fast Human Pose Estimation using Appearance and Motion via Multi-Dimensional

Learning Human Pose and Motion Models for Animation

Multi-View 6D Object Pose Estimation and Camera Motion ... · Multi-view 6D Object Pose Estimation and Camera Motion Planning using RGBD Images Juil Sock Imperial College London London,

Learning 3D Human Pose Estimation - uni-freiburg.de · University of Freiburg Seminar on Current Works in CV Existing 3D Pose Datasets A. Baak et al. showed that motion tracking techniques

A Comparative Study of Pose Representation and … Comparative Study of Pose Representation and Dynamics Modelling for Online Motion Quality Assessment Lili Tao, Adeline Paiement,

UNCOOPERATIVE OBJECTS POSE, MOTION AND INERTIA … · 2017-02-13 · UNCOOPERATIVE OBJECTS POSE, MOTION AND INERTIA TENSOR ESTIMATION VIA STEREOVISION M. Lavagna 1, V. Pesce , and

Rick Parent - CIS681 Motion Capture Use digitized motion to animate a character

3D Pose from Motion for Cross-view Action …...3D Pose from Motion for Cross-view Action Recognition via Non-linear Circulant Temporal Encoding Ankur Gupta , Julieta Martinez , James

Combining Skeletal Pose with Local Motion for Human ...jcorso/pubs/jcorso_AMDO2012.pdf · power of dynamic pose over raw skeletal pose in an activity recognition ... AMDO 2012. 2

MSP-AVATAR Corpus: Motion Capture Recordings to Study …...motion capture data, where the goal is to establish a sys-tematic evaluation of pose and motion estimation methods. It includes