Human Pose detection

Abhinav Golas

S. Arun Nair

Overview

Problem Previous solutions Solution, details

Problem

Segmentation of humans from video capture

Pose detection (by fitting onto body model) Resistant to noise (background etc.)

Previous procedures

View problem as sequential process1. Segmentation2. Pose detection

Problems: Not using prior knowledge of “what a human looks

like” in segmentation Uses only information from detected “foreground”

for pose detection All available information not used

Solution

Combine segmentation and pose detection as a single step Uses all available information in frame (for pose

detection) Uses prior knowledge of human body for better

segmentation

PoseCut: Bray, Kohli, Torr Model segmentation as Bayesian labeling problem

with 2 labels: foreground, background

Details

Model problem as energy minimization problem – model as an MRF

Use a basic stickman model as a human body model

Adaptive model for background – GMM Neighbourhood terms – Generalised Potts

model

MRF – Markov Random Fields

Markov property for time:P(event:t) depends on events at times k<t

Markov property for space:P(event:x) depends on events at N(x) – neighbourhood of x

Use Gibbs energy model for solving We use neighbourhood of 8 pixels

Stickman model

Basic model 26 degrees of

freedom

GMM – Gaussian Mixture Model

Model each pixel of image as a weighted sum of Gaussian functions

Adapt functions using each new frame Pixel matches expected value –

background, else foreground

Execution details

For each frame Calculate weights for GMM, Potts model For given value of 26 vector (based on degrees of

freedom of stickman model) calculate energy cost for stickman model (by distance transform)

Minimize energy for Bayesian labeling by graph cut Minimize 26 vector by repeated graph cuts by

Powell's algorithm

Sample results

A – original frame B – segmentation by

colour likelihood and contrast terms

C – when GMM terms are taken

D – with pose prior components

E – deduced pose

Comparisons