kinectarms cscw2013

KinectArmsA Toolkit for Capturing and Displaying Arm

Embodiments in Distributed Tabletop Groupware

Aaron Genest, Carl Gutwin, Anthony Tang*, Michael Kalyn, and Zenja Ivkovic

University of Saskatchewan and *University of Calgary

Develop a toolkit to help with representing gestures in

groupware.

easycheap

expressiveextensible

KinectArms:KinectTable -- KinectViz

Easy:KinectTable

Kinect

Table

Recognize Table

Person

Recognize Table

Recognize Table

Recognize Table

Recognize Table

Recognize Table

Cheap:KinectTable

Capture Arms and Hands

Raw Depth Image

Canny Edge Detection

Generate Arm Blobs

Detect Arm Crossings

Create Structure

Separate Arm Blobs

Cheap and Easy:KinectTable

Get Data

Raw Image DataKinectArmsClient *client = KinectArmsGetClient();KinectData data;client->GetData(data);DepthImage& depthImage = data.depthImage;

Table Data BinaryImage& maskImage = data.tableMaskImage;

Get Data

Arm and Finger Data

client -> GetData(data);Arm& arm1 = data.arms[0];int fingerHeight = client->getHeight(arm1.fingers[0]);

Get Data

Expressive:KinectViz

Hand and Arm Representation

Video Separation

Stick Figures

Hand and Arm Representation

Stick Figures

client-­‐>GetData(data);viz.updateData(data);viz.applyEffects();

viz.getEffect(kSkeleton).enabled­=­true;

Height

Expanding Circle

Rainbow ArmsShadow

Height

Expanding Circle

Rainbow ArmsShadow

viz.getEffect(kShadow).enabled­=­true;

viz.getEffect(kRainbow).enabled­=­true;

viz.getEffect(kPointerCircle).enabled­=­true;

Changing Visibility

Outline

Transparency

Changing Visibility

Outline

Transparencyviz.getEffect(kTransparency).enabled­=­true;

viz.getEffect(kOutline).enabled­=­true;

User Identification

Tattoos

User Identification

Tattoos

viz.getEffect(kTattoo).enabled­=­true;

Historical Traces

Motion Blur

Motion Lines

Historical Traces

Motion Blur

Motion Lines

viz.getEffect(kTraces).enabled­=­true;viz.getEffect(kMotionBlur).enabled­=­true;

Extensible

Combinations

...or new visualizations

github.com/aarongenest/KinectArms

Communication

Interaction

University of Waterloo

Queen’s University

UOIT

Olin College, MA

University of California, San Diego

Aaron Genest, Carl Gutwin, Anthony Tang*, Michael Kalyn, and Zenja Ivkovic

University of Saskatchewan and *University of Calgary

github.com/aarongenest/KinectArms

@aarongenest aaron.genest@usask.ca