bringing the real world into the game world

Bringing the Real World into the Game

World with Intel® RealSense™ Technology Geoffrey Douglas, Games Engineer

6

#buzzworkshop #gamedev @intelsoftware

About Me

#buzzworkshop #gamedev @intelsoftware

Executive Summary

• RealSense Technology can be used to develop games that bridge the gap between the real world and the virtual world to create new and powerfully immersive experiences.

• Developing with new technology like RealSense is a huge challenge. But sometimes the solutions to the hardest problems turn out to be the most amazing innovations.

• Ultimately, our goal is for RealSense to be the next big thing in gaming, and we need talented developers to help us make that happen.

4

RealSense Overview

• RealSense Cameras

• RealSense SDK Overview

• Developing with the SDK

Gaming Usages

• ChatHeads

• 3D Selfie

• Augmented Reality

Agenda

#buzzworkshop #gamedev @intelsoftware

5

RealSense Overview

#buzzworkshop #gamedev @intelsoftware

6

• Front-Facing (User-Facing)

• Short-Range

• Designed for laptops and 2-in-1s, on the front, above the screen

• Rear-Facing (World-Facing)

• Long-Range

• Designed for 2-in-1s and tablets, on the back, at the top of the device

RealSense Cameras

7

RealSense SDK

SDK

Core I/O

Modules Algorithm

Modules

C++ Interface

Algorithm

Modules

C# Interface Unity Interface JavaScript Java Interface

#buzzworkshop #gamedev @intelsoftware

Unreal Engine 4

Interface

8

RealSense SDK Middleware

Hand Tracking Hand Tracking

Speech Recognition

Blob Tracking

Head/Face Tracking

Gesture Recognition Scene Perception

Background Segmentation

3D Scanning

#buzzworkshop #gamedev @intelsoftware

#buzzworkshop #gamedev @intelsoftware 9

Using the SDK

int main() {

// Create SenseManager Object

PXCSenseManager *sm = PXCSenseManager::CreateInstance();

// Enable Color and Depth video streams

sm->EnableStream(STREAM_TYPE_COLOR, 1920, 1080, 30.0f);

sm->EnableStream(STREAM_TYPE_DEPTH, 640, 480, 30.0f);

...

#buzzworkshop #gamedev @intelsoftware 10

// Enable and Configure Middleware Module(s)

sm->Enable3DScan();

PXC3DScan* scan = sm->Query3DScan();

scan->SetConfiguration(/*...*/);

// Initialize the SenseManager pipeline

sm->Init();

...

#buzzworkshop #gamedev @intelsoftware 11

// Begin camera processing loop and acquire new frame

while(sm->AcquireFrame() == PXC_STATUS_NO_ERROR) {

// Process camera frame

PXCImage* image = scan->AcquirePreview();

// Release the working frame

sm->ReleaseFrame();

}

...

#buzzworkshop #gamedev @intelsoftware 12

// Close the SenseManager pipeline

sm->Close()

// Release the SenseManager resources

sm->Release();

return 0;

}

13

Gaming Usages

#buzzworkshop #gamedev @intelsoftware

#buzzworkshop #gamedev @intelsoftware 14

ChatHeads

#buzzworkshop #gamedev @intelsoftware

3D Selfie

15

#buzzworkshop #gamedev @intelsoftware 16

Video Demo

#buzzworkshop #gamedev @intelsoftware 17

Augmented Reality

#buzzworkshop #gamedev @intelsoftware 18

Scan-Then-Play

#buzzworkshop #gamedev @intelsoftware 19

Continuous Scanning

#buzzworkshop #gamedev @intelsoftware 20

Bonus: Head Tracking!

#buzzworkshop #gamedev @intelsoftware

Conclusion

• RealSense Technology can be used to develop new games that bridge the gap between the real world and the virtual world to create powerfully immersive experiences.

• There are a lot of new use cases that we’re exploring in 2016 and beyond that we hope will make RealSense a must-have device for gamers.

#buzzworkshop #gamedev @intelsoftware 22

Call to Action

• Contact us if you want to work together on a new RealSense game, or if you’d more information.

• Watch out for some RealSense games to hit the market this year and see what you think.

https://appshowcase.intel.com/en-us/realsense

• Get excited about the future of gaming. It’s going to be great.

#buzzworkshop #gamedev @intelsoftware 23

Thank You

https://software.intel.com/en-us/articles/ intel-realsense-sdk-plug-in-for-unreal-engine-4

https://software.intel.com/en-us/articles/ applying-intel-realsense-sdk-face-scans-to-a-3d-mesh

https://software.intel.com/en-us/articles/ intel-realsense-sdk-background-segmentation-feature

https://software.intel.com/en-us/articles/ native-intel-realsense-sdk-image-copying-in-unity

Contact: Geoffrey Douglas geoffrey.douglas@intel.com