605 working with metal advanced

8/10/2019 605 Working With Metal Advanced

1/161

2014 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

#WWD

Working With MetalAdvanced

Graphics and Games

Session 605

Gokhan Avkarogullari

GPU Software

Aaftab Munshi

GPU Software

Serhat Tekin

GPU Software


2/161

Agenda

Introduction to Metal

Fundamentals of Metal

Building a Metal application

Metal shading language

Advanced Metal Deep dive into creating a graphics application with Metal

Data-Parallel Computing with Metal

Developer Tools Review


3/161

Creating Multi-Pass Graphics

Applications with Metal


4/161

Graphics Application with Multiple Passe

Multiple framebuffer configurations

Render to off-screen and on-screen textures

Meshes that are used with multiple shaders

Multiple encoders


5/161

Deferred Lighting with Shadow Maps

Shadow Map

Depth-only render from the perspective of the directional light

Deferred Lighting

Multiple render targets

Framebuffer fetch for in-place light accumulation Stencil buffer for light culling


6/161


Shadow Map

Depth-only render from the perspective of the directional light

Deferred Lighting

Multiple render targets

Framebuffer fetch for in-place light accumulation Stencil buffer for light culling

Dont worry about the algorithm

Focus on the details of how the API is used


7/161


Command Queue

Render Command Encoderfor Deferred Lighting Pass

Command Buffer #1

Render Command Encoderfor Shadow Pass

Pipeline State

Shaders Blend, etc.

Depth State

Buffers

Textures

Pipeline State

Shaders Blend, etc.

Depth State

Buffers

Textures


8/161

Demo

Deferred Lighting


9/161

Render Setup

Actions to take once at application start time

Actions that are taken as needed

Level load time

Texture streaming

Mesh streamingActions to take every frame

Actions to take every render-to-texture pass


10/161

Render SetupDo once

Create device

Create command queue


11/161

Render SetupDo as needed

Create framebuffer textures

Create render pass descriptors

Create buffers for meshes

Create render pipeline objects

Create texturesCreate state objects

Create uniform buffers


12/161

Render SetupDo every frame

Create command buffer

Update frame-based uniform buffers

Submit command buffer


13/161

Render SetupDo every render to texture pass

Create command encoder

Draw many times

Update uniform buffers

Set states

Make draw calls


14/161

Render SetupDo as needed




Create render pipeline objects

Create texturesCreate state objects



15/161

Render SetupA word on descriptors

Descriptors are like blueprints

Once the object is created the connection to the descriptor is gone

Changing descriptors will not change the object

Same descriptor can be reused to create another object

Descriptor can be modified and then reused to create a new object


16/161

Render SetupCreating render pass descriptors

MTLRenderPassDescriptor

Color Color

Color Color

Depth Stencil


17/161


MTLRenderPassDes

Color Co

Color Co

Depth Ste


18/161


MTLRenderPassAttachmentDescriptor

MTLTexture

MTLResolveTexture

loadActionstoreAction

-

MTLRenderPassDes

Color Co

Color Co

Depth Ste


19/161


Texture

MTLRenderPassAttachmentDescriptor

MTLTexture

MTLResolveTexture

loadActionstoreAction

-

MTLRenderPassDes

Color Co

Color Co

Depth Ste


20/161

Render SetupRender pass descriptor for shadowMap pass

shadowRenderPassDescriptor

Depth


21/161


// Create texture

id shadow_texture;

MTLTextureDescriptor *shadowTextureDesc = [MTLTextureDescriptor

texture2DDescriptorWithPixelFormat: MTLPixelFormatDepth32Float

width: 1024

height: 1024

mipmapped: NO];shadow_texture = [device newTextureWithDescriptor: shadowTextureDesc];


22/161


// Create texture

id shadow_texture;



width: 1024

height: 1024

mipmapped: NO];shadow_texture = [device newTextureWithDescriptor: shadowTextureDesc];


23/161


// Create texture

id shadow_texture;



width: 1024

height: 1024

mipmapped: NO];shadow_texture = [devicenewTextureWithDescriptor: shadowTextureDesc];


24/161


// Create render pass descriptor

MTLRenderPassDescriptor *shadowMapPassDesc =

[MTLRenderPassDescriptor renderPassDescriptor];


25/161





// Set the texture on the render pass descriptor

shadowMapPassDesc.depthAttachment.texture = shadow_texture;

d


26/161







R d S


27/161







// Set other properties on the render pass descriptor

shadowMapPassDesc.depthAttachment.clearValue = MTLClearValueMakeDepth(1.0

shadowMapPassDesc.depthAttachment.loadAction = MTLLoadActionClear;

shadowMapPassDesc.depthAttachment.setStoreAction = MTLStoreActionStore;

R d S


28/161









shadowMapPassDesc.depthAttachment.loadAction = MTLLoadActionClear;

shadowMapPassDesc.depthAttachment.setStoreAction = MTLStoreActionStore;

R d S t


29/161









shadowMapPassDesc.depthAttachment.loadAction= MTLLoadActionClear;

shadowMapPassDesc.depthAttachment.setStoreAction= MTLStoreActionStore;

R d S t


30/161

Render SetupRender pass descriptor for the Deferred Lighting pass

deferredPassDesc

Color Color

Color Color

Depth Stencil

R d S t


31/161


// Create descriptorMTLTextureDescriptor *attachmentX_texture_desc = [MTLTextureDescriptor

texture2DDescriptorWithPixelFormat: MTLPixelFormatBGRA8Unorm

width: desc.width

height: desc.height

mipmapped: NO];

R d S t


32/161




width: desc.width

height: desc.height

mipmapped: NO];

// Create textures based on the descriptors

gbuffer_texture1 = [device newTextureWithDescriptor: attachmentX_tex_desc

Render Setup


33/161




width: desc.width

height: desc.height

mipmapped: NO];



Render Setup


34/161




width: desc.width

height: desc.height

mipmapped: NO];



// Modify descriptor and create new texture

[attachmentX_texture_desc setPixelFormat: ];


Render Setup


35/161




width: desc.width

height: desc.height

mipmapped: NO];






Render Setup


36/161




width: desc.width

height: desc.height

mipmapped: NO];






Render Setup


37/161


// Create render pass descriptorMTLRenderPassDescriptor *deferredPassDesc =


Render Setup


38/161


// Describe color attachment 0deferredPassDesc.colorAttachments[0].texture = nil; //will come from draw

Render Setup


39/161


// Describe color attachment 0deferredPassDesc.colorAttachments[0].texture = nil; //will come from draw

Render Setup


40/161


// Describe color attachment 0deferredPassDesc.colorAttachments[0].texture = nil;

deferredPassDesc.colorAttachments[0].clearValue = clearColor1;

deferredPassDesc.colorAttachments[0].loadAction = MTLLoadActionClear;

deferredPassDesc.colorAttachments[0].storeAction = MTLStoreActionStore;

Render Setup


41/161






Render Setup


42/161






// Describe color attachment 1

deferredPassDesc.colorAttachments[1].texture = gbuffer_texture1;

Render Setup


43/161








Render Setup


44/161








Render Setup


45/161










deferredPassDesc.colorAttachments[1].storeAction = MTLStoreActionDontCare

Render Setup


46/161











Render Setup


47/161











Render Setup


48/161

pDo as needed



Create textures


Create state objects

Create pipeline objects


Creating Textures


49/161

g

// Copy texture data to bitmapDataunsigned Npixels = tex_info.width * tex_info.height;

id texture = [device newTextureWithDescriptor: ];

Creating Textures


50/161



[texture replaceRegion: bitmapData

g

Creating Textures


51/161



[texture replaceRegion: bitmapData

g

Creating Buffers for Meshes


52/161

g

float4 temple[100];

spriteBuffer = [device newBufferWithBytes: &temple

length: sizeof(temple)

options: 0];

Creating Buffers for Meshes


53/161

g

float4 temple[100];

spriteBuffer = [device newBufferWithBytes: &temple

length: sizeof(temple)

options: 0];

Creating State Objects


54/161

g j

MTLDepthStencilDescriptor*desc = [[MTLDepthStencilDescriptor alloc] initdesc.depthCompareFunction = MTLCompareFunctionLessEqual;

desc.depthWriteEnabled = YES;

MTLStencilDescriptor *stencilStateDesc = [[MTLStencilDescriptor alloc] in

stencilState.stencilCompareFunction = MTLCompareFunctionAlways;

stencilState.stencilFailureOperation = MTLStencilOperationKeep;

desc.frontFaceStencilDescriptor = stencilStateDesc;

desc.backFaceStencilDescriptor = stencilStateDesc;

id shadowDepthStencilState= [devicenewDepthStencilStateWithDescriptor: desc];

Creating State Objects


55/161

MTLDepthStencilDescriptor*desc = [[MTLDepthStencilDescriptor alloc] initdesc.depthCompareFunction = MTLCompareFunctionLessEqual;

desc.depthWriteEnabled = YES;

MTLStencilDescriptor*stencilStateDesc = [[MTLStencilDescriptor alloc] in

stencilState.stencilCompareFunction = MTLCompareFunctionAlways;

stencilState.stencilFailureOperation = MTLStencilOperationKeep;

desc.frontFaceStencilDescriptor = stencilStateDesc;

desc.backFaceStencilDescriptor= stencilStateDesc;

id shadowDepthStencilState = [devicenewDepthStencilStateWithDescriptor: desc];

GPU Render Pipeline


56/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

BuffersTextures

Samplers

Buffers

Render Pipeline State Object


57/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers



58/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Shaders



59/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Framebuffer configuration

Number of render targets, pixel format, sample co



60/161

Buffers

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Depth and stencil state



61/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Render Pipeline State ObjectNot Includ


62/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Inputs

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers



63/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Outputs



64/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Primitive setup

Cull mode, facing orientation

depth clipping, polygon mode



65/161

Vertex Fetch

Vertex Shader

Rasterizer

FragmentShader

Framebuffer

Primitive Setup

Buffers

Buffers

Textures

Samplers

Buffers

Textures

Samplers

Buffers

Viewport and scissorDepth bias/slope/clamp

Occlusion queries

Creating Render Pipeline State Object


66/161

Every draw call requires a render pipeline state object to be setSame mesh usually has multiple pipeline state objects

The temple object in our demo is rendered twice

- The shadow passDepth only render with simple vertex shader

- The deferred passRendered to generate g-buffer attributes

- Each one requires a different render pipeline state object to be created

Creating Render Pipeline State Objecth d M


67/161

shadowMap pass

// Create the descriptorMTLRenderPipelineDescriptor *desc = [MTLRenderPipelineDescriptor new];

Creating Render Pipeline State Objecth d M


68/161

shadowMap pass

// Create the descriptor

MTLRenderPipelineDescriptor *desc = [MTLRenderPipelineDescriptor new];

// Get the shaders from the library

id zOnlyVert = [zOnlyLibrary newFunctionWithName:@"ZOnly"];

Creating Render Pipeline State ObjectshadowMap pass


69/161

shadowMap pass







70/161

shadowMap pass





// Set the states

desc.label = @"Shadow Render";desc.vertexFunction = zOnlyVert;

desc.stencilWriteEnabled = false;

desc.depthWriteEnabled = true;

desc.fragmentFunction = nil; //depth write only

desc.depthAttachmentPixelFormat = pixelFormat;



71/161

shadowMap pass





// Set the states

desc.label = @"Shadow Render";desc.vertexFunction = zOnlyVert;


desc.depthWriteEnabled = true;

desc.fragmentFunction = nil; //depth write only




72/161

shadowMap pass



desc.label = @"Shadow Render";

desc.vertexFunction = zOnlyVert;


desc.depthWriteEnabled = true;desc.fragmentFunction = nil; //depth write only


// Create the render pipeline state object

id pipeline = [device

newRenderPipelineStateWithDescriptor: desc error: &err];



73/161

shadowMap pass



desc.label = @"Shadow Render";

desc.vertexFunction = zOnlyVert;


desc.depthWriteEnabled = true;desc.fragmentFunction = nil; //depth write only


// Create the render pipeline state object

id pipeline = [device

newRenderPipelineStateWithDescriptor: desc error: &err];

Creating Render Pipeline State ObjectDeferred Lighting pass


74/161

Deferred Lighting pass

desc.vertexFunction = gBufferVert;

desc.fragmentFunction = gBufferFrag;

desc.colorAttachments[0].pixelFormat = gbuffer_texture0.pixelFormat;


desc.depthAttachmentPixelFormat = depth_texture.pixelFormat;

desc.stencilAttachmentPixelFormat = stencil_texture.pixelFormat;



75/161










76/161








Render SetupDo every frame


77/161

Do every frame

Create command bufferUpdate frame-based uniform buffers

Submit command buffer

Render SetupCreate and submit command buffer


78/161

Create and submit command buffer

// BeginFrame

commandBuffer = [commandQueue commandBuffer];

// EndFrame

[commandBuffer addPresent: drawable];

[commandBuffer commit];

commandBuffer = nil;



79/161


// BeginFrame


// EndFrame






80/161


// BeginFrame


// EndFrame




Render SetupDo every render to texture pass


81/161

y p

Create command encoder

Draw many times

Update uniform buffers

Set states

Make draw calls

Render SetupshadowMap pass render encoding


82/161

p p g

// Create encoder

id encoder = [commandBuffer

renderCommandEncoderWithDescriptor: shadowMapPassDesc];



83/161

p p g

// Create encoder



// Set states and draw

[encoder setRenderPipelineState: shadow_render_pipeline];

[encoder setDepthStencilState: shadowDepthStencilState];

[encoder setVertexBuffer: structureVertexBuffer offset:0 atIndex: 0

[encoder drawIndexedPrimitives:



84/161

p p g

// Create encoder










85/161

g

// Create encoder








// end encoding

[encoder endEncoding];



86/161

// Create encoder








// end encoding


Render SetupDeferred Lighting pass render encoding


87/161

deferredPassDesc.colorAttachments[0].texture = texture_from_drawable;



88/161


// Create encoder


renderCommandEncoderWithDescriptor: deferredPassDesc];



89/161


// Create encoder


renderCommandEncoderWithDescriptor: deferredPassDesc];

// End encoding


Agenda


90/161


Fundamentals of Metal

Building a Metal application


Advanced Metal




91/161


Aaftab MunshiGPU Software

Data-Parallel Computing with MetalWhat youll learn


92/161



93/161

What is data-parallel computing?



94/161


Data-parallel computing in Metal



95/161



Writing data-parallel kernels in Metal



96/161



Writing data-parallel kernels in Metal

Executing kernels in Metal

Data-Parallel ComputingA brief introduction


97/161



98/161

Similar and independent computations on multiple data elements



99/161

Similar and independent computations on multiple data elements

ExampleBlurring an image

Same computation for each input

All results are independent

Data-Parallelism in Metal


100/161



101/161

Code that describes computation is called a kernel



102/161


Independent computation instance

Work-item



103/161


Independent computation instance

Work-item

Work-items that execute together

Work-group

Cooperate by sharing data Can synchronize execution

Data-Parallelism in MetalComputation domain


104/161



105/161

Number of dimensions

1D, 2D, or 3D



106/161


1D, 2D, or 3D

For each dimension specify

Number of work-items in work-group also known as work-group size

Number of work-groups



107/161


1D, 2D, or 3D

For each dimension specify

Number of work-items in work-group also known as work-group size

Number of work-groups

Choose the dimensions that are best for your algorithm

Pseudo Code for a Data-Parallel Kernel


108/161

voidsquare(const float* input,

float* output,uint id

{output[id] = input[id] * input[id];

}



109/161

#include

using namespace metal;voidsquare(const float* input,



}



110/161

#include

using namespace metal;kernel voidsquare(const float* input,



}



111/161

#include

using namespace metal;kernelvoidsquare(const global float* input [[ buffer(0)]],

global float* output [[ buffer(1)]],uint id


}

Metal Kernel


112/161

#include

using namespace metal;kernelvoidsquare(const global float* input [[ buffer(0)]],

global float* output [[ buffer(1)]],uint id [[ global_id]])


}

Another Kernel Example in MetalUsing images in a kernel


113/161



114/161

kernel void

horizontal_reflect(texture2d src [[ texture(0) ]],texture2d dst [[ texture(1) ]], uint2id[[ global_id ]]){ float4 c = src.read(uint2(src.get_width()-1-id.x, id.y)); dst.write(c, id);}



115/161

kernel void




116/161

kernel void




117/161

kernel void




118/161

kernel void




119/161

kernel void


Built-in Kernel VariablesAttributes for kernel arguments


120/161

kernel void

my_kernel(texture2d img [[ texture(0) ]], ushort2 gid [[ global_id]],uint glinear [[ global_linear_id]],ushort2 lid [[ local_id]],ushort linear [[ local_linear_id]],uint wgid [[ work_group_id]],)

{

}


k l id


121/161

kernel void

my_kernel(texture2d img [[ texture(0) ]], ushort2 gid [[ global_id]],uint glinear [[ global_linear_id]],ushort2 lid [[ local_id]],ushort linear [[ local_linear_id]],uint wgid [[ work_group_id]],)

{

}


k l id


122/161

kernel voidmy_kernel(texture2d img [[ texture(0) ]], ushort2 gid [[ global_id]],

uint glinear [[ global_linear_id]],ushort2 lid [[ local_id]],ushort linear [[ local_linear_id]],uint wgid [[ work_group_id]],)

{

}


kernel void


123/161



{

}


kernel void


124/161



{

}


kernel void


125/161



{

}


kernel void


126/161



{

}


127/161

Executing Kernels in MetalPost-processing example

Post-Processing KernelKernel source


128/161


kernel void


129/161

kernel voidpostprocess_filter(texture2d inImage[[ texture(0) ]], texture2d outImage [[ texture(1) texture2d curveImage [[ texture(2) ]], constant Parameters& param [[ buffer(0) ]], uint2 gid [[ global_id ]]){ // Transform global ID using param.transformMatrix

float4 color = inImage.sample(s, transformedCoord);

// Apply post-processing effect

outImage.write(color, gid);}


kernel void


130/161

postprocess_filter(texture2d inImage[[ texture(0) ]], texture2d outImage [[ texture(1) texture2d curveImage [[ texture(2) ]], constant Parameters& param [[ buffer(0) ]], uint2 gid [[ global_id ]]){ // Transform global ID using param.transformMatrix





kernel void


131/161






kernel void


132/161





Post-Processing KernelProcessing multiple pixels/work-item

constexpr constant int num_pixels_work_item = 4;


133/161

kernel voidpostprocess_filter(, uint2 gid [[global_id]],

uint2 lsize [[local_size]]){ for (int i=0; i


134/161




135/161




136/161

Executing a KernelCompute command encoder

//Load library and kernel function


137/161

id library = [device newLibraryWithFile:libnameerror:&err];id filterFunc = [library

newFunctionWithName:@postprocess_filter]




138/161



// Create compute state

id filterKernel=[device newComputePipelineStateWithFunction:filterFuncerror:&err];




139/161





// Create compute command encoder




140/161





// Create compute command encoder

id computeEncoder=[commandBuffer computeCommandEncoder];

Executing a KernelEncode compute commands


141/161


// Set compute state

[ t E d tC t Pi li St t filt K l]


142/161

[computeEncoder setComputePipelineState:filterKernel];


// Set compute state

[comp teEncoder setComp tePipelineState filterKernel]


143/161

[computeEncoder setComputePipelineState:filterKernel];

// Set Resources used by kernel

[computeEncoder setTexture:inputImage atIndex:0];[computeEncoder setTexture:outputImage atIndex:1];[computeEncoder setTexture:curveImage atIndex:2];[computeEncoder setBuffer:params offset:0 atIndex:0];



144/161


// Calculate the work-group size and number of work-groups

MTLSize wgSize = { 16 16 1 };


145/161

MTLSize wgSize = { 16, 16, 1 };MTLSize numWorkGroups = { (outputImage.width + wgSize - 1)/wgSize. (outputImage.height + wgSize - 1)/wgSize 1 };





146/161






147/161


// Execute Kernel

[computeEncoder executeKernelWithWorkGroupSize:wgSizeworkGroupCount:numWorkGroups];



MTLSize wgSize = { 16, 16, 1 };


148/161

MTLSize wgSize { 16, 16, 1 };MTLSize numWorkGroups = { (outputImage.width + wgSize - 1)/wgSize. (outputImage.height + wgSize - 1)/wgSize 1 };

// Execute Kernel




MTLSize wgSize = { 16, 16, 1 };


149/161

MTLSize wgSize { 16, 16, 1 };MTLSize numWorkGroups = { (outputImage.width + wgSize - 1)/wgSize. (outputImage.height + wgSize - 1)/wgSize 1 };

// Execute Kernel


// Finish encoding

[computeEncoder endEncoding];

Executing a KernelSubmit commands to the GPU

// Commit the command buffer



150/161

[ ];


151/161

Demo

Post-processing kernels

Agenda



152/161

Fundamentals of Metal Building a Metal application


Advanced Metal



153/161

ToolsDebugging and profiling Metal applications in Xcode

Summary


154/161

Summary

A deeper dive into Metal


155/161

Structuring your application for Metal Using descriptors and state objects for rendering

Multi-pass encoding in Metal

Summary



156/161




How data-parallelism works in Metal

Write and execute kernels in Metal

Summary


S i li i f l


157/161




How data-parallelism works in Metal

Write and execute kernels in Metal

Tools

How to create and compile Metal Shaders in Xcode

Debug and profile a Metal application

More Information

Filip IliescuGraphics and Games Technologies Evangelist

fili @ l


158/161

[email protected]

Allan SchafferGraphics and Games Technologies Evangelist

[email protected]

Documentation

http://developer.apple.com

Apple Developer Forumshttp://devforums.apple.com

Related Sessions
http://devforums.apple.com/http://devforums.apple.com/


159/161

Labs


160/161


161/161

605 working with metal advanced

Documents