“mdl materials to glsl shaders: theory and...
TRANSCRIPT
April 4-7, 2016 | Silicon Valley
www.esi-group.com
Andreas Süßenbach, NVIDIA
Andreas Mank, ESI Group
04/04/2016
MDL MATERIALS TO GLSL SHADERS THEORY AND PRACTICE
2
“a pioneer and world-leading provider in Virtual Prototyping”
— www.esi-group.com/
3
INTERACTION
FIDELITY
BEHAVIOR APPEARANCE
4
MDL IN COMMERCIAL PRODUCTS
4/3/2016
MATERIAL CONSTRUCTION
Substance Designer
MATERIAL TWEAKING
Cinema 4D Maya mental ray others
MATERIAL SHARING (LIBRARY)
5
MDL IN CUSTOM RENDERERS
4/3/2016 03.04.2016
MATERIAL TWEAKING MATERIAL
CONSTRUCTION
MATERIAL SHARING (LIBRARY) N
VID
IA
IRAY
DEFINITION
MD
L S
DK
IMPLEMENTATION
GLSL (Rasterizer)
CUDA (Ray Tracer)
6
AGENDA
What is MDL, what is the MDL SDK ?
How do I use the MDL SDK ?
How do I map MDL materials to GLSL shaders ?
What is the issue with material parameters?
7
WHAT IS MDL, WHAT IS THE MDL SDK ?
8
WHAT IS MDL ?
Declarative language to specify visual material properties
No shader language !
Renderer agnostic
export material tintedStuff( color parTint = color(.6,.2,.2) ) = material ( surface: material_surface( scattering: df::specular_bsdf( tint: parTint ) ) );
9
MDL MATERIAL FIELDS
material_surface:
surface
material_surface:
backface
material_geometry material_volume
bool:
thin_walled
color:
ior
bsdf: scattering
material_emission: emission
bsdf: scattering
material_emission: emission
float3: displacement
float: cutout_opacity
float3: normal
vdf: scattering
absorption_coefficient
scattering_coefficient
10
WHAT IS THE MDL SDK ?
A dynamic library providing a C++ API
Used to load and compile MDL materials
Can be used to create and modify materials as well
Used to get detailed information out of the compiled material
11
HOW DO I USE THE MDL SDK ?
12
USING MDL SDK 1/5
Initializing
Material Compiling
Material Parsing
Expression Parsing
Call Parsing
HINSTANCE dll = LoadLibrary("libmdl_sdk.dll");
INeuray_factory* factory = (INeuray_factory*)GetProcAddress(dll, "mi_neuray_factory");
mi::base::Handle<mi::neuraylib::INeuray> neuray = factory(0,MI_NEURAYLIB_API_VERSION );
mi::base::Handle<mi::neuraylib::IMdl_compiler> mdlCompiler = neuray->get_api_component<mi::neuraylib::IMdl_compiler>();
13
USING MDL SDK 2/5
Initializing
Material Compiling
Material Parsing
Expression Parsing
Call Parsing
mi::Sint32 reason = mdlCompiler->load_module ("::nvidia::vMaterials::AEC::Concrete::concrete_blocks");
mi::base::Handle<const mi::neuraylib::IModule> module( mdlCompiler->access<mi::neuraylib::IModule>( "mdl::nvidia::vMaterials::AEC::Concrete::concrete_blocks"));
for ( mi::Size i=0 ; i<module->get_material_count() ; i++ )
{
mi::base::Handle<mi::neuraylib::IMaterial_definition const> materialDefinition(mdlCompiler->access <mi::neuraylib::IMaterial_definition>(module->get_material(i));
mi::base::Handle<mi::neuraylib::IMaterial_instance> materialInstance
(materialDefinition->create_material_instance(0,&result));
mi::base::Handle<mi::neuraylib::ICompiled_material> compiledMaterial
(materialInstance->create_compiled_material (mi::neuraylib::IMaterial_instance::CLASS_COMPILATION,1.0f ,&result);
}
14
USING MDL SDK 3/5
Initializing
Material Compiling
Material Parsing
Expression Parsing
Call Parsing
for (mi::Size i=0; i<compiledMaterial->get_parameter_count(); i++)
{
char const* parameterName = compiledMaterial-> get_parameter_name(i);
mi::base::Handle<mi::neuraylib::IValue const> value (compiledMaterial->get_argument(i));
}
for (mi::Size i=0; i<compiledMaterial->get_temporary_count(); i++)
{
mi::base::Handle<mi::neuraylib::IExpression const> expression
(compiledMaterial->get_temporary(i));
}
mi::base::Handle<mi::neuraylib::IExpression const> surfaceExpression
(compiledMaterial->get_field("surface"));
mi::base::Handle<mi::neuraylib::IExpression const> backfaceExpression
(compiledMaterial->get_field(„backface"));
15
USING MDL SDK 4/5
Initializing
Material Compiling
Material Parsing
Expression Parsing
Call Parsing
switch( expression->get_kind() )
{
case mi::neuraylib::IExpression::EK_CONSTANT: ...
break;
case mi::neuraylib::IExpression::EK_PARAMETER: ...
break;
case mi::neuraylib::IExpression::EK_TEMPORARY: ...
break;
case mi::neuraylib::IExpression::EK_DIRECT_CALL: ...
break;
}
16
USING MDL SDK 5/5
Initializing
Material Compiling
Material Parsing
Expression Parsing
Call Parsing
mi::base::Handle<mi::neuraylib::IType const> type(call-> get_type());
mi::base::Handle<mi::neuraylib::IExpression_list const> arguments (call->get_arguments());
for (mi::Size i=0; i<arguments->get_size(); i++)
{
char const* name = arguments->get_name(i);
mi::base::Handle<mi::neuraylib::IExpression const> argument
(arguments->get_expression(i));
}
17
MDL SDK: MAP MDL TO AN EXPRESSION TREE
MDL Material Expression Tree
export material tintedStuff( color parTint = color(.6,.2,.2) ) = material ( surface: material_surface( scattering: df::specular_bsdf( tint: parTint ) ) );
tintedStuff
surface
scattering emission
backface
scattering emission
18
SIMPLIFY YOUR WORK: MDLParser
Simple C++ class to derive from
About 40 pure virtual functions (visitor pattern)
Simple string and value based interface
“Guides” you through the Expression tree of a material
Hides all the intricacies of the MDL SDK
If you just want to convert MDL Materials into something
19
HOW DO I MAP MDL MATERIALS TO GLSL SHADERS ?
20
MAPPING MDL TO GLSL
MDL
Parameters
Temporaries
Constants
Operators (+,-,…)
Function calls
Expression tree
GLSL
uniforms
global variables
42;
temporary3 + n * color;
snippets !!
stitched snippets !!
21
float mdl_math_minValue( in vec3 a ) { return( min( min( a.x, a.y ), a.z ) ); }
GLSL SNIPPETS: FUNCTIONS
vec4 mdl_df_weightedLayer( in float weight, in vec4 layer, in vec4 base ) { return( mix( base, layer, weight ) ); }
22
vec4 specularReflectionBSDF( in vec3 N, in vec3 L, in vec3 lightSpecular , in vec3 materialSpecular ) { const float shininess = 256.0f; vec4 rgba = vec4( 0.0f, 0.0f, 0.0f, 1.0f ); float cosTheta = dot( N, L ); if ( 0.0f < cosTheta ) { vec3 R = reflect( -L, N ); float cosAlpha = max( 0.0f, dot( R, viewDir ) ); float shine = pow( cosAlpha, shininess ); rgba.rgb = shine * lightSpecular * materialSpecular; } return( rgba ); }
GLSL SNIPPETS: BSDF
23
GLSL SNIPPETS
A snippet per BSDF (currently, there are 6 different)
A snippet for each function (currently, there are about 50)
A snippet for some commonly used helper functions
-> about 80 snippets
24
mdl_base_fileBumpTexture(bump_texture, bump_strength * 4, mono_average, temporary14, vec2(0,1), vec2(0,1), wrap_repeat, wrap_repeat, temporary15, false )
GLSL SNIPPETS: STITCHED TOGETHER
::mdl::base::file_bump_texture( texture: bump_texture, bump_source: ::base::mono_average, scaling: float2((1.f/texture_scale.x) * 0.5f, (1.f/texture_scale.y) * 0.5f), translation: texture_translate, clip: false, factor: bump_strength * 4.f, texture_space: uv_space_index)
25
GLSL SNIPPETS: WHAT TO DO WITH THEM ?
Fixed shader skeletons !!
Call some functions in those skeletons
Fill the bodies of those functions with some stitched snippets
-> resembles kind of “virtual function calls”
26
void main(void) { stateNormal = normalize(varNormal); if (! gl_FrontFacing) { stateNormal = - stateNormal; } texCoord0 = varTexCoord0; tangent = normalize(varTangent); binormal = normalize(varBinormal); viewDir = normalize(varEyePos - varWorldPos); vec4 rgba = vec4(0.0f, 0.0f, 0.0f, 0.0f); if (0.0f < evalCutoutOpacity(stateNormal)) { vec3 normal = evalNormal(normal); materialIOR = evalIOR(normal); vec3 materialEmissive = vec3(0.0f, 0.0f, 0.0f); bool useFront = gl_FrontFacing; if (useFront) { materialEmissive = evalMaterialEmissiveFront(normal); } else { // there's no emission on the back-side, unless thinWalled is true useFront = !evalThinWalled(); if (!useFront) { materialEmissive = evalMaterialEmissiveBack(normal);
} materialIOR = 1.0f / materialIOR; } rgba = vec4(materialEmissive, 0.0f); if (0 < sys_NumLights) { vec3 lightAmbient; for (int i=0; i<sys_NumLights; i++) { sampleLight(sys_Lights[i], varWorldPos, lightDir, lightAmbient, lightDiffuse, lightSpecular); rgba += useFront ? evalColorFront(normal) : evalColorBack(normal); } rgba.a /= sys_NumLights; } else { rgba.a = 1.0f; } rgba.a *= alphaCutout; if (0.0f < rgba.a) { lightDir = reflect(-viewDir, normal); rgba.rgb += (useFront ? evalEnvironmentFront(normal) : evalEnvironmentBack(normal)).rgb; } } emitColor(rgba); }
MASTER FRAGMENT SHADER
27
float cutoutOpacity = evalCutoutOpacity(stateNormal);
if (0.0f < cutoutOpacity)
{
vec3 normal = evalNormal(stateNormal);
materialIOR = evalIOR(normal);
FRAGMENT SHADER DETAIL
28
“VIRTUAL” FUNCTIONS USED
evalCutoutOpacity
evalNormal
evalIOR
evalMaterialEmissiveFront
evalMaterialEmissiveBack
evalThinWalled
evalColorFront
evalColorBack
evalEnironmentFront
evalEnvironmentBack
29
YOUR WORK
Create a set of shader skeletons that fit your needs
Specify a set of “virtual” functions to be filled
Implement the snippets to be used for the materials
Stitch the snippets for those functions according to the materials
30
WHAT IS THE ISSUE WITH MATERIAL PARAMETERS ?
31
MATERIAL PARAMETERS
Compiled material just has simple parameters!
MDL provides great flexibility on material parameters
Parameter types can range from simple values to call graphs
The MDL compiler does some mapping from complex to simple
-> some parameters of a compiled material might not have a name !
32
if ( hasParameter( compiledMaterial, name ) ) { // real-time case: just feed changed parameter into your shader } else { // recompile material ! if ( previousHash == currentHash ) { // interactive case: feed all (unnamed) parameters into your shader } else { // recreate your shader !! // feed all parameters into your shader } }
PSEUDO-CODE TO HANDLE PARAMETER CHANGES
33
CONCLUSION
Material Designer: pay attention to exposed material parameters
Application Developer: be prepared to also support the complex parameters
34
SUMMARY
Have a set of fixed shader skeletons calling specified functions
Fill the bodies of those functions with some stitched-in GLSL snippets
Pay attention to complex material parameters
35
DEMO: MATERIAL CREATION
36
DEMO: MATERIAL CREATION
37
DEMO: MATERIAL CREATION
38
DEMO: MATERIAL CREATION
39
DEMO: MATERIAL CREATION
40
mdl 1.0; import df::*; import state::*; import base::*; import tex::*; import anno::*; import math::*; export material scratched_silver ( color base_color = color(0.9, 0.9, 0.9) [[ anno::display_name("Color"), anno::description("The color of the material") ]], uniform texture_2d roughness_texture = texture_2d("resources/scratch_damage.png", tex::gamma_srgb) [[ anno::display_name("Roughness texture"), anno::description("Roughness of the surface finish") ]] ) = let { base::texture_coordinate_info coordinate = base::transform_coordinate(
transform: base::rotation_translation_scale(0.0,0.0,float3(1, 1, 1)) ); float roughness = base::file_texture( texture: roughness_texture, mono_source: base::mono_average, uvw: coordinate ).mono; bsdf glossy_bsdf = df::simple_glossy_bsdf ( mode: df::scatter_reflect, tint: base_color, roughness_u: (roughness - 0.1) / 0.9 ); } in material ( surface: material_surface( scattering: glossy_bsdf ) );
SCRATCHED_SILVER.MDL
simple_glossy_bsdf
0.9
roughness_texture
.mdl .cu .ptx
41
SCRATCHED_SILVER.CU rtTextureSampler<float4, 2> roughness_texture;
rtDeclareVariable(float, roughness_texture_gamma, , "Roughness texture gamma: Gamma value for texture Roughness texture");
rtDeclareVariable(float3, base_color, , "Color: The color of the material");
class MaterialGetter_0 …
class GlossyGetter_1 … // simple_glossy_bsdf // mdl::df::simple_glossy_bsdf
class EmissionGetter_2 … // material emission
class EmissionGetter_3 … // material emission
class GeometryGetter_5 … // material geometry
typedef Glossy<GlossyGetter_1> Glossy_0;
typedef MaterialEmission<EDF, EmissionGetter_2> Emission_1;
typedef MaterialSurface<Glossy_0, Emission_1> Surface_2;
typedef MaterialEmission<EDF, EmissionGetter_3> Emission_3;
typedef MaterialSurface<BSDFs::Empty, Emission_3> Surface_4;
typedef MaterialVolume<VDF, VolumeGetter_4> Volume_5;
typedef MaterialGeometry<GeometryGetter_5> Geometry_6;
typedef Material<Surface_2, Surface_4, Volume_5, Geometry_6, MaterialGetter_0> TheMaterial;
RT_PROGRAM void monteCarloClosestHit() {
const TheMaterial mat = TheMaterial(
Surface_2(
Glossy_0(),
Emission_1( EDF() )
),
Surface_4(
BSDFs::Empty(),
Emission_3( EDF() )
),
Volume_5( VDF() ),
Geometry_6()
);
monteCarloEyeClosestHitDispatcher<TheMaterial>(mat);
}
Material parameters
42
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
43
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
44
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
45
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
46
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
47
DEMO: RASTERIZER VS. RAY TRACER
RAY TRACER RASTERIZER
Shadows
Reflections
Walnut
Silver
Car paint
48
CONCLUSION
Support MDL materials in your application
Share MDL materials across applications
Maintain material appearance across applications
“a single unified material library that can be used in multiple applications”
April 4-7, 2016 | Silicon Valley
www.esi-group.com
THANK YOU