calibrating lighting and materials in far cry 3
TRANSCRIPT
![Page 1: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/1.jpg)
![Page 3: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/3.jpg)
![Page 4: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/4.jpg)
Diffuse Albedo
![Page 5: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/5.jpg)
BRDF
Diffuse albedo:
![Page 6: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/6.jpg)
BRDF
Torrance-Sparrow microfacet BRDF:
![Page 7: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/7.jpg)
BRDF
Lambert diffuse BRDF:
![Page 8: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/8.jpg)
Motivation
Natural, saturated colours
Consistent across materials
Consistent under all lighting conditions
Diffuse lighting balanced with specular
![Page 9: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/9.jpg)
Diffuse Albedo vs Diffuse Albedo
![Page 10: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/10.jpg)
Diffuse Reflectance vs Specular Reflectance
Too Dark Correct Too Bright
![Page 11: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/11.jpg)
Capturing Diffuse Albedo
Remove lighting Remove camera curve
Albedo
![Page 12: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/12.jpg)
Original
![Page 13: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/13.jpg)
After Colour Correction
![Page 14: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/14.jpg)
Capturing Diffuse Albedo
Use the Macbeth ColorChecker™ as reference.
Made by X-Rite, http://www.xrite.com/
24 colour patches with known sRGB values.
![Page 15: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/15.jpg)
Capturing Diffuse Albedo
Photograph material alongside ColorChecker™.
Lighting must be consistent.
Use ColorChecker™ patches to find transform.
![Page 16: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/16.jpg)
Affine Transform [Malin11]
Pro: removes cross-talk between channels
Con: linear transform
![Page 17: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/17.jpg)
Polynomial Transform [Malin11]
Pro: accurately adjusts levels
Con: channel independent
![Page 18: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/18.jpg)
Linear Least Squares
Given X and Y, find A such that:
Linear least squares: best approximation is:
Solve using Gauss-Jordan elimination.
![Page 19: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/19.jpg)
Affine Transform
Let xj be our photographed ColorChecker™ patches.
Let yj be the target ColorChecker™ values.
![Page 20: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/20.jpg)
Polynomial Transform
Let xj be our photographed ColorChecker™ patches.
Let yj be the target ColorChecker™ values.
![Page 21: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/21.jpg)
Colour Correction Tool
Command line tool.
Launched by a Photoshop script.
Operates in xyY colour space.
Applies the following transforms
[Malin11]:
Affine Polynomial Affine
![Page 22: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/22.jpg)
![Page 23: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/23.jpg)
![Page 24: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/24.jpg)
Future Work
Improve tool:
ColorChecker™ detection
dcraw to obtain raw sensor data
Adobe Camera Model for lens
correction
Improve capturing:
Investigate polarising filters.
![Page 25: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/25.jpg)
Original
![Page 26: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/26.jpg)
After Lens and Colour Correction
![Page 27: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/27.jpg)
![Page 28: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/28.jpg)
Sky Colour
Hue and saturation given by two gradient textures:
Luminance given by CIE Sky Model.
Cloud layers blended on top.
Sun side Sun opposite side
![Page 29: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/29.jpg)
CIE Sky Model
Models the luminance distribution of the sky.
Based on angular distances:
φ - sky element to zenith.
θ - sky element to sun.
θz - sun to zenith.
θ
φθz
sun
sky element
zenith
![Page 30: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/30.jpg)
CIE Sky Model
Luminance is given relative to that of the zenith:
Five coefficients to give different sky models.
φ - sky element to zenith
θ - sky element to sun
θz - sun to zenithθ
φθz
![Page 31: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/31.jpg)
CIE Sky Model
Make relative to sun intensity not zenith intensity:
More consistent sky luminance over all times of day
φ - sky element to zenith
θ - sky element to sun
θz - sun to zenithθ
φθz
![Page 32: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/32.jpg)
CIE Sky Model
Chose custom coefficients:
a b c d e
CIE clear sky model -1.0 -0.32 -10.0 -3.0 0.45
Far Cry 3 sky model -1.0 -0.08 -24.0 -3.0 0.30
![Page 33: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/33.jpg)
Sky Lighting
Use sky dome as hemispherical light.
Create SH coefficients from gradients and CIE model.
Sky lighting and sky will always match.
Variations in sky lighting.
![Page 34: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/34.jpg)
Original Light Probe
Midday
![Page 35: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/35.jpg)
Final Light Probes
Dawn Morning Afternoon Dusk
![Page 36: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/36.jpg)
Before
![Page 37: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/37.jpg)
After
![Page 38: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/38.jpg)
Problems
Loss of control:
Concept art had different ambient and sky colours.
Saturation:
Sky lighting too saturated.
Sky dome not saturated enough.
![Page 39: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/39.jpg)
Polarising Filter
PiccoloNamek at the English language Wikipedia
Without a polarising filter With a polarising filter
![Page 40: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/40.jpg)
Polarising Filter
Apply a fake polarising filter in the sky dome shader:
Also reduces direct sky reflection:
Applied to the sky in captured cube maps.
skyColor.rgb *= g_SkyPostProcess.rgb
![Page 41: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/41.jpg)
![Page 42: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/42.jpg)
Microfacet BRDF
Torrance-Sparrow microfacet BRDF: [Lazarov11]
![Page 43: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/43.jpg)
Distribution Function
Use normalised Blinn-Phong:
Specular power m in range [1, 8192]
Encoded as glossiness g in [0, 1] where:
![Page 44: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/44.jpg)
Fresnel Term
Schlick’s approximation for Fresnel:
Spherical gaussian approximation: [Lagarde12]
![Page 45: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/45.jpg)
Visibility Term
Call the remaining terms the visibility term:
Many games have V(l, v, h) = 1 making specular too
dark. [Hoffman10] [Lazarov11]
![Page 46: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/46.jpg)
Visibility Term
Schlick-Smith:
Roughness dependent.
Calculate a from Beckmann roughness k or Blinn-
Phong specular power m: [Lazarov11]
![Page 47: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/47.jpg)
Approximating Schlick-Smith
Cost of Schlick-Smith is 8 instructions per
light. [Lazarov2011]
Too expensive for our budget:
Especially when applied to every light.
Approximate by integrating Schlick-Smith
over the hemisphere.
![Page 48: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/48.jpg)
Approximating Schlick-Smith
![Page 49: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/49.jpg)
Approximating Schlick-Smith
Still 8 instructions!
Dependent only on glossiness…
…just like our existing energy conservation term.
Find a cheap curve to approximate both:
![Page 50: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/50.jpg)
Approximating Schlick-Smith
![Page 51: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/51.jpg)
Approximating Schlick-Smith
![Page 52: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/52.jpg)
Approximating Schlick-Smith
Chose the following approximation:
Simple (and free) modification of initial energy
conservation term.
Preferred to underestimate:
Schlick-Smith known to over brighten.
Makes artifacts less visible.
![Page 53: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/53.jpg)
Needed specular filtering to:
Reduce shimmering.
Preserve appearance in the distance.
Specular Filtering
With FilteringWithout Filtering
![Page 54: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/54.jpg)
Needed specular filtering to:
Reduce shimmering.
Preserve appearance in the distance.
Specular Filtering
With FilteringWithout Filtering
![Page 55: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/55.jpg)
Specular Filtering
Use Toksvig’s formula to scale the specular power,
using the length of the filtered normal: [Hill11]
Store these scales in a texture.
Ideally use to adjust gloss map.
![Page 56: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/56.jpg)
Specular Filtering
Cost of adding an extra texture too high.
Gloss maps not used in every shader.
Could not combine Toksvig map with existing gloss map.
Free channels in our DXT5 compressed normal maps:
0xFF Y 0x00 X
![Page 57: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/57.jpg)
Specular Filtering
Artists paint gloss in alpha
channel of normal map.
Store gloss combined with
Toksvig in red channel:
Gloss Y 0x00 X
![Page 58: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/58.jpg)
Toksvig Maps
Without gloss With gloss
Compression of normal y component affected:
![Page 59: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/59.jpg)
Toksvig Maps
Without gloss With gloss
Compression of normal y component affected:
![Page 60: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/60.jpg)
Toksvig Maps
Gloss map is optional.
With:
Combine with Toksvig.
Without:
Average Toksvig to single value.
Gloss +
Toksvig
Averaged
Toksvig
![Page 61: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/61.jpg)
Toksvig Off
![Page 62: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/62.jpg)
Toksvig On
![Page 63: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/63.jpg)
Conclusions
Calibrate your materials!
Physically-based is a great starting point:
Tweaking is needed.
Not everything is simulated.
Good tools are essential.
Can make the change late in a project.
Possible on current-generation consoles.
![Page 64: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/64.jpg)
References
[Darula02] Stanislav Darula and Richard Kittler, CIE General Sky Standard Defining
Luminance Distributions, eSim 2002, http://mathinfo.univ-reims.fr/IMG/pdf/other2.pdf
[Hill11] Stephen Hill, Specular Showdown in the Wild West,
http://blog.selfshadow.com/2011/07/22/specular-showdown/
[Hoffman10] Naty Hoffman, Crafting Physically Motivated Shading Models for Game
Development, SIGGRAPH 2010, http://renderwonk.com/publications/s2010-shading-
course/
[Lagarde12] Sébastien Lagarde, Spherical Gaussian Approximation for Blinn-Phong,
Phong and Fresnel, 2012, http://http://seblagarde.wordpress.com/2012/06/03/spherical-
gaussien-approximation-for-blinn-phong-phong-and-fresnel/
[Lazarov11] Dimitar Lazarov, Physically Based Lighting in Call of Duty: Black Ops,
SIGGRAPH 2011, http://advances.realtimerendering.com/s2011/index.html
[Malin11], Paul Malin, personal communication
![Page 65: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/65.jpg)
Thanks
Naty Hoffman, 2K Games (@renderwonk)
Paul Malin, Activision Central Tech (@p_malin)
Stephen Hill, Ubisoft Montréal (@self_shadow)
Philippe Gagnon, Ubisoft Montréal
Mickael Gilabert, Ubisoft Montréal (@mickaelgilabert)
Vincent Jean, Ubisoft Montréal
Minjie Wu, Ubisoft Montréal
Derek Nowrouzezahrai, Université de Montréal
![Page 66: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/66.jpg)
Any Questions?
@stevemcauley
http://blog.stevemcauley.com/
![Page 67: Calibrating Lighting and Materials in Far Cry 3](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6eed717f8b9a70728b6cdb/html5/thumbnails/67.jpg)