gdc 2008 post processing in the orange box
DESCRIPTION
TRANSCRIPT
![Page 1: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/1.jpg)
![Page 2: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/2.jpg)
OutlineOutline
• sRGB – DX9, DX10, XBox 360
• Tone Mapping
• Motion Blur
![Page 3: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/3.jpg)
sRGB OutlinesRGB Outline
• sRGB & gamma review
• Alpha Blending: DX9 vs. DX10 & XBox 360
• sRGB curve: PC vs. XBox 360
![Page 4: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/4.jpg)
sRGB ReviewsRGB Review
Terminology:• Color textures are stored in “gamma space”• Want our pixel shader to run in “linear space”
![Page 5: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/5.jpg)
sRGB & Gamma ConversionssRGB & Gamma Conversions
sRGB Read:
sRGB Write:
HardwareGamma Curve:
![Page 6: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/6.jpg)
Alpha Blending w/ sRGB WritesAlpha Blending w/ sRGB Writes DX9 vs DX10 & XBox 360DX9 vs DX10 & XBox 360
![Page 7: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/7.jpg)
Alpha Blending w/ sRGB WritesAlpha Blending w/ sRGB Writes DX9 vs DX10 & XBox 360DX9 vs DX10 & XBox 360
Notice the thicker smoke, glows on the gravity gun, and health GUI
![Page 8: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/8.jpg)
Alpha Blending on DX10 HW and XBox 360Alpha Blending on DX10 HW and XBox 360
DX9 with sRGB writes
DX10 & XBox 360 with sRGB writes
![Page 9: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/9.jpg)
Important DetailsImportant Details
• DX10 hardware running on DX9 will blend with DX10’s behavior!
• This affects DX9 games that have already shipped!
![Page 10: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/10.jpg)
SolutionsSolutions
1. Detect DX10 behavior and simulate the sRGB write in shader code forcing gamma blending
2. Let your artists tweak your materials for the obvious cases
We chose #2, and the artists only modified 40 out of thousands of materials in HL2, Ep1, Ep2, TF2, and Portal
![Page 11: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/11.jpg)
Example From HalfExample From Half--Life 2Life 2
![Page 12: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/12.jpg)
sRGB Curve: PC vs. XBox 360sRGB Curve: PC vs. XBox 360
• PC hardware uses the actual sRGB curve• XBox 360 uses a piecewise linear approximation to
the sRGB curve
NOTE: The smooth curve representing the PC sRGB curve in these slides doesn’taccurately represent the actual sRGB curve that is linear in the low end.
![Page 13: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/13.jpg)
Different Gamma SpacesDifferent Gamma Spaces
• We stopped using the term “Gamma Space” and instead...
• “PC Gamma Space” – Official sRGB curve
• “360 Gamma Space” – Piecewise linear sRGB approximation found on the XBox 360
• “Linear Space”
![Page 14: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/14.jpg)
PC Gamma on PCPC Gamma on PC
(Using PC Gamma textures on the PC)
![Page 15: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/15.jpg)
Uncorrected 360 ResultsUncorrected 360 Results
(Using PC Gamma textures on the XBox 360)
![Page 16: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/16.jpg)
What Just Happened?What Just Happened?
On PC, linear in pixel shader:
On 360, nonlinear in pixel shader:
![Page 17: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/17.jpg)
SolutionsSolutions
1. XBox 360-only developers: Use a Photoshop color space plug-in
2. Simulate sRGB reads and writes in shader code (Performance!)
3. Convert color textures at tool time and use the hardware gamma curve
Orange Box uses #3. Let’s take a closer look...
![Page 18: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/18.jpg)
The Orange Box Solution for XBox 360The Orange Box Solution for XBox 360
• Want to use the hardware “sRGB” reads & writes• We can modify the input textures so that the 360’s
piecewise linear read gets us to linear space!
Preprocess:
Run-time:
• But, 360 gamma space looks wrong on a TV or monitor! Need to get back to PC gamma space...
![Page 19: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/19.jpg)
Using the Hardware Gamma CurveUsing the Hardware Gamma Curve
On 360:
HW Gamma:
We also use the hardware gamma curve to optionally correct for “blacker-than-black” colors displayed on a television and the deeper gamma of televisions.
![Page 20: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/20.jpg)
XBox 360 Lossy sRGB ReadXBox 360 Lossy sRGB Read
• In linear space, the lossy range is 0.0-0.14, so generally OK. This caused very few issues for us, but...
• Don’t use the hardware sRGB reads for post processing or feedback effects! Simulate the piecewise linear sRGB read in shader code...it’s only ~11 asm instructions.
![Page 21: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/21.jpg)
sRGB SummarysRGB Summary
• Alpha blending differences exist• We let the artists tweak around this
• XBox 360 has a different gamma space• Convert color textures PC -> 360 Gamma Space• Set hardware gamma ramp for end correction
• XBox 360 HW sRGB read is lossy at the dark end (in linear space, 0.0-0.14)
![Page 22: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/22.jpg)
Tone Mapping OutlineTone Mapping Outline
• Brief overview of Valve’s HDR rendering
• Building the histogram
• Computing the tonemap scalar
![Page 23: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/23.jpg)
Overview of ValveOverview of Valve’’s HDR Renderings HDR Rendering
• Lighting data and environment maps are stored in HDR linear space
• Every pixel shader scales the linear HDR value by our tonemap scalar (Back buffer is RGBA8888!)
• Incrementally build histogram each frame• Tonemap scalar is generated from the
current histogram each frame
• More details on the first 2 points: http://www.valvesoftware.com/publications.html
![Page 24: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/24.jpg)
Building the HistogramBuilding the Histogram
• Amortize the cost of building the histogram over 16 frames• Update one bucket per frame• Sample post-tonemapped frame
• Use an asynchronous occlusion query to count pixels in range
![Page 25: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/25.jpg)
Sampling Each Histogram BucketSampling Each Histogram Bucket
![Page 26: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/26.jpg)
Evaluating the HistogramEvaluating the Histogram
• Our first implementation was based on median luminance (Shipped in HL2: Episode One, Day of Defeat, Lost Coast)
(NOTE: All histograms are in linear space!)
• But, we ran into too many cases in The Orange Box that caused tonemapping to behave strangely.
![Page 27: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/27.jpg)
Dark Skies!Dark Skies!
This environment was tonemapping too dark
![Page 28: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/28.jpg)
Dark Skies!Dark Skies!
This is what we wanted
![Page 29: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/29.jpg)
Bright Pixels MatterBright Pixels Matter• Experiment: Don’t use median luminance (50%)• Use a different histogram threshold: Keep 5% of bright pixels
in top bins
![Page 30: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/30.jpg)
Results From Using 95% ThresholdResults From Using 95% Threshold
This worked great! Except for...
![Page 31: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/31.jpg)
Zombies on Fire!Zombies on Fire!
Bright pixels from the fire causedtone mapping to over darken the screen!
![Page 32: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/32.jpg)
Need a Secondary RuleNeed a Secondary Rule
•• Primary rulePrimary rule: Brightness threshold•• Secondary ruleSecondary rule: Use median luminance as a darkness barrier
(NOTE: All histograms are in linear space!)
![Page 33: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/33.jpg)
Zombies Fixed With Both RulesZombies Fixed With Both Rules
This worked! But we still had one issue...
![Page 34: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/34.jpg)
Oscillations From Blinking LightsOscillations From Blinking Lights
Blinking lights cause oscillations in the histogram thatcause unwanted oscillations in the final tonemap scalar!
![Page 35: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/35.jpg)
The The ““Sticky BinSticky Bin””• Make bin containing 95% target “sticky”
• This causes minor variations in light to have no effect until passing threshold
![Page 36: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/36.jpg)
““Sticky BinSticky Bin”” Fixes OscillationsFixes Oscillations
![Page 37: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/37.jpg)
Final Tonemapping HeuristicsFinal Tonemapping Heuristics
1. Bright pixel threshold
2. Median luminance (darkness barrier)
3. Sticky bin
![Page 38: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/38.jpg)
Motion BlurMotion Blur
(A section from the non-real-time Portal trailer 2006)
![Page 39: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/39.jpg)
Motion Blur GoalsMotion Blur Goals
• Isolated, self-sufficient system
• Shader models 2.0, 2.0b, 3.0
• No additional memory (system or video)
• Performance!
• I don’t want to spend more than one week
![Page 40: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/40.jpg)
Evaluating Types of Motion BlurEvaluating Types of Motion Blur
1. Camera rotations – Can be done in post2. Camera translations – Needs depth or
vector image for correct parallax3. Object translations – Needs vector image or
“fins”4. Object rotations & animation – Needs
vector image or “fins”
• We chose #1 with some of #2
![Page 41: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/41.jpg)
Motion Blur: Where in the Pipeline?Motion Blur: Where in the Pipeline?We don’t want to blur the weapon!
1. Render full scene2. Motion blur3. Render view model / weapon4. Render GUI
![Page 42: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/42.jpg)
Rendering Motion BlurRendering Motion Blur
![Page 43: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/43.jpg)
Camera Rotation: PitchCamera Rotation: Pitch
• Blur vector is just vertical
![Page 44: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/44.jpg)
Camera Rotation: YawCamera Rotation: Yaw
• Not as simple as pitch
• Need two separate solutions
• We roll when we turn left/right while looking down!
![Page 45: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/45.jpg)
Camera Rotation: Yaw (Part A)Camera Rotation: Yaw (Part A)
• Blur vector is horizontal• This fades in/out with pitch
![Page 46: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/46.jpg)
• Roll motion blur• This fades in/out with pitch
This approximation is very efficient to implement!
Camera Rotation: Yaw (Part B)Camera Rotation: Yaw (Part B)
![Page 47: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/47.jpg)
Portal Falling BlurPortal Falling Blur
• When falling and looking down generate forward motion vectors
![Page 48: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/48.jpg)
Generating the Final Blur VectorGenerating the Final Blur Vector
• Blur vectors computed per-pixel:• Pitch: Full screen vertical vector• Yaw: Full screen horizontal vector• Yaw: Roll vector• Falling: Inside/out vector
• Combine these individually weighted vectors
• Sample along the vector and average
![Page 49: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/49.jpg)
Special Case: Portal TransitionsSpecial Case: Portal Transitions
• Moving through portals caused a jolt
• Use last frame’s blur values when moving a far distance in a single frame
![Page 50: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/50.jpg)
Special Case: System HitchesSpecial Case: System Hitches
• Another process stole CPU cycles from the game and caused a hitch• “You’ve got mail!”• “Time to update your software!”
• System hitches can cause one very blurry frame
• Time lapse between frames greater than 0.1 seconds, we disable motion blur for that frame
![Page 51: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/51.jpg)
Special Case: HeadacheSpecial Case: Headache
• Variable frame rate and blur made people sick• Only an issue when frame rate is low with variable frame
rate (Does not apply to the 360 since we’re vsync’d!)• Motion blur vector is globally scaled down as frame rate
drops from 50-30 fps• Use minimal motion blur to achieve the effect. We only use
15% of full-frame shutter!• Limit blur to 4% of screen width
![Page 52: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/52.jpg)
Motion Blur SummaryMotion Blur Summary
• Isolated system
• Blur from camera rotation only
• Special case Portal falling blur
• Acceptable performance & no additional memory
• 90% of Orange Box customers
![Page 53: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/53.jpg)
SummarySummary
• sRGB – DX9, DX10, XBox 360
• Tone Mapping
• Motion Blur
• Additional details about our rendering: http://www.valvesoftware.com/publications.html
![Page 54: GDC 2008 Post Processing In The Orange Box](https://reader034.vdocument.in/reader034/viewer/2022051311/540b941b8d7f72f36a8b4860/html5/thumbnails/54.jpg)
Thanks!Thanks!
Alex Vlachos, ValveAlex Vlachos, Valve