![Page 1: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/1.jpg)
The GPGPU Creation Story
Mikola Lysenko
University of Wisconsin-MadisonDept. of Mechanical Engineering
Spatial Automation Lab
![Page 2: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/2.jpg)
Overview
Going to cover major historical developments in roughly chronological order Time period: 1970 – present
Focus on: History
Key technologies
Market forces
Really a story about PC games
![Page 3: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/3.jpg)
The Early Graphics Displays
Vector graphics (1960s)
Based on the oscilloscope
Consist of electron gun + phosphor display
Examples: SketchPad
Asteroids
Pong
DutchTronix (2007)
![Page 4: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/4.jpg)
Problems with Vector Displays
No color Though you can put a filter over part of the screen
Limited number of objects
Crude images Will always look like lines
Atari, Inc. (1979)
![Page 5: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/5.jpg)
Raster Displays
Represent an image with a “framebuffer” A 2D array of pixels
Requires a RAMDAC: Random Access Memory Digital-to-Analog
Converter
Translates the framebuffer into a video signal
Can be used to display any image (in theory)
![Page 6: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/6.jpg)
Early Raster Games
Sprite based rendering (late 1980s-early 1990s) 'Blit' rectangular subregions quickly to animate
objects
Later games added scrolling backgrounds
Examples: Duke Nukem, Commander Keen
Apogee (1991)id Software (1990)
![Page 7: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/7.jpg)
Ray Casting Games
Enabled by raster displays and faster CPUs
Render a column of pixels per ray
Examples: Wolfenstein (1992) & Doom (1995)
id Software (1992)
id Software (1993)
![Page 8: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/8.jpg)
Problems with Ray Casting
3D is an illusion
Enemies/objects are sprites
World is limited to 2D geometry Though Doom had height
Apogee (1994)
![Page 9: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/9.jpg)
Polygons
Pioneered by SGI (1980s)
Basic idea is to represent 3D objects by a collection of polygons on their boundary
Made famous in games by Quake (1996)
Basis for all modern 3D graphics
id Software (1996)
![Page 10: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/10.jpg)
Polygon Rasterization
1. Trace edges of polygon into 2 buffers (LEFT, RIGHT)
LEFT RIGHT
![Page 11: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/11.jpg)
LEFT RIGHT
Polygon Rasterization (cont.)
2. Traverse LEFT-RIGHT scan lines in frame buffer
![Page 12: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/12.jpg)
Polygon Rasterization (final)
Fill in the scan lines
That's it!
![Page 13: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/13.jpg)
Problems
We need to draw a lot of polygons to get a good approximation of a 3D scene
Drawing polygons is expensive Quake was able to get around this with very
aggressive visibility culling
CPU performance was not scaling fast enough
![Page 14: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/14.jpg)
First 3D Hardware
Originally developed for Hollywood and industrial/academic applications First machines built by SGI (Iris) (late 1980s)
Brought to mass market by 3Dfx (1996)
id Software (1998)
![Page 15: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/15.jpg)
How the First GPUs Worked
Use PCI slot, hijacked video signal
Could not do 2D and 3D simultaneously! No windowed mode 3D
Only drew 1 triangle at a time
3Dfx Interactive, (1996)
![Page 16: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/16.jpg)
Further Evolution
Originally, PCI was fast enough The GPU could barely keep up!
But triangle rendering got faster
Needed more bandwidth to use this speed
Made new AGP slot ... Then AGP 2X, 4X, 8X ...
Then PCI-E, PCI-Ex16 ...
Asymptotically, this is a losing battle
![Page 17: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/17.jpg)
Hardware Caching
On board RAM was getting cheaper
Take some of the texture memory and use it to store vertices!
Then the CPU can draw many polygons with a single command, saving a lot of bandwidth
BUT... only works for static objects
What about animated characters?
![Page 18: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/18.jpg)
Vertex Programs
Allow programmers to parameterize animations
GPU runs a small user defined program for each vertex
Can generate an arbitrary animated shape from a few parameters and some cached memory
![Page 19: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/19.jpg)
How Vertex Programs Work
Vertices and topology are stored on the GPU
Each vertex is a collection of user-defined values called 'attributes' e.g. Position, Color, Texture Coordinates, etc.
May also pass 'uniform' values that are constant for all shaders e.g. Transform matrix, animation coordinates, etc.
Attributes + Topology + Uniforms + Vertex Program => Triangles!
![Page 20: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/20.jpg)
Example: Skinning(Skeletal Animation)
Parameterize animations for a character in terms of a collection of 'bones'
Vertices form a 'skin'
Attributes: Relative position
Bone weights
Color/Texture
Martinez, “Ontology for Virtual Humans” (2007)
![Page 21: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/21.jpg)
Skinning (cont.)
To draw a character, call the vertex program with bone positions passed as uniforms
Vertex program interpolates vertices
Bandwidth savings: Thousands of vertices, each with >30 bytes of data
vs.
A handful of bones (16 bytes each)
![Page 22: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/22.jpg)
Skinning Results
More characters, more polygons
Faster rendering
Flexible animation Can do ragdolls
Win-win situation
Croteam, (2001)
![Page 23: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/23.jpg)
Rendering Quality
Over time, people began to look for ways to add more detail to scenes, beyond simple texturing Alternate Lighting Models
Bump Mapping
Shadows
Environment Mapping
These techniques produced higher quality images, but required special GPU features
![Page 24: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/24.jpg)
Proliferation of Features
To keep up, hardware vendors kept adding more gizmos New updates almost every month
A programmer's nightmare!
Impossible to take advantage of all the new features Bad for coders
Bad for gamers
Bad for video card manufacturers
![Page 25: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/25.jpg)
Fragment Shaders
Make per-pixel color calculations programmable
Reduces interface complexity Implementing features no longer vendor's problem
Opened up brand new rendering techniques Far too many to even begin covering in this talk
Dramatic improvements in image quality
![Page 26: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/26.jpg)
Where does the term “Shader” come from?
Shaders were invented by Pixar for their Renderman platform (1982)
Meant to help artists specify lighting / textures
Used in Toy Story
Pixar, (1995)
![Page 27: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/27.jpg)
Fragment Shader Examples
Decaudin, “Cartoon-LookingRendering of 3D Scenes” (1996)
Gosselin, “Phat Lewt: Drawing a Diamond” (2004)
Heidrich & Seidel, “Realistic Hardware Accelerated Shading and Lighting” (1999)
Tatarchuk, “Practical Dynamic Parallax Occlusion Mapping” (2005)
D'Eon, “NVIDIA Human Head Demo” (2007)
Lengyel et al. “Realtime Fur Over Arbitrary Surfaces” (2001)
![Page 28: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/28.jpg)
GPU Shaders Today
Can create beautiful, lifelike imagery
Requires massive computing power Fast memory, many processors
Bethesda Softworks, (2007)
![Page 29: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/29.jpg)
Exponential Growth of GPU Power
To keep up with demands from gaming, GPUs grew very powerful
Source: linux-mag.com
![Page 30: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/30.jpg)
The Birth of GPGPU
GPUs had become very powerful Parallel execution for processing vertices/fragments
Massive memory bandwidth for texture lookups
Can it be used for something besides graphics? Yes!
Pixel Shaders + PBOs are Turing complete How does it work?
![Page 31: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/31.jpg)
The Ping-Pong Trick
Encode data in texture memory
Render full screen quad into PBO
Swap buffers
Repeat
GPU 'program' is nothing more than a perverted lighting calculation!
![Page 32: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/32.jpg)
The Animated Ping Pong
BUFFER 1
BUFFER 2
BUFFER 1
DRAW QUAD
DRAW QUAD
![Page 33: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/33.jpg)
Early GPGPU
Basic physics simulation
Linear algebra
Radiosity & Photon Mapping
Harris et al. (2002) “Physically-Based Visual Simulation on Graphics Hardware”
Purcell et al. (2003) “Photon Mapping on Programmable Graphics Hardware”
Coombe et al. (2004) “Radiosity on Graphics Hardware”
![Page 34: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/34.jpg)
Problems With Ping Ponging
Code is really obfuscated Need to split single algorithm across multiple
shaders and files
Performance is difficult to predict
Depends on rendering API and shader language Not all cards / systems support same features
Scatter is expensive
Limited atomic operations
![Page 35: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/35.jpg)
GPGPU Today: CUDA
Biggest change in GPGPU recently
Pros: Much easier to program, better for learning
Fast scatter / atomic operations
More control over hardware features
Cons: Only works on recent NVIDIA GPUs
Likely to change with hardware
![Page 36: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/36.jpg)
CUDA Examples
SVI Pro Seismic 3D (2008)
Stone et al. (2007) “Accelerating molecular modeling applications with graphics processors”
Nyland (2007) “Fast n-body simulation with CUDA”
Sijbers et al. (2008)
![Page 37: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/37.jpg)
Conclusion
Graphics have come a long way
GPUs have evolved into a powerful and versatile computing platform
GPGPU is now very big
Many emerging commercial and academic applications
![Page 38: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/38.jpg)
The Future
Will CPUs and GPUs merge? IMO: Highly unlikely
What is going to happen to CUDA?
Other GPU programming languages: Hou et al. (2008) “BSGP: Bulk Synchronous GPU
Programming”
Compute Shaders
OpenCL
????
![Page 39: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/39.jpg)
Acknowledgments
Dan Negrut
Vadim Shapiro
Roshan D'Souza
![Page 40: The GPGPU Creation Story Mikola Lysenko University of Wisconsin-Madison Dept. of Mechanical Engineering Spatial Automation Lab](https://reader035.vdocument.in/reader035/viewer/2022062517/56649edb5503460f94beb312/html5/thumbnails/40.jpg)
Thank you