cs 354 interaction
DESCRIPTION
March 29, 2012 lecture; CS 354 Computer Graphics; University of Texas at AustinTRANSCRIPT
CS 354Interaction &Project 2
Mark KilgardUniversity of TexasMarch 29, 2012
CS 354 2
Today’s material
In-class quiz On compression lecture
Lecture topic Project 2 Interaction
CS 354 3
My Office Hours
Tuesday, before class Painter (PAI) 5.35 8:45 a.m. to 9:15
Thursday, after class ACE 6.302 11:00 a.m. to 12
Randy’s office hours Monday & Wednesday 11 a.m. to 12:00 Painter (PAI) 5.33
CS 354 4
Last time, this time
Last lecture, we discussed Project 2 on Programmable Shaders Compression for Computer Graphics
This lecture Project 2 discussion
Get started now
Interaction
CS 354 5
Daily Quiz
1. True or False: A hardware texture compression scheme must be capable of random access decompression.
2. True or False: Floating point values are required to have at least 23 bits devoted to the mantissa and 8 bits for the exponent.
3. True or False: Run-length encoding works well for digital photographs.
4. If the surface normals in a mesh are each stored as an index into a set of 256 quantized normals rather than as three 32-bit floating point numbers, what compression ratio is achieved over the floating-point representation?
On a sheet of paper• Write your EID, name, and date• Write #1, #2, #3, #4 followed by its answer
CS 354 6
Project 2
Project is NOW available PDF + code to use
Requirements Mostly writing GLSL shaders Small amount of C/C++ code
Convert height map to normal map Implement NormalMap::computeNormal method
Intermediate milestone Monday, April 2nd
Submit snapshots and GLSL code for first two shaders Complete project
Due Friday, April 6th All 9 shaders Embellish for additional credit
CS 354 7
By April 2
Have these first two shaders tasks implemented
Task 0: roll torus
Task 1: apply decal
CS 354 8
Procedurally Generating aTorus from a 2D Grid
2D grid over (s,t)[0,1]
Tessellated torus
CS 354 9
GLSL Standard Library Routines You’ll Need for Project 2
texture2D—accesses a 2D texture through a sampler2D and a 2-component texture coordinate set (s,t)
textureCube—access a cube map with a samplerCube and a 3-component texture coordinate set (s,t,r)
normalize—normalizes a vector cross—computes a cross product of 2 vectors dot—computes a dot (inner) product of 2 vectors max—compute the maximum of two values reflect—compute a reflection vector given an incident vector and a normal
vector vec3—constructor for 3-component vector from scalars mat3—constructor for 3x3 matrix from column vectors *—matrix-by-vector or vector-by-matrix multiplication sin—sine trigonometric function cos—cosine trigonometric function pow—raise a number to a power, exponentiation (hint: specular)
CS 354 10
Interaction withComputer Graphics
Graphics isn’t just about pretty pictures It’s also about interacting with pictures
SketchPad1963Ivan Sutherland
CS 354 11
Event Models
Polling “while loop” reads events & processes them Good for events arriving at a regular rate Appropriate when events aren’t “digital”
May require sampling analog signals
Event-driven Structured to avoid “busy waiting” Callback-driven programming
GLUT’s event loop is callback-driven
CS 354 12
Physical Events vs.Logical Events
Physical events Real-world input Examples
Key presses Physical link layer for
networking Charged-coupled
device (CCD) captures image
May require signal processing to make into a digital event
Prone to noise
Logical events Examples
Key press delivered to a particular window the mouse cusor is in
Byte stream from a network socket
Mouse clicked on an object in a 3D scene
Higher-level
CS 354 13
Routing Events
What entity is interested in an event? How is the event routed to the interested
entity? Example: window system events
Mouse events generated by mouse Button clicks and movement must go to the
correct window region Window regions arranged hierarchically
Two or more events may be connected The press & release of a mouse must be paired
CS 354 14
Model-View-Controller
Software architecture approach Separates concerns
Model
Data and logic
View
Interface / display
Controller
User input
updates modifies
user responds to view with input
modifies
CS 354 15
Interactivity
Interaction Latency Time from visual display to user action to
displayed response to user interaction Low latency provides illusion of continuous
interaction
Graphics device display frames Roughly 60 frames/second provides
continuous interaction Roughly 16.6 milliseconds Display devices tuned to this
CS 354 16
Frame Buffering
Problem with just one framebuffer Updating the buffer you are displaying results in flicker and
tearing One solution
Timing framebuffer updates to not be updating pixels about to be scanned out
Known as “racing the beam”
Solution: Two buffers One to display NOW One to render NOW so to display NEXT Ping-pong between the two buffers Alternatively, copy “draw” buffer to “display” buffer every frame Either approach needs synchornization
CS 354 17
Synchronized Buffer Swaps
Stall generate next frame until displaying the last frame
16.6 milliseconds
CS 354 18
Unsynchronized Buffers Swaps
More frame rendered and lower latency to display But tearing artifacts on displays And some frames rendered but never displayed
Wasted work16.6 milliseconds
tearing artifacts
CS 354 19
Triple Buffering Expensive Compromise
Needs more memory, now 3 buffers Renders frames that may never be displayed But avoids tearing while minimizing latency
16.6 milliseconds
CS 354 20
Complex Interactions
Dragging objects Cut-and-paste Pop-up and pull-down menus Selecting an object within a 3D scene
CS 354 21
State Machines to Managed Complex Interactions
Think of on-screen “button” You can click with the mouse How does it really logically work?
CS 354 22
Manipulators and Choosers
Open Inventor
CS 354 23
Intuitive Interactions
Make computer behave in a way that’s natural to humans NOT the other way around
Good rules of thumb Hint what the result of an interaction will be
Locate highlight buttons Change cursor shape based on its position
Provide immediate and continuous display of state Show mercy, people make mistakes, be able to undo Make manipulation as direct as possible
Examples Dragging and dropping objects Pointing to indicate interest
CS 354 24
Back Projection
User clicks on screen… Question: What object got “clicked”?
Known as “picking” in 3D applications Basic operation in 3D applications
Process Window system maps click to (x,y) in a particular window
Delivers the event to application selecting events on the window Applications gets (x,y) in window space
Walk through scene graph Invert viewport, projection, and modelview transforms
Back project point to object space ray Intersect the ray with each object in scene graph
Does the ray intersect the object? What intersected object is the closest?
Return the nearest object intersecting the ray as a “hit”
CS 354 25
Conceptual Vertex Transformation
glVertex*API
commands
Modelviewmatrix
User-definedclip planes
View-frustumclip planes
to primitiverasterization
object-space coordinates
(xo,yo,zo,wo) eye-space coordinates
(xe,ye,ze,we)
clipped eye-space coordinates
clipped clip-space coordinates Perspective
divisionProjection
matrix
Viewport + Depth Rangetransformation
(xc,yc,zc,wc)
window-spacecoordinates
(xw,yw,zw,1/wc)
normalized device coordinates (NDC)
(xn,yn,zn,1/wc)
clip-spacecoordinates
(xc,yc,zc,wc)
(xe,ye,ze,we)
(xe,ye,ze,we)
CS 354 26
Reversed Vertex Transformation
glVertex*API
commands
Modelviewmatrix
User-definedclip planes
View-frustumclip planes
to primitiverasterization
object-space coordinates
(xo,yo,zo,wo) eye-space coordinates
(xe,ye,ze,we)
clipped eye-space coordinates
clipped clip-space coordinates Perspective
divisionProjection
matrix
Viewport + Depth Rangetransformation
(xc,yc,zc,wc)
window-spacecoordinates
(xw,yw,zw,1/wc)
normalized device coordinates (NDC)
(xn,yn,zn,1/wc)
clip-spacecoordinates
(xc,yc,zc,wc)
(xe,ye,ze,we)
(xe,ye,ze,we)
(x,y) mouse event
CS 354 27
New CurrentInteraction Paradigms
Multi-touch screens Phones, tablets… work surfaces?
Perceptive Pixel
CS 354 28
Where next?
Visual, tactile, gestures, speech, highly responsive, emotionally aware Minority Report interface…
CS 354 29
Next Class
Next lecture Procedural methods Making shape, appearance, and animation from
programs—instead of data Project 2
Start learning GLSL and doing project Due Friday, April 6th
Incremental deadline on April 2nd
First two tasks