distributed graphics: two case studies jian huang, cs 594, spring 2002
TRANSCRIPT
![Page 1: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/1.jpg)
Distributed Graphics: Two Case Studies
Jian Huang, CS 594, Spring 2002
![Page 2: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/2.jpg)
Distributed Graphics
• Two major categories:– Data can be replicated– Data can not be replicated
• Dependent on the choice– Drastically different approach– Specific pros and cons
![Page 3: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/3.jpg)
This Lecture
• Look at a sample system in each category:– A Distributed 3D Graphics Library, Blair MacIntyre
and Steven Feiner, SIGGRAPH’2002
– Deep Compression for Streaming Texture Intensive Animations, Daniel Cohen-Or, Yair Mann, Shachar Fleishman, SIGGRAPH’1999
• Lots of research issues are currently still open problems
![Page 4: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/4.jpg)
Distributed Graphics• Systems for distributing the shared graphical state
of multi-display, multi-person, distributed, interactive applications
Computer Supported Corporative Work (CSCW) and Distributed Virtual Environments (DVE)
![Page 5: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/5.jpg)
High Level Libraries
• Have excellent programming libraries for stand-alone 3D applications: Inventor, Performer, Java3D, …
• No such libraries for distributed graphics applications, although some DVE systems have been proposed:– A mechanism to distribute application state (either a custom
solution or using a general-purpose lib)– Maintain the state of the graphical display separately and locally
• A “dual databases” approach– Synchronizing the dual database is complex, tedious, and error-
prone
![Page 6: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/6.jpg)
General Goal• To use a “single database” approach for distributed 3D
graphics library• Repo-3D is an object-oriented, high-level graphics package
– The objects used to create the graphical scenes are directly distributable from the programmer’s perspective
– The objects reside in one large distributed shared memory (DSM)
– The underlying system automatically replicates the fine-grained objects across as many processes as needed
– Updates to objects are automatically reflected in all replicas, with any required objects automatically distributed as needed.
![Page 7: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/7.jpg)
Different between Distributed and Monolithic Applications
• Distributed control– In a non-trivial distributed application, different
components need to be notified of changes to the distributed state
• Interactivity– Network latency bandwidth limitations make updates to
distributed state much slower. For performance, need to perform some operations locally
• Local variations– There are times when a shared graphical scene may need
to be modified locally
![Page 8: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/8.jpg)
Repo-3D• Repo-3D addresses these problems in two ways:
– Notification object: associated with any replicated object. Invoke notification object’s methods when the replicated object is updated.
– Local variation: to allow the properties of a graphical object to be modified locally, and parts of the scene graph to be locally added, removed, or replaced.
![Page 9: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/9.jpg)
Motivation• Authors’ fundamental belief:
– With uniform high-level support for distributed programming in the languages and toolkits, prototyping and experimenting with distributed interactive applications can be (almost) as simple as multi-threaded programming in a single process
• Useful for:– True with exploration and prototyping phase
– Object-oriented programming (high data transparency)
![Page 10: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/10.jpg)
Client-Server Data Sharing Inappropriate for Distributed Graphics
• Client-server data sharing (e.g. CORBA,
Modular-3 Network Objects, Java RMI) impossible for high frame-rate distributed graphics
• Data replication is necessary
• Update of shared/replicated data should be asynchronous
![Page 11: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/11.jpg)
Underlying Infrastructure• Developed in Modula-3 language: descendant of
PASCAL, and heavily influenced Java
• In particular, Modula-3:– strong type safety
– exception handling
– concurrency
– object-oriented programming
– automatic garbage collection
![Page 12: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/12.jpg)
Distributed Objects
• Within the paradigm of distributed shared memory (DSM)
• Three types of objects– Simple objects: normal local data obj
– Remote objects: client-server semantics. All method invocations go across the network to the original object
– Replicated objects: replicated at all locations, need to be consistently updated
![Page 13: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/13.jpg)
Remote Objects
• The Network Object package provides support for remote objects
• It implements – distributed garbage collection
– exception propagation back to the calling site
– automatic marshalling and unmarshalling of method arguments
– return values of virtually any data type between heterogeneous machine architectures
![Page 14: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/14.jpg)
Replicated Objects• The Replicated Object package supports replicated objects.
Each process can call any method of an object it shares
– All operations on an instance of an object are atomic and serializable
– All operations are performed in the same order on all copies of the object
– If two methods are invoked simultaneously, the order of invocation is nondeterministic
– Full replication: no partial replication
![Page 15: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/15.jpg)
Update Scheme for Replicated Objects
• Each method is marked either read or update• Only update methods trigger an update event• Use a sequencer process (maybe a remote process)
to enforce the same sequence of operations for a replicated object
• May have multiple sequencers in the distributed system
• All replicas of an object, including the one invoked this update method, executes the update after receiving update msg from sequencer
![Page 16: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/16.jpg)
Anim-3D
• Graphical objects (GO): – all entities of a graphical scene: geometry, light, …– GOs can be grouped in any Directed Acyclic Graph
(DAG)– Each GO is associated with a set of properties
• Properties: – behavior of GO– Defined by a name and a value:
• say, TextureReconstruction, nearest neighbor
• Input event call-backs
![Page 17: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/17.jpg)
GO• GO hierarchy:
• GO property of g, is inherited by all g’s descendants, unless overridden
• But, property is always attached to separate nodes• Change of a property won’t affect its siblings
![Page 18: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/18.jpg)
Property
• 4 behavior types for each property:– Constant– Synchronous– Asynchronous– Dependent
![Page 19: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/19.jpg)
Basic Design of Repo-3D
• The interface is programmatical and declarative– No graphics scene file format: all scenes exist
as a Repo-3D program– Scenes can be updated with a new Repo-3D
program as a replicated object
![Page 20: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/20.jpg)
Replicated Objects?
• GO: 21 GO types, all replicated, except RootGO, which corresponds to window to be rendered into
• Properties: most of the Modula-3 objects that comprise a property are not replicated– Properties with a constant binding between name and
value
– All names
– Behaviors and requests
• Input callbacks are not replicated
![Page 21: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/21.jpg)
Local Variations
• Needed to maintain interactivity and local modifications to shared scene graph
• Local state: state added to each object – only accessible to the local copies – not passed to remote processes when the object is copied to create a
new replica– Typically: additions, deletions, and replacements to the properties or
children
• New methods that manipulate the local variations are added to the GOs as non-update method
• Repo-3D combines both the global and local state when creating the graphical scene
![Page 22: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/22.jpg)
Local Variations - property
• Property variations. – 3 methods to set, unset, and get the global property list
attached to a GO– methods to manipulate local variations: – add or remove local properties (overriding the value
normally used for the object)– hide or reveal properties (causing the property value of
the parent node to be inherited)– flush the set of local variations (removing them in one
step) or atomically apply them to the global state of the object
![Page 23: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/23.jpg)
Local Variation - children
• Child variations– 5 methods to add, remove, replace, retrieve, and flush
the set of children contained in a group node
– Additional ones for local variation: • add a local node
• remove a global node locally
• replace a global node with some other node locally
• remove each of these local variations
• flush the local variations (remove them all in one step)
• atomically apply the local variations to the global state
![Page 24: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/24.jpg)
Example
![Page 25: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/25.jpg)
Streaming Texture Intensive Applications
• The problem faced by all network-based applications is bandwidth
• A network-based compression scheme is a two-part compressed sequence– header download– streaming
• MPEG takes advantage of frame-to-frame coherency only, still too big for streaming
• Need to reduce to 2KB/sec bandwidth
![Page 26: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/26.jpg)
Introduction
• Synthetic animations have higher potential to be compressed
• because we know more :– geometry– texture data– (temporal) viewing information
![Page 27: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/27.jpg)
Background
• An MPEG video sequence consists of intra-frames (I), predictive (P) and interpolated frames (B)
• The compression ratio is depends on the successful motion estimation
• For synthetic animation, model-based motion estimation could provide further improvement
![Page 28: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/28.jpg)
Background (2) : A different approach [Levoy]
• Render the animation on-the-fly at both ends of communication rather than compress post-rendered images
• Sever: render high and low quality images, compute the residual error between them, and transmit the residual image compressed.
• Client: render only the low quality images and to add the transmitted residual images.
![Page 29: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/29.jpg)
Background (3) [Levoy]
• Require pre-computing the residual images and downloading entire model and animation sequence before streaming
• Not clear how to treat texture intensive animations
• Other related techniques use imposters and sprites
![Page 30: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/30.jpg)
View-dependent Texture Streaming
•All you need: streaming the environment requires the server to transmit the animation script and visible parts of the model.
•No need for the whole texture
•It’s better to use nearby views as textures rather than using the original textures
![Page 31: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/31.jpg)
• Nearby views are advantageous as texture because:– they have an “almost” one-to one
correspondence with the current view and therefor
– they are post-rendered and may include various global illumination effects.
View-dependent Texture Streaming(2)
![Page 32: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/32.jpg)
View-dependent Texture Streaming(3)
• The reuse could be either forward or backward
• Occasionally need to request new texture from server from pre-computed future frames
• Always need to consider geometry streaming (visibility, gaps …)
![Page 33: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/33.jpg)
View-dependent Texture Streaming(4)
![Page 34: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/34.jpg)
The Texture Quality Factor
• Texture quality factor: a local scaling factor to estimate that locally shrink or expand when mapped onto target image.
• 2-Norm of matrix A
• sqrt(max Eigenvalue)
![Page 35: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/35.jpg)
The Texture Quality Factor (linear case)
![Page 36: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/36.jpg)
The Texture Quality Factor (non-linear case)
![Page 37: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/37.jpg)
The Texture Quality Factor (non-linear case)
![Page 38: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/38.jpg)
Geometry Streaming
• Geometry stream gets a higher priority than view-dependent texture stream
• Use stencil buffer to identify visibility information for each frame
![Page 39: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/39.jpg)
Bandwidth (compared with MPEG)
![Page 40: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/40.jpg)
Two Reference Views
![Page 41: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/41.jpg)
Three In-Between Views
![Page 42: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/42.jpg)
Results
![Page 43: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/43.jpg)
More results
![Page 44: Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002](https://reader035.vdocument.in/reader035/viewer/2022062322/56649ebc5503460f94bc537d/html5/thumbnails/44.jpg)
More results