Download - The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)
![Page 1: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/1.jpg)
The Visual Display Transform for Virtual Reality
Cyrus Moon
Computer Integrated Surgery II (600.446)
![Page 2: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/2.jpg)
Presentation Outline
• Synopsis of Current Project
• Concepts introduced in the Reading– VQS Representation– Coordinate System Graph– Object-to-Screen Transform
• Relevancy of concepts to the current project
![Page 3: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/3.jpg)
Current Project: Image/Video Overlay
Image Overlay:
-the merging of relevant computer generated information with the user’s actual view of the real world
Video Overlay:
-the merging of relevant computer generated information with display output from a video source
![Page 4: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/4.jpg)
Relevant Concepts
• Registration
• Tracking in 3D real space
• 3D modeling/rendering
• Frame transformations
![Page 5: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/5.jpg)
Background Reading
Robinett Warren, Halloway Richard. The Visual Display
Transformation for Virtual Reality. Technical Report
TR94-031 (1994), Dept. of Computer Science,
University of North Carolina at Chapel Hill.
![Page 6: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/6.jpg)
VQS - Vector Quaternion Scalar Representation
(for frame transformations)
• Vector(v) (3 terms) displacement
• Quaternion(q) (4 terms) rotation
• Scalar(s) (1 term) uniform scaling
[ (vx, vy, vz), (qx, qy, qz, qw), s ]
![Page 7: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/7.jpg)
Advantages of VQS (over the Euler 4x4 homogeneous matrix)
• Translation, rotation, and scaling components are separated
• Renormalizing the rotation is simpler (since rotation and scaling are separate)
• Uniform scaling in the virtual world a useful operation
• Quaternions a better method of manipulating 3D rotations than Euler rotations
![Page 8: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/8.jpg)
Advantages of the Quaternion (over the Euler 3x3 Rotation Matrix)
• Fewer components (4 instead of 9); fewer redundant parameters
• More elegant, numerically robust• Explicit representation of the angle and axis
of rotation• Allows easy interpolation between two
orientations
![Page 9: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/9.jpg)
The Quaternion
[(qx, qy, qz), qw]
• (qx, qy, qz) axis of rotation
• qw angle of rotation:
– qw takes values from –1 to 1
)(cos2 1wq
![Page 10: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/10.jpg)
Quaternion MathAddition:
Multiplication:
Multiplication by Scalar:
]),,,[(]),,,[(]),,,[( wwzzyyxxwzyxwzyx rqrqrqrqrrrrqqqqrq
]),,,[(*]),,,[(* wzyxwzyx rrrrqqqqrq
wwzzyyxx
zwwzxyyx
ywxzwyzx
xwyzzywx
rqrqrqrq
rqrqrqrq
rqrqrqrq
rqrqrqrq
,,
,
]*),*,*,*[(]),,,[(** wzyxwzyx qqqqqqqqq
![Page 11: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/11.jpg)
Quaternion Math (cont’d)
Taking the norm:
Normalization:
Inversion:
Interpolation:
2222),,,( wzyxwzyx qqqqqqqqq
qnormalize *1
)(
]),,,[(*1
]),,,[(2
11wzyxwzyx qqqq
qqqqqq
rqnormalizerqnterp **)1(),,(
![Page 12: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/12.jpg)
Using Quaternions
• Rotation of a vector p by quaternion q:pnew = q*p*q-1
-the vector p is treated as a quaternion w/ 0 as the
4th (scalar) term
-the result will always have a 4th term of 0, as well
![Page 13: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/13.jpg)
VQS Math
Conversion to 4x4 Matrix:
[v,q,s] = Mtranslate * Mrotate * Mscale =
1000
000
000
000
*
1000
02212222
02222122
02222221
*
1000
100
010
001
22
22
22
s
s
s
qqqqqqqqqq
qqqqqqqqqq
qqqqqqqqqq
v
v
v
yxxwzyywzx
xwzyzxzwyx
ywzxzwyxzy
z
y
x
![Page 14: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/14.jpg)
Using VQS Transforms
• Transformation of a vector (p) with VQS:p’ = [v, q, s]*p = s(q*p*q-1) + v
• Composition of two VQS transforms:TA_B*TB_C = [vA_B,qA_B,sA_B]*[vB_C,qB_C,sB_C]
= [(sA_B*(qA_B*vB_C*qA_B)-1) + vA_B, qA_B*qB_C, sA_B*sB_C]
• Inverse of a VQS Transform:TA_B
-1 = [vA_B, qA_B, sA_B]-1
= [1/sA_B*(qA_B-1*(-vA_B)*qA_B), qA_B
-1, 1/sA_B]
![Page 15: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/15.jpg)
The Coordinate System Graph
Robinett & Halloway
![Page 16: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/16.jpg)
The Coordinate System Graph
• Representation:– Each Node represents a coordinate system– Each line represents some kind of independent
transformation
• Properties:– Connected: each node is connected with every other node– Acyclic: there is only one pathway between any two given
nodes
![Page 17: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/17.jpg)
Transformations
• Independent: characterized by being independent variables within the software– Measured by tracker– Constant (rigid)
• Dependent: calculated from independent transforms
The Coordinate System Graph:-intuitively organizes all independent transforms and coordinate systems; easily expandable-allows easy calculation of any dependent transform present within the VR system
![Page 18: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/18.jpg)
The Object-to-Screen Transform
TS_O = TS_US * TUS_N * TN_E * TE_H * TH_HS * THS_TB * TTB_R * TR_W * TW_O
-TB_A is defined as the transformation from frame A to B
S = Screen HS = Head Sensor
US = Undistorted Screen TB = Tracker Base
N = Normalized R = Room
E = Eye W = World
H = Head O = Object
![Page 19: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/19.jpg)
• TW_O: World_Object – object in the virtual world.
– v - position– q - orientation– s - size
• TR_W: Room_World – the user position in the virtual world
– v - position– q - tilt of the world– s - user's size (shrinking or expanding of the world)
• TTB_R: TrackerBase_Room – position of tracker base (stored in calibration file)– s - must always be one (both cs’s in real-space)– Pre-calculated
VQS Transforms
![Page 20: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/20.jpg)
• THS_TB: HeadSensor_TrackerBase – inverse of the head position and orientation read from tracker– s = 1
• TTH_HS: Head_HeadSensor – position and orientation of the HMD sensor w/ respect to the head (center of the eyes)– s = 1– Pre-calculated
• TE_H: Eye_Head – position/orientation of head coordinate system
w/ respect to each eye– v – different for each user (though a default value can be used– q – dependent on orientation of HMD displays– s = 1– Pre-calculated
VQS Transforms (cont’d)
![Page 21: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/21.jpg)
TEye_Head
Robinett & Halloway
![Page 22: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/22.jpg)
• TN_E: Normalized_Eye – perspective projection, normalization
– Three to two dimensions (projection of world onto viewing plane)
• TUS_N: UndistortedScreen_Normalized– conversion to pixel coordinates– Simple scaling process
• TS_US: Screen_UndistortedScreen – correction of image
distortion
Non-VQS Transforms
![Page 23: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/23.jpg)
Applicable Concepts
• VQS Representation– Tracker Data Transform calculations– Registration Transforms (non-deformable objects)
• Elimination of the possibility of warping
• Coordinate System Graph
• Other concepts discussed in the individual transforms– Example: perspective transform
![Page 24: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/24.jpg)
Coordinate Graph (Video Overlay)
Patient markers
Model (from Imaging)
World (Tracker)
Tool 1 Tool k
. . . . . . . .
Camera
Lens
Screen
![Page 25: The Visual Display Transform for Virtual Reality Cyrus Moon Computer Integrated Surgery II (600.446)](https://reader036.vdocument.in/reader036/viewer/2022062716/56649dc55503460f94ab9221/html5/thumbnails/25.jpg)
Coordinate Graph (Image Overlay)
Patient markers
Model (from Imaging)
World (Tracker)
Tool 1 Tool k
. . . . . . . .
Silvered Glass
Head
Virtual Projection Plane
Screen