teleimmersion
DESCRIPTION
Tele-Immersion is a new technology that enables a user to share a virtual space with remote participants.TRANSCRIPT
![Page 1: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/1.jpg)
Internet-Wide Multi-Party Tele-Immersion Framework
for
Remote 3D Collaboration
RINU MARKS.PREG [email protected]
![Page 2: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/2.jpg)
Tele-Immersion is a new technology that enables a user to share a virtual space with remote participants.
WHAT IS TELE-IMMERSION ???
![Page 3: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/3.jpg)
Existing System
• Video conferencing.• Fails to keep eye to eye contact.• User interaction is limitted.• When no. of persons conferencing
increases it will be hard to concentrate on person who is actually talking.
![Page 4: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/4.jpg)
site 3site 1
site 2
BASIC CONCEPT
![Page 5: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/5.jpg)
• Not merely the 3D reproduction of the human body.
• It also allows us to interact with virtual objects in the shared virtual environment
![Page 6: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/6.jpg)
Interaction with virtual objects
![Page 7: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/7.jpg)
• Tele-immersion reconstructs the objects in the real world in real-time, and mixes the real objects with the virtual environment
![Page 8: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/8.jpg)
Remote Controller Function(wii)
![Page 9: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/9.jpg)
• Various operations on 3d virtual objects such as “move”, “rotate”, “select”, “change texture”, “expand”, “shrink”, “highlight”
• Helps in eye to eye communication in multi party enironment.
![Page 10: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/10.jpg)
ARRANGEMENTS
![Page 11: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/11.jpg)
ARRANGEMENTS
•B&W cameras are for stereo construction.•The color camera is for extracting the appearance
![Page 12: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/12.jpg)
ARRANGEMENT
![Page 13: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/13.jpg)
3D IMAGE CONSTRUCTION
![Page 14: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/14.jpg)
3D image construction
![Page 15: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/15.jpg)
Stages in 3D conversion
• Acquisition stage• Reconstruction stage
![Page 16: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/16.jpg)
Acquisition stage
• capture a variable amount of camera feeds .
• image-based calculations and color silhouette extraction.
• Image compression.
![Page 17: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/17.jpg)
Sterio based reconstruction stage
![Page 18: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/18.jpg)
Sterio based reconstruction1. Calculate initial matching cost
• and are the color intensities in the left and right images
• d is the disparity hypothesis.• denotes pixel in the left image
and denotes pixel in the right image
![Page 19: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/19.jpg)
2. Construct a square window of predefined size with p as centre.
3. For each pixel q in window calculate adaptive support weight.
• and denotes the color and euclidean distance between p and q.
• are constants.
![Page 20: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/20.jpg)
4. Calculate aggregated cost of each support window.
5. By using optimization technique we calculate depth of discontinuous areas and reconstruct the body.
![Page 21: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/21.jpg)
The user whole body is extracted by taking the difference between the current video frame and the registered background image.
BACK GROUND SUBTRACTION
![Page 22: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/22.jpg)
Real time compression
• Reconstructed 3d model is very large in size.
• Real-time compression is essential for real-time delivery.
• Modified jpeg compression.
![Page 23: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/23.jpg)
JPEG COMPRESSION
Each block is defined as M[x,y] (i -1) * d <=x < i * d , 0 <= i < w ( j -1) * d <= y < j * d , 0 <=j < h d is the size w width and h height.
![Page 24: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/24.jpg)
For each M[i , j]
• If all M[x, y] == 0 , then M '[i, j] = N .
• If all M[x, y] == 1, then M '[i, j] = A .
• If exists any M[x, y] == 0 and any M[x, y] == 1 , then M '[i, j] = B
![Page 25: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/25.jpg)
Multi-Party Tele-Immersion using internet
![Page 26: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/26.jpg)
Server side
• The TI server provides a virtual environment and room model to divide the users into groups.
• The system information such as user login/logout and their manipulations are written in database.
• The portal provides user login and real-time status display through web browsers with the help of database
![Page 27: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/27.jpg)
Communication model
![Page 28: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/28.jpg)
Hardware Requirements
• 4-core Intel Xeon W3530 processor
• 6GB DDR3 RAM• 2*500GB 7200rpm SATA Hard disk• NVIDIA Geforce GTX 470 display
card
![Page 29: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/29.jpg)
Time delays
![Page 30: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/30.jpg)
Time cost of 3D reconstruction & encoding
![Page 31: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/31.jpg)
Time cost of decoding & rendering
![Page 32: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/32.jpg)
Remote interaction delay on 3D objects
![Page 33: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/33.jpg)
Bandwidth requirement
![Page 34: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/34.jpg)
Applications
• Remote medical diagnosis and therapy.
• Remote and distributed education.• Distributed artistic performance and
interaction.• Virtual travel and tourism.• Automobile Engineering
![Page 35: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/35.jpg)
Challenges
• Quality of transmission affected by high packet loss
• Very Expensive• High bandwidth requirement.• High speed internet is required
(min of 60 MBPS)
![Page 36: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/36.jpg)
CONCLUSION
• Still in its early stages of development.
• Challenges are need to be met.• 3d reconstruction technology
needs to be improved since the reproduced images are jittering.
• Tele immersion technology can surely lead to great developments in the field of communication.
![Page 37: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/37.jpg)
References
• Proposal of Multimodal Communications System Using Tele-Immersion Technology ; IEEE Conference
• Tele-Immersion Environment for Video Avatar Based CVE ; IEEE Conference
• Distance Learning in Tele-Immersion Environment; IEEE Conference
• http://tele-immersion.citris-uc.org• http://graphics.cs.brown.edu/research/telei
![Page 38: Teleimmersion](https://reader035.vdocument.in/reader035/viewer/2022062705/556ffa68d8b42a84618b49f1/html5/thumbnails/38.jpg)
THANK YOU