virtual and augmented reality 2019 miect, mieet, mei...
TRANSCRIPT
Virtual and Augmented Reality – 2019
MIECT, MIEET, MEI
Introduction to course
Universidade de Aveiro
Departamento de Electrónica,
Telecomunicações e Informática
Realidade Virtual e Aumentada Beatriz Sousa Santos, 2019/2020
Virtual and Augmented (Mixed) Reality
• Has been expanding from a research field into a commercially viable
technology.
• is highly interdisciplinary, combining work in diverse disciplines:
- more technology oriented research:
computer graphics, display technology, computer vision,
sensors, signal processing and computer networking.
and
- human centered research:
interaction techniques, design, human factors,
wearable computing and mobile computing.
• The growing interest in VAR applications is creating new challenges
for research in all of these areas
2
What is Virtual Reality?
“For better or worse, the label virtual reality stuck to this particular branch of
computer graphics.
I define a virtual reality experience as any in which the user is effectively
immersed in a responsive virtual world. This implies user dynamic control of
viewpoint.”
“A high-end user-computer interface that involves real-time simulation and
interaction through multiple sensorial channels (vision, sound, touch, smell,
taste)”.
(Fred Brooks, 1999)
3
(Burdea et al., 2004)
“The computer-generated simulation of a three-dimensional image or
environment that can be interacted with in a seemingly real or physical
way by a person using special electronic equipment, such as a helmet
with a screen inside or gloves fitted with sensors.” (Lexico, Oxford)
https://www.youtube.com/watch?v=izaBq084b0k
Early Virtual Reality – example
Division - 1991
4
Virtual Reality in practice for years – Automotive Industry
SEAT: VR in Car Production
https://www.youtube.com/watch?v=JAIn_xZ1-dY
5
Virtual Reality in practice for years – examples
McLaren uses Virtual Reality to design its sportscars and supercars
https://www.youtube.com/watch?v=mWaQfjEJIMQ
6
Toyota VR Experience: Reliability
http://vrworld.com/2016/03/14/toyota-creates-scary-vr-experience/
7
Virtual Reality in practice for years – Automotive Industry
Pain management
https://health.usnews.com/health-
care/patient-advice/articles/virtual-
reality-in-medicine
Medical training
https://www.youtube.com/watch?v=
PzPKdlFbvYM
Phobia therapy
https://www.youtube.com/watch?v=
GMttQHMjbJo
8
Virtual Reality in practice
– examples in Medicine
Entertainment
http://www.oculusvr.com/
Oculus Rift
2014; ~300 USD
9
http://www.youtube.com/watch?v=N8uuDT5AYts
“… systems that have the following three characteristics:
1) Combine real and virtual
2) Interactive in real time
3) Registered in 3-D” (Azuma, 1997)
“AR is a novel human machine interaction tool that overlays computer-
generated information on the real world environment. The information display
and image overlay are context-sensitive, which means that they depend on the
observed objects.” (Ong et al., 2008)
“A technology that superimposes a computer-generated image on a user's view
of the real world, thus providing a composite view.” (Lexico, Oxford)
10
What is Augmented Reality?
Note: this is a broader definition;
we will adopt Azuma’s definition.
http://www.youtube.com/watch?v=8UxWkZtUKaI (2009)
Early Augmented Reality – examples
Lego - 2009
11
Ray Ban Virtual Mirror - 2009
12
http://www.youtube.com/watch?v=Ag7H4YScqZs
http://www.ray-
ban.com/usa/virtual-mirror
13
Augmented Reality got known to the public
Pokemon Go - 2016
How Augmented Reality Is Driving Today's Automotive Industry
14
https://www.youtube.com/watch?v=VEwIZ7zwA0w
Augmented Reality – recent examples
http://www.techradar.com/reviews/wearables/microsoft-hololens-1281834/review
The Future of AR?
Hololens
2016; ~3000 USD
https://www.microsoft.com/microsoft-hololens/en-us
R. Furlan,”The future of augmented reality: Hololens - Microsoft's AR headset
shines despite rough edges”, IEEE Spectrum, 2016, 53 (6), p. 21
Mixed Reality: “Continuum”
“Augmenting natural feedback to the operator with simulated cues”
16
(Milgram et al., 1994)
Mixed Reality (MR)
Real Envinoment Augmented Reality Augmented Virtuality Virtual Reality
(Steinicke et a., 2009) http://www.youtube.com/watch?feature=endscreen&v=UgQfo7eNFdw&NR=1
• P. Dias, R. Silva, P. Amorim, J. Lains, E. Roque, I. Serôdio, F. Pereira, Beatriz
Sousa Santos, “Using Virtual Reality to Increase Motivation in Poststroke
Rehabilitation”, IEEE Computer Graphics and Applications, vol.39, no.1, pp. 65-70,
2019
• Bernardo Marques, R. Carvalho, P. Dias, Beatriz Sousa Santos, “Pervasive
Augmented Reality for Indoor Uninterrupted Experiences: a User Study”,
UbiComp/ISWC'19, London, September 2019
• A. Andreikanich, Beatriz Sousa Santos, P. Amorim, H. Zagalo, Bernardo Marques, P.
Margalho, J. Laíns, F. Faim, M. Coelho, Teresa Cardoso, Paulo Dias, “An Exploratory
Study on the use of Virtual Reality in Balance Rehabilitation“, 41th IEEE Engineering
in Medicine and Biology Conference, Berlin, July 2019
• Bernardo Marques, Beatriz Sousa Santos, T. Araújo, N. Martins, J. Alves, P. Dias,
“Situated visualization in the decision process through augmented reality”, 23rd
International Conference Information Visualisation 2019 - IV 2019, p. 13-18, Paris,
July 2019
• Bernardo Marques, R. Esteves, J. Alves, C. Ferreira, P. Dias, Beatriz Sousa Santos,
“Investigating different Augmented Reality Approaches in Circuit Assembly: a User
Study”, Eurographics Short Papers 2019, p. 45-48, May 2019
• J. Alves, Bernardo Marques M.Oliveira, M., T. Araújo, P. Dias, Beatriz Sousa Santos,
“Comparing Spatial and Mobile Augmented Reality for Guiding Assembling
Procedures with Task Validation. 2019 IEEE International Conference on
Autonomous Robot Systems and Competitions (ICARSC), April, 2019
19
Recent publications
Topics
• Introduction to Virtual Reality (VR) Augmented (AR) and Mixed Reality (MR)
• Definition, historical perspective, evolution, and applications
• Architectures and Frameworks for MR
• Input and output devices, tracking
• 3D user interfaces and interaction techniques
• Human factors in MR
• Human-Centered Design for MR
• Evaluation of MR applications
• Traditional and emerging applications
http://sweet.ua.pt/bss/disciplinas/RVA/RVA-home.htm
20
Lectures and Lab classes (tentative)
1 (Sep/17) - Introduction to the course, syllabus, assessment, bibliography
Paper presentation guidelines
2 (Sep/24) - Work on Virtual, Augmented in DETI/IEETA
3 (Oct/1) - Introduction to VR and AR (cont.) / Paper presentation
VR and AR Frameworks; Introduction to Unity
4 (Oct/8) - Input Devices and trackers / Paper presentation
Unity
5 (Oct/15) – Output Devices (visual) - Stereoscopy / Paper presentation
Toolkits for AR (Vuforia/ Unity)
6 (Oct/21) - Output Devices (other) - Paper presentation
Practical assignment
21
7 (Oct/29) – Human Factors in MR - Paper presentation
Toolkits for AR (Vuforia/ Unity)
8 (Nov/5) - Human Centered Design for MR – Paper presentation
Practical assignment
9 (Nov/12) - Interaction in MR– Paper presentation
Practical assignment
10 (Nov/19) – Hardware Demo – Paper presentation
Practical assignment follow-up presentation
10 (Nov/26) - Interaction in MR (cont.) - Paper presentation
Practical assignment
12 (Dec/3) – Evaluation methods – Paper presentation
Practical assignment
13 (Dec/10) – Emerging applications of MR - Paper presentation
14 (Dec/17) – Presentation and demo of practical assignments
22
Lectures and Lab classes
• ~2h – lecture + paper presentation and discussion
• ~1h – lab
• 2 sessions devoted to presentation and demo of the assignments
23
Main bibliography
• Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and
Morgan & Claypool, 2016
• LaValle, S., Virtual Reality - Virtual Reality. Cambridge University Press, 2017
(http://vr.cs.uiuc.edu/)
• LaViola, J., Kruijff, E., McMaha, R., Bowman, D, Poupyrev, I. J., 3D User
Interfaces: Theory and Practice, 2nd ed., Addison Wesley, 2017
• Schmalstieg, D., Hollerer, T., Augmented Reality: Principles and Practice (Usability).
Addison-Wesley Professional, 2016
• Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications:
Foundations of Effective Design, Morgan Kaufmann, 2009
• Carmigniani, J., & Furht, B, (eds) , Handbook of Augmented Reality, Springer, 2011
• Burdea, G., P. Coiffet, Virtual Reality Technology, 2nd ed., Wiley, 2003
• Journal and Conference Papers
24
- Exam – 40%
- Practical assignment – 45%
- Paper presentation and discussion – 15%
• Working students must contact us until September 30
Assessment
25
Practical assignment (general characteristics)
• Groups of two students
• Mini-project
• Development of a simple MR application using a human-centered
approach, specific devices and libraries
• Existing code may be used, provided that it is explicitly identified
and its source adequately cited
26
Our devices
27
Analyze and present a conference/ journal paper
a book section or a MR framework/tool:
1 student – short paper (~4 pages)
2 students– long paper (8+ pages) or book section Each student must:
- Decide if s/he will present alone or with a colleague
- select two papers,
a book section
send preferences to ([email protected])
propose another paper until September, 30
or a framework
- Read the paper presentation guidelines (presentation guidelines)
- Make a 15 mim (1 student) or a 30 min (2 students) presentation
- Send the slides to ([email protected])
28
Papers to read, present and discuss
http://ismar.vgtc.org/
29
http://ieeexplore.ieee.or
g/Xplore/
http://dl.acm.org/
http://www.springer.com
/computer/image+proce
ssing/journal/10055
And other
journals or
conferences
Discuss papers
• After each paper presentation:
- All students vote on their colleagues presentation
(through googleforms)
- Two students will discuss the paper
30
https://docs.google.com/forms/d/1rXAb2LlJTaj74-
xWMhhYMwWcNoUlzBUiyY9gZp2OFT0/viewform
• Students’ profile/background?
• Questions?
32