Virtual and Augmented Reality – 2017

MIECT, MIEET, MEI

Introduction to course

Universidade de Aveiro

Departamento de Electrónica,

Telecomunicações e Informática

Realidade Virtual e Aumentada Beatriz Sousa Santos, 2017/2018

2

Virtual and Augmented (Mixed) Reality

• Has been expanding from a research field into a commercially viable

technology.

• is highly interdisciplinary, combining work in diverse disciplines:

- more technology oriented research:

computer graphics, display technology, computer vision,

sensors, signal processing and computer networking.

and

- human centered research:

interaction techniques, design, human factors,

wearable computing and mobile computing.

• The growing interest in VAR applications is creating new challenges

for research in all of these areas

3

What is Virtual Reality?

“For better or worse, the label virtual reality stuck to this particular branch of computer graphics.

I define a virtual reality experience as any in which the user is effectively immersed in a responsive virtual world. This implies user dynamic control of viewpoint.”

“A high-end user-computer interface that involves real-time simulation and interaction through multiple sensorial channels (vision, sound, touch, smell, taste)”.

(Fred Brooks, 1999)

4

(Burdea et al., 2004)

Virtual Reality in practice for years – examples

Industry

Jaguar / Land Rover

Ford

http://www.youtube.com/watch?v=j3qcnvgVlNk

https://www.youtube.com/watch?v=BEFfp2QhHZ

U

5

http://www.youtube.com/watch?v=CMU0HtmBt48

6

Medicine

http://www.youtube.com/

watch?v=zJmrcEM-uvA

7

Entertainment

http://www.oculusvr.com/

Oculus Rift

2014; ~300 USD

8

http://www.youtube.com/watch?v=N8uuDT5AYts

“AR is a novel human machine interaction tool that overlays computer-

generated information on the real world environment. The information display

and image overlay are context-sensitive, which means that they depend on the

observed objects.” (Ong et al., 2008)

10

What is Augmented Reality?

• According to Azuma Augmented Reality systems have the following

characteristics:

“1) Combine real and virtual

2) Interactive in real time

3) Registered in 3D”

11

http://www.youtube.com/watch?v=8UxWkZtUKaI

Early Augmented Reality – examples

Lego

12

Ray Ban Virtual Mirror

13

http://www.youtube.com/watch?v=Ag7H4YScqZs

http://www.ray-

ban.com/usa/virtual-mirror

Augmented Reality Sandbox

14

http://idav.ucdavis.edu/~okre

ylos/ResDev/SARndbox/

http://www.techradar.com/reviews/wearables/microsoft-hololens-1281834/review

The Future of AR?

Hololens

2016; ~3000 USD

https://www.microsoft.com/microsoft-hololens/en-us

R. Furlan,”The future of augmented reality: Hololens - Microsoft's AR headset

shines despite rough edges”, IEEE Spectrum, 2016, 53 (6), p. 21

Mixed Reality: “Continuum”

“Augmenting natural feedback to the operator with simulated cues”

16

(Milgram et al., 1994)

Mixed Reality (MR)

Real Envinoment Augmented Reality Augmented Virtuality Virtual Reality

(Steinicke et a., 2009) http://www.youtube.com/watch?feature=endscreen&v=UgQfo7eNFdw&NR=1

Team

Beatriz Sousa Santos; Paulo Dias

(bss@ua.pt; paulo.dias@ua.pt)

18

Topics

• Introduction to Virtual Reality (VR) and Augmented Reality (AR)

Definition, historical perspective and evolution, examples

• Modeling of Virtual Environment (VE)

• Input and output devices, tracking, cameras

• 3D user interfaces and interaction techniques

• Human factors in VR and AR

• Frameworks for VR and AR

• Traditional and emerging applications

http://sweet.ua.pt/bss

20

Lectures and Lab classes (tentative)

1 (15/09/17) - Introduction to the course

Frameworks para RV e RA / VR and AR Frameworks

2 (22/09/17) - Introduction to Virtual and Augmented Reality

Paper presentation Guidelines

Unity

3 (29/9/17) - Introduction to Virtual and Augmented Reality (cont.)

Unity

4 (6/10/17) - Input Devices / Paper presentation

Unity

5 (13/10/17) – Output Devices (visual) - Stereoscopy / Paper presentation

Unity - Practical assignment

6 (20/10/17) - Output Devices (other) - Paper presentation

Unity – Practical assignment

21

7 (27/10/17) - Interaction in VR and AR - Paper presentation

Unity or Introduction OpenCV

8 (3/11/17) - Cameras and world registration - Paper presentation

Camera calibration with OpenCV

9 (10/11/17) - Human Factors in VR and AR - Paper presentation

Toolkits for AR (ARToolkit and others)

10 (17/11/17) - VR and AR Hardware

Hardware Demo

11 (24/11/17) - Assignments: mid-term demo and presentation

12 (15/12/17) - Evaluation methods

Unity – Practical assignment

Assignments: demo and presentation (tbd)

22

Lectures and Lab classes

• ~1h30 min – lecture; including paper presentation and discussion

• ~1h 30 min – lab

• 2 sessions devoted to presentation and demo of the assignments

Unity, OpenCV, ARToolkit ....

23

Main bibliography

• Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and

Morgan & Claypool, 2016

• Schmalstieg, D., Hollerer, T., Augmented Reality: Principles and Practice (Usability).

Addison-Wesley Professional, 2016

• Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications:

Foundations of Effective Design, Morgan Kaufmann, 2009

• Burdea, G., P. Coiffet, Virtual Reality Technology, 2nd ed., Wiley, 2003

• Bowman, D., E. Kruijff, I. Poupyrev, J. LaViola, 3D User Interfaces: Theory

and Practice, Addison Wesley, 2004

• Anthes, C., M. Wiedemann, D. Kranzlmüller, “State of the Art of Virtual Reality

Technology”, IEEE Aerospace Conference, 2016.

• Journal and Conference Papers

24

Other bibliography

• Kim, G., Designing Virtual Reality Systems: The structured approach, Springer 2005

• Mullen, T., Prototyping Augmented Reality, Sybex, 2011

• Furht (ed.), Handbook of Augmented Reality, Springer, 2011

• Stanney, K.(ed.), Handbook of Virtual Environments, Design, Implementation and Applications, Lawrence Erlbaum Associates, 2002

• Journal and Conference Papers

25

- Exam – 40%

- Programming assignment – 20% + 30%

- Paper presentation and discussion – 10%

• Working students must decide on the type of assessment and inform us until September, 30

Assessment

26

Practical assignment (general characteristics)

• Groups of two students

• Mini-project

• Development of a simple application using specific devices and

libraries

• Existing code may be used, provided that it is explicitly identified

and its source adequately cited

28

This year’s possible “client”:

29 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-

doentes-com-avc-4486208.html

Our devices

30

Analyze and present a conference/ journal paper:

1 student – short paper (~4 pages)

2 students– long paper (8+ pages)

Each student must:

- Decide if he/she will present alone or with a colleague

- select two papers

or send preferences to (bss@ua.pt)

until September, 30

- Propose another paper

- Read the paper presentation guidelines (presentation guidelines)

- Make a 12 mim (short paper) or a 25 min (2 students - long paper) presentation

- Send the slides to (bss@ua.pt)

31

Papers to read, present and discuss

http://ismar.vgtc.org/

32

http://ieeexplore.ieee.org/Xplore/

http://dl.acm.org/

http://www.springer.com

/computer/image+proce

ssing/journal/10055

And other

journals or

conferences

Discuss papers

• After each paper presentation:

- All students vote on their colleagues presentation

(through googleforms)

- Two students will discuss the paper

33

https://docs.google.com/forms/d/1rXAb2LlJTaj74-

xWMhhYMwWcNoUlzBUiyY9gZp2OFT0/viewform

• Students’ profile?

• Questions?

35

Interesting recent papers and links

• Rod Furlan,”The future of augmented reality: Hololens - Microsoft's

AR headset shines despite rough edges”, IEEE Spectrum, 53 (6), p.

21, DOI: 10.1109/MSPEC.2016.7473143

36