Augmented Reality and Visualization Lecture 29 By Prof. Dr. Sajjad Mohsin Today we shall cover Augmented Reality History, Applications, Limitations, Future Visualization Definitions History Types Domains of Applications 2 What is Augmented Reality? Augmented reality is a combination of a real life scene from a person, and a virtual scene from a computer that augments the scene with additional information Specifically, augmented reality superimposes audio, graphics and other virtual affects upon the surrounding environment. 3 The Goal Augmented reality enhances one's perception. The real world is simply the foundation. The goal is to make a system that will be so efficient that the user will not be able to tell the difference between the real world and the virtual augmentation. 4 Virtual vs. Augmented Virtual Reality is defined as, "a computer generated, interactive, three-dimensional environment in which a person is immersed. In other words, virtual reality completely immerses the use in the virtual environment Augmented reality virtually changes the real world while allowing the user to keep a sense of presence about his surroundings 5 Performance Criteria The mechanism that augments reality can be judged based on two things Update rate for generating the augmented image Accuracy of the registration of the real and virtual image So in order for AR to be perceived as real, the rate at which the system renders the image must be no fewer than 10 images a second. It also means that the system must be able to register its environment accurately enough so that a virtual change can be super imposed 6 The Hardware In order for the AR system to work, certain hardware is needed, including: display, tracking, input devices, sensors and a processor. Currently, components such as a CPU, camera, display, accelerometer, GPS and solid state compass. 7 Display One can perceive augmented reality in generally three different ways: Head mounted: the most immersive experience, HMD puts the virtual scene over the users view of the world. The device is either optical see- through or video see-through Handheld: the most commercially successful AR system, handheld display is generally video see-through with a gyroscope in order to orient the image Spatial: any AR device separated from the user. Can be used by multiple people. In some cases, users are able to touch virtual physical objects giving a sensation of realness. 8 Software AR vision methods are based on visual odometry, or determining position using camera images This generally includes feature detection and other image processing methods It also requires a real world coordinate system to be made once the image is processed, created by perhaps fiduciary marks on the image, simultaneous localization and mapping of the image, or other mapping methods 9 Applications: Medical Augmented Reality has the ability to enhance the efficiency of surgeries and diagnostics Virtual x-rays Real time ultra-sound images Nuclear magnetic resonance images MRI image 10 Applications: Military Numerous applications: Detect objects not discernable to the naked eye, such as toxins Examine sound of gunfire to determine location of shooter and type of weapon Enhanced visual feedback of surroundings Represent physical objects that arent yet visible, such as an airplane that has yet to land Displays in cockpits of military vehicles War game simulation technology Etc. 11 Applications: Navigation Building navigation; in order to analyze and explore buildings effectively Military operations Disaster management Heads up displays that cover the displays of automobiles, planes, etc. Individual navigation through new urban areas (street to street, place to place (GPS)) 12 Applications: Entertainment, Performance and Education Augmented Reality systems can create virtual objects and environments that enhance the experience of entertainment, and the quality of education Musical performances can be augmented to add to the experience or create certain sound effects Interaction with educational tools Virtual guides that can even activate based on location on GPS or proximity to marker 13 Applications: Digital Applications Since smart phones come with most of the basic components for an AR system, applications soon began to flood the smart phone market, making AR commercially successful. 14 Smart Phone App -- Wikitude Gives the user data about their surroundings by overlaying information on the real-time scene taken by the phones camera 15 Smart Phone App -- Augmented ID Gives the user the ability to identify someone by looking at them through the camera of the phone 16 Smart Phone App -- Image Space Lets the user record messages, photos and video and tag them with both place and time. Then, when someone else goes to that same location they can see the images that were left in the area 17 Smart Phone App -- Layer Positioning the camera over your environment, the phone can overlay images from web pages and information about businesses in the area 18 Smart Phone App -- Video Games A game is generally overlayed on to the users surroundings, then the user may interact with the scene through the phone 19 Everyday Augmented Reality Powers immersive video games Very common to use it to navigate ones surroundings Interact and understand digital content more interestingly Clear benefits recognized in using AR technology Now found on most smart phones, opening up a whole new way to interact with the digital and physical environment There is now an entire market for AR apps, including ones that possibly give schematics for a building, the constellations in the sky or the information about nearby restaurants AR now available in Adobe Flash, making AR accessible through web browsing Recently there has been a new form of crude 2D bar code technology, which can be used to quickly transfer information 20 Limitations and Cons Does not work as well indoors GPS only accurate within 9 meters Danger of too much information, where a disconnect forms between the user and the real world Privacy will be harder to keep due to virtual interaction 21 Limitations and Cons Over reliance on AR technology Image recognition technology not developed to potential yet Inaccurate depth perception Calibration and latency still poor 22 The Future of Augmented Reality Next generation of smart phones will be able to handle augmented reality to a much greater extent due to improved batteries, compasses, graphics chips and processors. The mobility of the smart phone and the location based services make AR a very popular investment. 23 Future. The line between virtual and physical begins to blur as augmented technology grows Greater access to information based on AR will mean greater awareness of surrounding Virtual graffiti and art could be developed Desire for increased AR technology in the field of medicine, as well as other fields. 24 Information and Data Visualization Visualization A picture is worth more than a thousand words. a Chinese proverb A picture is worth more than a thousand numbers. 26 It looks like a swirl. There are smaller swirls at the edges. It has different shades of red at the outside, and is mostly green at the inside. The smaller swirls have purple highlights. The green has also different shades. Each small swirl is composed of even smaller ones. The swirls go clockwise. Inside the object, there are also red highlights. Those have different shades of red also. The green shades vary in a fan, while the purple ones are more uni-color. The green shades get darker towards the outside of the fan (10,20,21), (12,13,14), (13,32,12),...., (1,2,3), (2,4,5),(3,5,6),..... Terrain geometry: Terrain Texture: Time 0: (23,34,54), (23,34,23), (45,26,78),.... Volumetric cloud cover: 0, 0, 12, 14, 15, 15, 17, 12, 23, 45,..... Wind vectors: (0.2, 0.3, 0.93,5), (0.4,0.5,0.76,12),..., Volumetric cloud cover: 0, 0, 11, 12, 13, 16, 20, 12, 32, 45,..... Wind vectors: (0.4,0.5,0.76,12),(0.5,0.5,0.7,6),... Time 1: 28 What Is Visualization? seeing is believing we observe and draw conclusions seeing is also understanding beware of illusions (magicians) 29 What Is Visualization? Transformation of data or information into pictures engages primary human sensory apparatus - vision 30 What Is Visualization? Is a Tool for: Aid For Learning/Understanding Compact Representation Of Information (e.g. Numbers) Carrier of Information 31 Visualization Flavors? Scientific Vis. - User Interfaces, data representation/processing Algorithms, Visual Representations Data Visualization - Include financial data and statistical methods Information Visualization - Abstract Data: WWW documents, file structures, arbitrary relationships 32 History earliest known map (China) first star charts by Johann Beyer cartesian coordinate system (Descartes) 33 History (2) - Statistical first meteorological chart (Halley) mortality tables of city of Breslau (Halley) -> first attempt to correlate two variables 34 History (3) - 2D Approx contour lines (height) isotherms (temperature) isochromatic lines (color) isobars (pressure) 35 History (4) - 3D Imaging X rays by W. Rntgen stereo X rays (mackenzie- davidson) - locating foreign bodies in humans x-ray sections or slices (3D!) x-ray crystallography (Laue) - position of atoms in a crystal 36 History (5) - Computer Graphics SAGE air defense - tracked position of aircraft by radar, analyzed results and display on CRT sketchpad (Sutherland) - interactive graphical drawing system Used to be BIG and EXPENSIVE 37 History (6) - Scientific Visualiz NSF report [McCormick87] Personal/exploratory graphics - to enable a scientist to gain more knowledge (interact with data) Peer graphics - enable scientist to show information to their colleagues and to collaborate Presentation graphics - communicate information and results (high quality, fully annotated) Publication of visualization - enable others to use the data (replicable) 38 History (7) - Augmented Reality responsive environments (Myron Krueger) 39 Visualization Domains Volumetric data sources are usually produced by: Scanning devices Computation (mathematical), or Simple measuring 40 Applications - Vis. As a Toolkit Application tools usually coupled with Haptic feedback devices Stereo output (glasses) Interactivity demanding of the rendering algorithm 41 Scanning - Domains Medical scanners (MRI, CT, SPECT, PET, ultrasound) 42 Scanning - Applications Primary education Medical education for surgery, anesthesia Illustration of medical procedures to the patient 43 Scanning - Applications Surgical simulation for treatment planning Tele-medicine Inter-operative visualization in brain surgery, biopsies, etc. Industrial purposes (quality control, security) Games with realistic 3D effects? 44 Scanning (2) Domain - biological scanners, electronic microscopes, confocal microscopes Apps - paleontology, microscopic analysis 45 Scientific Computation - Domain Mathematical analysis ODE/PDE (ordinary and partial differential equations) Finite element analysis (FE), Supercomputer simulations, 46 Scientific Computation - Apps Computational fluid dynamics (CFD), Computational field simulations (CFS), 47 Vector Field Viz Applications Computational Fluid DynamicsWeather modeling 48 Vector Field Visualization Challenges General Goal: Display the fields directional information Domain Specific: Detect certain features Vortex cores, Swirl 49 Streamlines A curves that connect all the particle positions 50 Streamlines (contd) - Displaying streamlines is a local technique because you can only visualize the flow directions initiated from one or a few particles - When the number of streamlines is increased, the scene becomes cluttered - You need to know where to drop the particle seeds - Streamline computation is expensive 51 Measuring - Domains Orbiting satellites Spacecraft Seismic devices Statistical Data 52 Measuring - Applications for military intelligence, weather and atmospheric studies planetary and interplanetary exploration oil, precious metal exploitation, and earth quake studies Statistical Analysis - Info Vis (Financial Data ) 53 Taxonomy Volumes Surfaces Surface extraction polygonalization Computer graphics Image processing CT, MRI, Ultrasound Seismic Numerical Simulations Data Geometric model Voxelization discretization Scanners, sensors, cameras sampling scanning Supercomputers Computation / Simulation Image (signal) visualization Computer vision / Pattern Recognition Display Video Recording 54 Viz vs. Graphics vs.. Imaging Imaging - Enhance, analyze, manipulate and store 2D/3D images Graphics - Make pictures! Digital Image Synthesis: sampling + illumination Visualization - Exploration, transformation, viewing data as images 55 Relation To Other Fields Visualization Vision Signal/Image Processing Illumination Engineering Optics Computational Geometry Applied Mathematics Hardware User Interfaces Psychology Cognition 56 Famous topics of Visualization Data representation on various types of grids Rendering of scalar and multi-modal data sets Rendering of vector fields and diffusion data sets Efficient iso-surfacing algorithms Distance fields and voxelization Parallel graphics and visualization Point-based graphics Image-based graphics Information visualization Basic geometrical modeling concepts 57 Summary By interacting with the visualization, the user may obtain a deeper understanding of the data and discover features that would be impossible with static representations. Visualization research is a growing field in computer science. 58