The Environment,

Optics, Resolution, and

the Display

Ware Chapter 2

University of Texas – Pan AmericanCSCI 6361, Spring 2014

Last Time

• Considered Ware’s (and others) ideas about a “science of visualization”– What it is… and is it even possible to have a

“science of visualization?”• Ware argues it is possible because there are

“sensory symbols”, i.e., not all symbols are arbitrary

• Sensory vs. arbitrary symbols

Sensory vs. Arbitrary Symbols

• At core of issue of efficacy of visualization in understanding is:

– How “natural” vs. “learned” are elements of visual representations

• Sensory symbols:– “Symbols and aspects of visualization

that derive their expressive power from their ability to use the perceptual processing power of the brain without learning”

• Arbitrary symbols:– “Aspects of representation that must

be learned, because the representation have no perceptual basis”

Record of a hunt

Arbitrary Symbols

• “Aspects of representation that must be learned, because the representation have no perceptual basis”

• Derive power/utility from (learned) culture, so dependent on cultural milieu

– E.g., the ink of the characters “dog” on paper

• Which obviously has no chance to be perceptual, i.e., is completely a code

– vs. a picture of a dog• Which most likely has some unlearned

correspondence with the real animal

• And there are those that argue that all pictorial representations arbitrary

Record of a hunt

Sensory Symbols

• “Symbols and aspects of visualization that derive their expressive power from their ability to use the perceptual processing power of the brain without learning”

• Effective because well matched to early stages of perceptual processing– Human visual system has evolved to

detect forms and relationships in world– HVS not a fully general purpose system

• Not tabla rasa

• Tend to be stable across individuals, cultures, and time– E.g., cave drawing still conveys

meaning across millenia

Record of a hunt

Theory of Sensory Languages/Symbols

• Based on idea that human visual system evolved as an instrument to perceive physical world

– In contrast to view that visual system is “universal machine”, “undifferentiated neural net” that can configure for any world

• Brain tissue appears to be undifferentiated, but in fact morphology has specific neural pathways

– Anatomically same pathways among primates • And through experimentation some functions of

some areas are know, as shown on next slide– “collection of highly specialized parallel

processing machines with high bandwidth interconnections”

• System is designed (better, of course, evolved) to extract information from the (particular) world we live in

So, We’ll Have a Look at the System …

But, first…

• Ware poses an interesting perspective on how we, as visualization scientists – and computer scientists, might usefully “stand back” and think about things a bit differently …

• Recall, this is all about putting things on an “information display”

• I.e., on a computer display– Guy from last time– Viewing a visualization– During some task

• Guy is looking at the display– “(metaphoric) world is his screen”

How about “taking it to the next level”

• How about, “The (whole) world as information display”– Chapter 2, opening sentence:

• “We can think of the work itself as an information display”

• I.e., what we get into our eyes can be considered a stream of information that we interpret to form our perceptions, etc.

• and ears, etc.

The World Itself as Information Display

• We can think of the world itself as an information display.

• Objects may be used as tools or as construction materials, or they may be obstacles to be avoided.

• Every intricate surface reveals the properties of the material from which it is made.

• Creatures signal their intentions inadvertently or deliberately through movement.

• There are almost infinite levels of details in nature, and we must be responsive to both small and large things, but in different ways: large things, such as boulders, are obstacles; smaller things, such as rocks, can be used as tools; still smaller things, such as grains of sand, are useful by the handful.

• If our extraordinary skill in perceiving the information inherent in the environment can be applied to data visualization, we will have gained a powerful tool.

Ware, 3rd Ed., p. 31

The World Itself as Information Display

• We can think of the world itself as an information display. …

• The visual display of a computer is only a single rectangular planar surface … It is astonishing how successful it is as an information display, given how little it resembles the world we live in.

Ware, 2nd Ed., p. 29

The World as “Information Display?”

• A thought experiment to set the stage for Ware’s ecologically / evolutionarily / … orientation to information display

– Recall - Core idea: • Some things are “natural” (built-in in human sensation and perception in human sensation and

perception ) due to millions of years of evolution: sensory representation• Some things are learned: arbitrary representation

• “Objects in display” – things in (real) world– Objects are part of world, and we (humans) are constantly viewing the scene– Objects are, so, seen and perceived

• Can be things to walk on, over, around, to eat, make tools, …• And, e.g., surface signals material from which is made, plant, rock, …

• Creatures (animate objects) signal intentions through movement, …

• Levels of detail, … to the minute

• So, we humans, through vision, have powerful (and successful) tool

• That today are able to come even somewhat close to approximating the experience is perhaps surprising

– But, we have, even at 1920 x 1080 … and Moore’s Law is good

Overview

• Look at general description of “environment”– E.g., light, and consider ecological optics

• Revisit Ware’s model of perceptual progression

• The eye, nerve cells, transmission and such– Depth of view and chromatic aberration

• How to approximate objects on screen to get some semblance of “realism”– Looking back at computer graphics

• Visual acuities

• Brain pixels and optimal screen, and Ware’s “optimal display”

“The Environment” and Visualization

• Of course, in computer visualization “environment” considered as metaphor

– E.g., data “landscapes” – And, so, to some extent able to transfer skills in interpreting real environment to

understanding data• Hills, valleys, etc.

– More generally, role of metaphor in interpretation, use• E.g., pervasive “office” metaphor• But, not positive and negative transfer

– E.g., “folders” are nearly so hierarchical as a directory tree structure

• More fundamentally, understanding human perception facilitated by understanding the purpose of human perception

– The visual system has survival value– It works (relatively well) for its possessors to contribute to gene pool– Works well for skills such as

• Navigation (moving in environment)• Food seeking (like information seeking)• Tool use (depends on object-shape perception)

Environment: Sensation, then PerceptionAgain, …

• Sensation vs. perception– Sensation: excitation of sensory receptors

• Low level– Perception: process of creating “interpretation” of sensations

• Higher level– Perception is about “understanding”, “giving meaning to” patterns of sensation

• Sensory receptors respond to absolute levels, but that’s not what perceived

– Sensory receptors respond to energy, pressure, chemicals• Light, pressure (in air), pressure (on skin), chemicals (in air), chemicals (in mouth)• Transducers

– Change one form of signal (in environment) to another (in nervous system)

• Neurons signal each other by increasing or decreasing firing rate relative to background

– Neuron can receive input from hundreds or thousands of neuron• Some increase firing rate• Some decrease firing rate

– rate of chemical discharge • Some …• Will see more later

A Model of Perceptual Processing – Ware, Ch. 2

• The big picture … again

A Model of Perceptual Processing – Ware, Ch. 2

• An information processing model– “Information” is transformed and processed

• Physical light does excite neurons, but at this “level of analysis” consider information– Gives account to examine aspects important to visualization

• Here, clearly, many neural subsystems and mapping of neural to ip is pragmatic– In spirit of visualization as evolving discipline, yet to develop its theories, laws, …

• Stage 1: Parallel processing - extract lo-level properties of vis. scene

• Stage 2: Pattern perception

• Stage 3: Sequential goal-directed processing

What we do is design information displays!

Stage 1: Parallel Processing to Extract Low-level Properties of Visual Scene (1/2)

• (Very first) neurons fire – Sensation

• Visual information 1st processed by:– large array of neurons in eye – primary visual cortex at back of brain

• Individual neurons and sets selectively tuned to certain kinds of information

– e.g., orientations of edges or color of light– Evoked potential experiments

• In each subarea large arrays of neurons work in parallel

– extracting particular features of environment (stimulus)

Stage 1: Parallel Processing to Extract Low-level Properties of Visual Scene (2/2)

• At early stages, parallel processing proceeds involuntarily

– largely independent of what choose to attend to (though not where look)

• Rapid– if want people to understand information

fast, should present in way so is easily detected by these large, fast computational systems in brain

• Pre-attentive processing

• Stage 1 processing is:– Rapid and parallel– Entails extraction of features, orientation,

color, texture, and movement patterns– “transitory”

• only briefly held in iconic store

– Bottom up, data-driven

Stage 2: Pattern Perception (1/2)

• Rapid processes continue

• Divide visual field into regions and simple patterns, e.g.,

– Continuous contours – Regions of same color – Regions of same texture …

• “Active”– but not conscious processes

• Specialized for object recognition– Visual attention and memory

• E.g., for recognition must match features with memory

– Task performing will influence what perceived

– Bottom up nature of Stage 1, influenced by top down nature of Stage 3

Stage 2: Pattern Perception (2/2)

• Specialized for interacting with environment

– E.g., tasks involving eye-hand coordination

• “Two-visual system hypothesis”– One system for locomotion and eye-hand

coordination• The “action system”

– One system for symbolic object manipulation

• The “what system”

• Characteristics:– Slow serial processing– Involvement of both working (vs. iconic)

and long-term memory– Both bottom up and top down

• More emphasis on arbitrary aspects of symbols than Stage 1

• Top-down processing– Different pathways for object recognition

and visually guided motion

Stage 3: Sequential Goal-Directed Processing (1/2)

• At highest level of perception are objects held in visual memory by demands of active attention

• To use an external visualization– we construct a sequence of visual queries

that are answered through visual search strategies

• Only few objects can be held at a time

• Objects constructed from available patterns providing answers to the visual queries

– E.g., if use a road map to look for a route, visual query will trigger a search for connected red contours (representing major highways) between two visual symbols (representing cities)

Stage 3: Sequential Goal-Directed Processing (2/2)

• Are other subsystems, as well:

– Visual object identification process interfaces with the verbal linguistic subsystems of the brain so that words can be connected to images

– The perception-for-action subsystem interfaces with the motor systems that control muscle movements

• 3 stage model of perception is basis for organization of book (next slide)

Ware’s Book Organized by 3 Stages

– 3 stage model of perception is basis for organization of book:

– Stage1, Low level parallel processing: • 2: The Environment, Optics, Resolution, and the

Display• 3: Lightness, Brightness, Contrast, and Constancy• 4: Color• 5: Visual Attention and Information that Pops Out

– Stage 2, Pattern perception• 5: Visual Attention and Information that Pops Out• 6: Static and Moving Patterns• 7: Visual Objects and Data Objects• 8: Space Perception and the Display of Data in

Space

– Stage 3, Sequential goal directed processing• 9: Images, Words, and Gestures• 10: Interacting with Visualizations• 11: Thinking with Visualizations

Environment: Ecological Optics

• Again …

Environment: Ecological Optics

• Again, JJ Gibson’s “top down” orientation

• Bottom up – Individual “primitives” are successively

assembled …

• Top down, “utility” drives perception, affordances– Perceptual systems have evolved for the

organism to “do well”• i.e., survive in particular environments

– Not, fully general purpose systems

• Surely both, matter of degree

Environment: Ecological Optics

• Emphasized perception of surfaces …– …and textures– In contrast to earlier/other accounts of

perception that considered perception in terms analogous to classic geometric terms

• In terms of points, lines, planes, …

• A (geometric) plane is not the same as a surface (of an object) in the environment (in which we evolved)– New, or revolutionary, view of perception

• Affordances are those things which let us (provide us the opportunity to, or, afford us the opportunity) do something– … and something with survival value

Environment: Ecological Optics: FYI - Surfaces (note)

• Again, JJ Gibson’s “top down” orientation– Perceptual systems have evolved for the organism to “do well”, i.e., survive in particular

environments– Not, fully general purpose systems

• Emphasized perception of surfaces …– And textures

• Affordances are those things which let us (provide us the opportunity to) do something

– And something with survival value

• Rather than concentrating on image on retina, emphasized perception of surfaces in the environment

– Not grounded in classical geometry of points, lines and planes, rather (Gibson, 1979, p. 35):• A surface is substantial; a plane is not. A surface is textured; a plane is not. A surface is never

perfectly transparent; a plane is. A surface can be seen; a plane only visualized• A fiber is an elongated object of small diameter, such as a wire or thread. A fiber should not be

confused with a geometrical line.• In surface geometry the junction of two flat surfaces is either an edge or a corner; in abstract

geometry the intersection of two planes is a line– And, from the principal of survival value, it makes sense that system would have evolved for

“real” elements– A key function of the human visual system is to extract properties of surfaces

• Surfaces are the primary contact with objects• So, essential for humans understanding potential for interaction and manipulation in the environment

Environment: Visible Light

• Body’s sensory system is way it is because it had survival value– Led to success (survival and reproduction)

• Focus on human vision, but all organisms & senses share basic notions

• Humans have sensory receptors for (a small part of) electromagnetic spectrum

– Have receptors sensitive to (fire when excited by) energy 400-700nm• Transducers: change energy to neuron firings

– Snakes “see” infrared, some insects ultraviolet, i.e., have receptors that fire– What would life be like if could see other parts of electromagnetic spectrum???

Environment: Ecological Optics: Ambient Optical Array

• Have seen “ambient light” in cg shading of objects:

– Light strikes and object (and is reflected to our eyes) after a most complex series of interactions

– “Photons traveling everywhere”: absorbed, reflected, refracted, diffracted, as interacts with objects

• Ambient optical array is (all) light reaching a point

• Raytracing in computer graphics uses this approach

• Constrained in practice to relatively small number of elements in cg scene, vs. nature

– All light reaching eye results in what is seen

• So, simply put, it’s just what you see• … or what comes in the lens

– Human or camera!

Environment: Ecological Optics: Optical Flow: Movie Example

• Changing ambient optical array

– Changes over time as viewpoint moves

• Flow field develops

• Demo

Environment: Ecological Optics: Optical Flow

• Ambient optical array is dynamic– Changes over time as viewpoint and objects move

• “scenes” change• In illustration below, consider moving to apple …

– Some elements “go backwards” as approach– Others subtend larger percent of fov as become closer and so appear to move apart

• As advance in a static environment, a characteristic flow field develops– Evidence that visual system includes processes to interpret flow patterns

• Motion perception is– “built in” and “important”

• Pre-attentive (not consciously constructed)– Hence can be used in sensory symbols!

• Animation exploits

Environment: Ecological Optics: Optical Flow: Example

• Changing ambient optical array

– Changes over time as viewpoint moves

– flow field develops

Stereoscopic Vision - FYI

• More built in things …– More later

• Perceptual processes integrate different views from the two eyes

• Many other “cues” for depth perception

Environment - Ecological Optics: Textured Surfaces and Gradients

• Texture is a fundamental property of an object– Surface just unformed patch of light, unless textured– “Ecologically” useful, so likely organisms handle well in

sensation and perception

• Texture critical to perception– Facilitates shape and location recognition– Provides ground plane of locomotion– Again, pre-attentive, not consciously constructed

• Texture facilities in most visualization packages

Environment: Textured Surfaces (supplementary)

• Again, from Gibsonian perspective, surfaces and textures are fundamental elements of perception

– With emphasis on utility of action– Processed preattentively, so candidates for

sensory elements

• Textons– fundamental micro-structures in generic natural

images– basic elements in pre-attentive visual perception

• Textons can be classified into three general categories:

– 1. elongated “blobs”• line segments, rectangles, ellipses with specific properties

such as hue, orientation, and width, at different level of scales

– 2. terminators (end of line segments)– 3. crossings of line segments

Objects their textons

Environment: Textured Surfaces (supplementary)

• Only a difference in textons or in their density can be detected pre-attentively

– no positional information about neighboring textons is available without focused attention

• pre-attentive processing occurs in parallel (fast)• focused attention occurs in serial (slower)

• Example: although the two objects look very different in isolation

– (a), they are actually the same texton (b)

Objects their textons

Textures, Symbols, and Design (supplemental)

• When designing textures to indicate different regions of a visualization, make sure that the textons are as different as possible

• Same rules apply when designing symbol sets

• Example: A tactical map may require the following symbols:

– aircraft targets– tank targets– building targets– infantry position targets

Textures, Symbols, and Design (supplemental)

• Each of these target types can be classified as friendly or hostile

• Targets exist whose presence is suspected but not confirmed

– This uncertainty must be encoded

• Set of symbols designed to represent different classes of objects

– symbols should be as distinct as possible with respect to their pre-attentive processing

– military reconnaissance must occur fast

Paint Model of SurfacesSaw this in cg class!

• “Simple model” of shading:– Good enough, or in fact may

capture what is important to humans in perceiving surfaces

– Phong, 1975

• Ware: “glossy paint surface”– Pigment particles embedded

in a (more or less) clear medium

– Color of particles show thru– Reflected light is color of

illumination

Diffuse (Lambertian) + Specular + Shading

Paint Model of Surfaces… and visualization

• It may be the model that the brain uses for shape– A more complex model may

actually impede estimation understanding of the surface

• Lambertian (diffuse) and texture better for overall shape perception

• Specular better for small details, if the lighting is just right

• Shadows indicate relative heights of objects, distances

Paint Model of SurfacesGuidelines

• As a note, Ware presents guidelines throughout book

• Worth a look

Paint Model of SurfacesCg again

• Ambient, diffuse, and specular– No reflections– No shadows– 30 frames/second– Hardware– Plus tricks

• Here, “smoothing” intersection of polygons by linear interpolation

• Gouraud shading

Shading Depth Cues

• Shape cues are given from shading

About the Human System (wetware)

About the Human System (wetware)

• Visual pathways – and where some things happen

• Eye– Nerve cells

• In fact … Not at all do they “just fire”, which is bad news for simple models– Lens

• Depth of focus, depth of field– Chromatic aberration

• Example of how understanding human wetware system helps in visualization (and graphic) design

– Receptor distribution

• Acuities – how well things can be “resolved”– Grating, point, letter, stereo, vernier– Acuity distribution in visual field– Spatial contrast sensitivity function

• Aliasing

• Ware’s “optimal display”

Preattentive Processing and Illusions

Preattentive Processing and Illusions

Preattentive Processing and Illusions

• What’s wrong with this triangle?

• Impossible (or at least difficult) to build

• Cues for perception are, here, misleading

– Rapid, “preattentive” (“unconscious”) processes of sensation lead later perceptual processing

– Must rely on conscious (rational) processes intelligence to figure it out

– Conscious/rational processes much slower– Nice illustration of “confirming and

disconfirming perceptual hypotheses”• Trying to figure it out• Go back and forth from whole to parts!

Visual Pathways and Where Some Things Happen

• Sensation vs. Perception

• “Preattentive”– Early in the system– Candidates for sensory

language

• Interaction between early and later stages– E.g., “impossible figures”

Visual Pathways

• Neural systems and pathways from front to back …

• Can loosely talk about where things happen neurologically

Visual Pathways

• Neural systems and pathways from front to back …

• E.g., edge detection early– More later,

receptive fields

Pathways

• 1st neurons on retina

• Make way via various structures, e.g., lgn, to visual cortex

• … and beyond

• From Ware’s “Visual Thinking for Design”

Visual Cortex

• Cortex top layers

Visual Cortex

• E.g., “where”, “what”

Visual Cortex

• “where”

Visual Cortex

• “what”

Human Eye

• Eye as the “instrument of sight”

– As camera does, has equivalent of lens, aperture (pupil), and film (retina)

– Note that image is upside down on retina!

• … ah, perception!

Human Eye: Retinal Receptors

• Two types of (photo) receptors on retina: rods and cones

• Rods:– Spread over retinal surface (75 - 150

million)– Low resolution, no color vision, but very

sensitive to low light (scotopic or dim-light vision)

• Cones:– Dense array around the central portion of

the retina, fovea centralis (6 - 7 million)– High-resolution, color vision, but require

brighter light (photopic or bright-light vision)

Nerve Cells: Complex Connectivity

• Complex “connectivity”– Many inputs,many interconnections

Nerve Cells: Complex Interaction

• Complex interaction across synaptic cleft– Chemical discharge, uptake, pulsing, rate, …

Nerve Cells: Complex Interaction

• Complex interaction across nerves– Chemical discharge, uptake, pulsing, rate, …

Nerve Cells: Complex Interaction: Animation

• Complex interaction across nerves– Chemical discharge, uptake, pulsing, rate, …

(not working 3/4) http://harveyproject.science.wayne.edu/development/nervous_system/cell_neuro/synapses/release.html

http://highered.mcgraw-hill.com/sites/0072495855/student_view0/chapter14/animation__transmission_across_a_synapse.html

Human Eye: Visual Angle

• Visual angle is angle subtended by object at eye of viewer– In degrees, minutes, seconds of arc– Thumbnail at arm’s length subtends ~1o of visual angle

• … try it sometime

– 1cm (2/5”) object at 57 cm (20”), monitor distance, ~1o of visual angle

Human Eye: LensFYI

• Eye has compound lens: – cornea (power) and lens (adjust focal length)

f = focal length of lensd = distance to objectr = distance to image that is formed

– Flexibility of lens changes with age, approaching 0 at 60 years

Human Eye: Depth of FocusFYI

• Depth of focus– Distance over which objects are in focus without change in focus– Varies with size of pupil– Range of focus:

Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

– Rarely do computer systems model

Human Eye: Depth of Field:FYI - Example

• Photographic Images:– Depth of field longer with small aperture (f stop)– Range of focus:

Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

Human Eye: Chromatic Aberration

• Different wavelengths of light focused at different distances within eye

• Short-wavelength blue light refracted more than long-wavelength red light

• Focusing on a red patch, an adjacent blue patch will be significantly out of focus

• Human eye has no correction for chromatic aberration

• Strong illusory depth effects

Human Eye: Receptors and Fovea

• Lens focuses image on mosaic of photoreceptor cells lining retina

• Fovea– Small area in

center of retina densely packed with cones

– Vision sharpest– ½o - 2o of arc

• Blind spot, tooReceptor mosaic in fovea

Human Eye: Receptors and Fovea“The first complexity”

• Lens focuses image on mosaic of photoreceptor cells lining retina

• Fovea– Small area in center of

retina densely packed with cones

– Vision sharpest– ½o - 2o of arc

• From 1 – 100 receptors feed into 1 ganglion cell, the first “complexity” …

Receptor mosaic in fovea

Human Eye: Acuity Distribution and the Visual Field

• Again, receptors densely packed at fovea

• Binocular overlap – Region of visual field viewed by both eyes– Only here, stereopsis

• Visual acuity non-uniformly distributed over visual field

– E.g., Can resolve only about 1/5 detail at 10o from fovea

• Next slide (tries to) demonstrate “equi-resolvability” of characters as a function of distance from fovea

– resolvability = f(dist. fovea)– Focus on center, and letters throughout field

about equally “sharp” or “clear”

Human Eye: Resolution

Acuity Distribution and Visual Field

• Previous showed equi-resolvability

• As does right– From C. Ware, “Visual

Thinking for Design”

Human Eye: Simple Acuities

• Acuities– Measurements of abilities to see detail– Provide ultimate limits of information densities can perceive

• Grating Acuity

• Point, Line, Stereo, Vernier

• Acuity distribution and the visual field

Human Eye: Simple Acuities: Grating Acuity

• Acuities– Measurements of abilities to see detail– Provide ultimate limits of information

densities can perceive

• Resolve to ~1 min. (1/60 deg) f arc– Roughly corresponds to receptor spacing

in fovea– E.g., to see 2 lines as distinct blank space

between should lie on receptor– So, should be able to perceive lines

separated by twice receptor spacing

• Superacuities– Resolution above what expected by

receptor density due to integration of signals, recall retinal structure, etc.

Grating Acuity - 1-2 minutes of arc - Ability to distinguish a pattern of bright and dark bars from a uniform gray background

Human Eye: Simple Acuities: Point, Letter, Stereo, Vernier

• Point acuity– 1 minute of arc– Ability to resolve two distinct point

targets

• Letter acuity – 5 minute of arc– Ability to resolve letters– Snellen eye chart

• 20/20 means a 5-minute letter target can be seen 90% of time

• Stereo acuity– 10 seconds of arc– Ability to resolve objects in depth– Measured as difference between 2

angles (a and b) for a just-detectable difference

• Vernier acuity– 10 seconds of arc– Ability to see if two lines are collinear

Brain Pixels and the Optimal Screen (opt.)

• So, visual acuity highest at fovea and lowest at periphery

– Now optimal screen, later optimal display

• Consider “brain pixels” (bp) as number of eye receptors

– As screen resolution is from number of picture elements (pixels) per unit size

• E.g., 100 dpi for 12” wide 1200 pixel screen

• Acuity graph at right shows:– foveally (and at center of screen) many

bp’s, for each screen pixel (sp, cf. a, b)• Circles = receptors, bp • Squares = screen pixels, sp• So, could use more screen pixels, i.e.,

higher resolution screens– At edges, mismatch of bp/sp is less (c, d)

• Large screen might even match bp and sp• With small screen actually more screen

pixels than brain pixels– Receptors can make use of this level of

resolution– So, wasted screen pixels

Brain Pixels and the Optimal Screen (opt.)

• Wasted pixels (inefficiency) in display (c)

• To describe display and brain pixels:– TPB, total n brain pixels stimulated by display– USBP, n uniquely stimulated brain pixels

• Some bp get same information (a,b)– USBP = TPB – redundant brain pixels

• Which takes a while, e.g., in a = 7, and different for each screen pixel, as different bp density at different angles

– DE, display efficiency• How efficiently a display is being used, in terms

of sp and bp– DE = USPB / SP

• Finally, proportion of bp in screen area that are getting unique information

– VE, visual efficiency– VE = USPB/TPB

• Figure (2.22)– 1m pixel display at 50 cm, considering angle of

view

Human Eye: Visual Field and Immersive Displays

• More later …

• Immersive displays, well, immerse the user in the scene– Provide relatively large field of

view– Sense of engagement

increased– “Virtual reality”

• (but it all is)• Just more or less

• HMD and glove

• Motion tracking for head position and update of displayed image

• Computationally expensive

Human Eye: Visual Field and Immersive Displays

• Data wall– Still relatively wide fov– Unincumbered– Multiple viewers– As theaters– Stereo, sound

• CAVE– More is better

Human Eye: Visual Field and Immersive Displays

Percent of eye receptors stimulated

Human Eye: Spatial Contrast Sensitivity Function (opt.)

• Grating luminance– Light intensity variance over

distance (contrast)

• Sine wave grating pattern– Brightness varies sinusoidally in

one direction only

• Can also vary– Spatial frequency

• Number of bars of the grating per degree of visual angle

– Orientation– Contrast

• Amplitude of sine wave– Phase angle

• Lateral displacement of pattern– Area covered by grating pattern

• Can measure sensitivity of eye/brain to lowest contrast that can be detected and see how varies with spatial frequency

Spatial frequency

(h

igh

er)

C

on

tra

st

Use part to test with subject

Human Eye: Spatial Contrast Sensitivity Function (opt.)

• Illustrates high frequency fall off in sensitivity

• Sinusoidally modulated pattern of stripes

– varies left to right in spatial frequency

– top to bottom in contrast

• Where can you detect differences?

Co

ntr

ast

Spatial frequency

Human Eye: Spatial Contrast Sensitivity Function (opt.)

• Contrast sensitivity decreases with age

Human Eye: Spatial Contrast Sensitivity Function

• Consider presenting grating pattern at some frequency

– Oscillate contrast high to low

• Contour map of human spatiotemporal threshold surface

– Each contour represents a particular combination of spatial and temporal frequencies detected

Human Eye: Visual Stress

• Pattern-induced epilepsy

Human Eye: Visual Stress

• Pattern-induced epilepsy

• Ware, 12/17/97 700 Japanese children ill by repetitive flashing light of computer-generated cartoon

– Most recovered in ambulance, but some hospitalizations– Epilepsy and vomiting blood

• Combinations of spatial and temporal frequencies are more or less stressful

– Striped patterns of 3 cycles per degree– Flicker rates of 20 Hz

• Striped patterns in general– Consider some fonts– FFFFFFFFFFFF– FFFFFFFFFFFF

Pokemon Seizure Image

• Red and blue flashing lights when rocket exploded

– 12 hz

• Plenty on You Tube …but US versions

– ~ 1 hz

“Optimal Display” – Ware (opt.)

• Acuity information useful in determining parameters of optimal (or adequate) display

• Typical “high resolution monitor”– ~ 35 pixels / cm

• ~100 dpi– (decent laser printer ~600 dpi)

– Translates to ~40 cycles / degree at normal viewing distance

• Fovea has ~180 receptors / degree– 4000 x 4000 ~20” monitor should do it

• Vs. 1280 x 1026 or 1920 x 1080• Note, resolution, formally, is metric of “dots” per inch, size matters

– Handheld displays recently reach 400+ dpi, so, … we’re there! (Moore’s law is good!)

• Spatial modulation transfer function suggests same– Humans can resolve grating ~50 cycles / degree– Sampling theory says need to sample at twice highest frequency wish to detect– So, ~ 100 pixels / degree, and ~150 pixels / degree would be reasonable

Optimal Display: Headtracked, Focus, etc.

• Consider immersive display with above functionalities

Optimal Display: Aliasing

• Term “aliasing” comes from signal theory

– “there are aliases for results of sampling”

– Need to sample (at least) at twice the frequency of the source signal

– If not, multiple signals can result in same results

Optimal Display: Aliasing

• In computer graphics problems arise as well

• Idea is that digital device “samples” from analog “things” to be displayed– Grid– Line

• Techniques for attenuating effects, but are inherent in representing analog elements on discrete devices

Optimal Display: Aliasing

• Always will be a problem

Optimal Display: Aliasing

• Antialiasing helps– Average over pixels

End

• .