Download - Euro Va2011talk2
![Page 1: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/1.jpg)
Brian Fisher
Interactive Arts and Technology & Program in Cognitive Science
Simon Fraser University
Visual Analytics as a Cognitive Science
![Page 2: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/2.jpg)
![Page 3: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/3.jpg)
![Page 4: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/4.jpg)
Cognitive &
Perceptual
Sciences
VisualInformation
Systems
Graphic & Interaction Design
Mathematical & Statistical
Methods
EffectiveSituated
R&D
![Page 5: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/5.jpg)
![Page 6: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/6.jpg)
•“This science must be built on integrated perceptual and cognitive theories that embrace the dynamic interaction between cognition, perception, and action. It must provide insight on fundamental cognitive concepts such as attention and memory. It must build basic knowledge about the psychological foundations of concepts such as ‘meaning,’ ‘flow,’ ‘confidence,’ and ‘abstraction.’ “
•“Illuminating the Path” (IEEE Press)
“The science of analytical reasoning facilitated by interactive
visual interfaces”
Visual Analytics
![Page 7: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/7.jpg)
•Technical, functional & operational drivers
• Availability of massive data & processing
• Perceptually rich interactive displays
• Cognitive performance is key
• Diverse users: Novices, experts, multidiscipline, working in collaboration
•Complex design space with weak metrics
•Design stalls for lack of knowledge sufficient to steer through complex design space
Need for a Scientific Approach
![Page 8: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/8.jpg)
≠ ≠
Diversity of users ranging from novice ⇒ expert collaborating on tasks that require analysis, judgment, and coordination
Bill Buxton/Dave Kasik
![Page 9: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/9.jpg)
•Some cognitive processes are hard-wired-
• Sensory processes, e.g. 3 primary colours
• Performance bottlenecks e.g. STM of 7 +- 2
•Other aspects can change over time, based on experience and knowledge
•Intuitions about architectural processing are usually inaccurate = “metacognitive gap”
Implications of Cognitive
Architecture
![Page 10: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/10.jpg)
![Page 11: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/11.jpg)
•The study of thought, learning, and mental organization, which draws on aspects of psychology, linguistics, philosophy, and computer modelling. (Oxford)
•Cognitive Science Society founded in 1979
Cognitive Science
![Page 12: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/12.jpg)
Cognitive Science Society
![Page 13: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/13.jpg)
•Daniel Bobrow - AI
•Eugene Charniak - AI
•Allan Collins - Psychology
•Edward Feigenbaum - AI
•Charles Fillmore - Linguistics
•Jerry Fodor - Philosophy
•Walter Kintsch - Psychology
•Donald Norman - Psychology
•Zenon Pylyshyn - Psychology
•Raj Reddy - AI
•Eleanor Rosch - Psychology
•Roger Schank - AI
Society Founders
![Page 14: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/14.jpg)
Key figures not at Dallas
AILinguisti
csNeurosci
Philosophy
Psychology
Chomsky ✓ ✓
Miller ✓ ✓ ✓ ✓
Minsky ✓ ✓ ✓
Newell ✓ ✓
Simon ✓ ✓ ✓
![Page 15: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/15.jpg)
![Page 16: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/16.jpg)
•A division of labour?
•Simon’s“nearly decomposable problems”
•Conceptual & methodological “trading zone”?
•Galison: Image and Logic: a Material Culture of Microphysics
•Thagard: Trading Zones in Cognitive Science
What Happens in the Maze?
![Page 17: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/17.jpg)
•“If politics is the art of the possible, research is surely the art of the soluble. Both are immensely practical-minded affairs”.
Sir Peter Medawar
Visual Analytics Science & Technology
![Page 18: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/18.jpg)
"Il n'existe pas une catégorie de sciences auxquelles on puisse donner le nom de sciences appliquées. Il y a la science et les applications de la science, liées entre elles comme le fruit à l'arbre qui l'a porté"
Louis Pasteur
Pure Basic Research(Bohr)
Use-inspired Basic
Research (Pasteur)
Sampling,Description,Taxonomy(Audubon)
Pure Applied Research (Edison)
Quest for Fundamental
Understanding?
No
Yes
Consideration of Use ?
No
Yes
(1822–95)
Pasteur’s Quadrant (Stokes)
![Page 19: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/19.jpg)
Mangle of Practice (Pickering)
Mangle 2 |ˈmøŋgəl| |ˈmaŋg(ə)l| noun
A large machine for ironing sheets or other fabrics,
usually when they are damp, using heated rollers.
chiefly Brit. a machine having two or more cylinders turned by a handle, between which wet laundry is squeezed (to
remove excess moisture) and pressed
![Page 20: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/20.jpg)
CognitionPerceptual
ScienceMethods
SocialScienceMethods
Computation and
Visualization Methods
Graphic & Interaction DesignMethods
Cognitionin the wild
Mangle of Visual Analytics?
![Page 21: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/21.jpg)
•Focused on specific situations:
•Understand processes of cognition as supported by interactive visualization
•Understand communication and distribution of cognition via technical means in organizations and communities
•Incorporate these in design and evaluation of technologies for augmenting cognitive processes
Visual Analytics Mangle
![Page 22: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/22.jpg)
•External representations can support
• Increasing the memory and processing resources available to users
•Reducing the search for information by using visual representations to enhance the detection of patterns
•Engaging perceptual inference operations
•Using perceptual attention mechanisms for monitoring
•Supporting manipulation of information
Cognitive Amplification (Card)
![Page 23: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/23.jpg)
•Design guidelines & evaluation methods
•Perceptual cognition
•Reasoning & problem solving
•Social cognition
•Cognitive linguistics
•Cognitive neuroscience
Cognitive Psychology
![Page 24: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/24.jpg)
•Visual expertise - how do we assess, model, teach, and build for it
• Individual differences, quantitative predictions
•Visuomotor expertise - how do we assess, model, teach, and build for it
•Multimodality
Understand cognition supported by interactive
visualization
Cognitive,PerceptualSciencesMethods
![Page 25: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/25.jpg)
•Unified Theories of Cognition (Newell)
• Models = theories
• Models should explain a range of behaviours
• “You can’t play 20 questions with nature and expect to win”
•Fodor “Modularity of Mind”
•Egon Brunswick “Conceptual Framework of Psychology”
Cognitive Architecture
![Page 26: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/26.jpg)
•Pull key unanswered research questions from field work and conduct lab research to answer them.
•The innovation here is in the integration of HCI variables with measures from cognitive/perceptual psychology, hence greater predictive validity for cognitive tasks.
New Methods
![Page 27: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/27.jpg)
•Cognitive processes emerge as a result of a “close coupling” of mental representations & processes with external representations & processes
•All information is equal-- there is “no privileged access” for internal representation to cognitive processes (Gray)
Cognition & Interactivity
![Page 28: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/28.jpg)
•Focus on human-information discourse• Perceptual & motor learning and
expertise• Tight-loop perception/action
coordination• Cognitive processes as supervisory
control• Temporal coordination “Rhythms” of
events & flow of cognitive processing (individual and group)
• Customization for individual’s “personal equation”
• Tracking & training development of interactive expertise
Cognitive augmentation Engelbart
![Page 29: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/29.jpg)
•VR Design Environment (GMR)
•Fishtank VR air traffic control (Hughes/Raytheon)
•Car interfaces (Nissan)
•Aircraft safety (Boeing)
Perception of depth in VR displays (Fechner)Ability to track targets in moving spaces (Pylyshyn - FINST)Perception of touch on the outer legDiscourse in “pair analytics” (JAT)
Enactive Cog VA Projects
![Page 30: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/30.jpg)
•Build on social science (GT, JAT) approaches to understand organizations and cognitive work practices.
•The innovation here is in the extension of social science to bridge to the perceptual and cognitive science theories that apply to the use of visualization in analytical tasks.
Understand technological distribution of cognition
SocialSciences
![Page 31: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/31.jpg)
•Activity theory (Vygotsky via Engeström)
• Multiple human actors and technology
• The activity is the unit of analysis
• Technology is typically passive, e.g. diagrams
•Psycholinguistics Joint Activity theory (Clark)
• Unit of analysis is the structure of joint actions that make up a joint activity
• Avoids metarepresentational infinite recursion
Distributed Cognition
![Page 32: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/32.jpg)
•“Pair analytics” sessions
• Student visual analyst & trained domain expert collaborate on analytic task
• Student “drives”, expert “navigates”
• Video session & capture screen
•Analysis of session based on knowledge of human cognition & communication
• Analysis bridges socsci & cognitive science (Joint Activity Theory & Grounded Theory)
Dr. Richard Arias-Hernández (Postdoc)Daniel Ha (Doctoral Student)Aaron Smith (Doctoral Student)Andrew, Wade (Masters Student)
Distributed Cognition
![Page 33: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/33.jpg)
Engineering Humans & Organizations
•Focus on technologies, but true visual analytics should have impacts on:
• Selection and training of people for perceptual, cognitive, and motor abilities needed to support human-technological cognitive systems
• Structures of organizations & communities that support technological distributions of cognition
![Page 34: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/34.jpg)
•We certainly do not advocate that business schools, in revising MBA curricula, abandon science. Rather, they should encourage and reward research that illuminates the mysteries and ambiguities of today's business practices. Oddly, despite B schools’ scientific emphasis, they do little in the areas of contemporary science that probably hold the greatest promise for business education: cognitive science and neuroscience.
•Bennis & O’Toole, HBR, May 2005
“How Business Schools Lost Their Way”
![Page 35: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/35.jpg)
•Case studies of technologies designed for specific people, situations and cognitive tasks
•Selected theories and methods from cognitive and social sciences that have proven useful
•Aggregate case studies to infer methods that are successful over a class of applications
Cognitive/Social Oriented Informatics
Graphic & Interaction
Design
Computation & Visualization
![Page 36: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/36.jpg)
•Radical DCog would argue equivalence of actors as well as information
•Distribution of agency across human and non-human actors.• Emergent cognition >> summed
cognition of actors• Clark’s Joint Activity theory may
provide a framework for extending DCog theory to complex heterogenous actor networks.
•Scientific approach might address nearly decomposable sub-systems, not individual actors
Cognitive Systems
![Page 37: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/37.jpg)
•System blends human & computational cognitive processes
•Bound together through high-bandwidth interface of vision/visualization for human-information discourse
• Stream processor, many modular subprocessors working in parallel, a learning system with large variation among individuals in methods, capabilities, and time course of processing
• Scalable visual analysis systems, automatic data analysis and interactive visualization for custom-designed processes for the exploration and analysis of complex information spaces.
Cognitive Systems
![Page 38: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/38.jpg)
•Interdisciplinary events
• VAST and EuroVA, workshops
• HICSS - Business/tech
• Cogsci - Psyc, Neuro, AI, Anthro, Ling, Ed...
• Smart Graphics -interdisc. science/art/tech
•Disciplinary
• VSS - Vision Science
• APS - Psychological Science
•Need for a society, journal?
Conferences & workshops
![Page 39: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/39.jpg)
•Visual Analysis of Massive Data for Decision Support and Operational Management
•Submissions due June 15, 2011
•http://www.hicss.hawaii.edu/hicss_45/apahome45.htm
![Page 40: Euro Va2011talk2](https://reader033.vdocument.in/reader033/viewer/2022061202/547c1b58b4af9fb17b8b45c7/html5/thumbnails/40.jpg)
•Proceedings in IEEE digital library, top 2% of IEEE Conferences in IEEE access
•Participants usually from over 40 countries (approximately 50% non-US).
•2nd in citation ranking among 18 Information Systems conferences
•The Australian Government's Excellence in Research project (ERA) has given HICSS an "A" rating, one of 32 Information Systems conferences so honored out of 241