Bridging the Gap
Translating Perception and Cognition Expertise
into Visualization Research and Practice
In principle, perceptual and cognitive science form an important foundation for visualization and visual analytics. However, while some basic principles (e.g. color contrast, pre-attentive popout) have had significant impact on visualization practice, the application of the sciences of perception and cognition has been slow. A simplistic view might expect that the visualization community should be able to mine the extensive literature of perceptual and cognitive science for ideas to influence and improve our work. In practice, such direct translation seems rare. There are many potential causes of this. For example, there may be differences in goals (understanding people vs. making visualizations), differences in methodology (reductionist empiricism vs. wanting realistic applications), etc. In this panel, we propose to explore why the translation of perceptual and cognitive science into visu alization research and practice is so challenging. While it would be unrealistic to expect ready made solutions we are deeply convinced that a panel on this topic structured around the following questions will help fostering research in visualization and visual analytics.
- Is there really a gap between what perception and cognition is providing and what the visualization community needs, or is this just a misconception based on the challenges of communicating across fields?
- What topics would we like to learn from Perception and Cognition Sciences? Can we develop an agenda to work with the respective communities to create a knowledge base that will be most likely to be applicable to drive future Visualization research and practice?
- Given that our goals as a community appear different (which may be a topic for debate), are we doomed to have to develop our own perception and cognition scientific results?
- Can we learn more than just empirical methodology from the perception and cognition community?
- What are the challenges within our community that are making this harder than it should be? What can we do to foster this kind of research? (or shouldnt we bother)
II Position Statements
Visualisation, Perception, Cognition, where does the synergy lie?
We know what we are, but know not what we may be.
Visualization unique feature is that of providing both data presentation and analytical support, with visualization scientists ideally following the evolutionary principle of “maximum benefit with minimum effort” by maximizing the information throughput while minimizing the analytical effort. This poses intrinsically challenging scientific questions such as: what are the factors that mainly determine user performance with visual displays? Does performance result depend only on the type of visual encoding or other variables have a greater impact? How can we define performance as a measure of impact? Answers to such questions clearly depend on a number of factors not necessarily pertaining to the pure data visualization domain: nature of a task, skill level of the users and, more fundamentally, limits set by nature to human perception, attention and cognition. We argue that the field is mature enough for a synergy between Psychology, social sciences and Visualization to be formalized: computer science as a field has shown us the importance of frameworks, psychology and social science the importance of maintaining frameworks abstract nature and flexibility grounded in solid evidence. Last but not least we argue that a deeper understanding of the fundamental structure behind visualization and analytical processing would also support the visualization community in better communicating the value of our field externally.
For a translational cognitive science of interaction
The visualization community should not try to reinvent psychological science as it is represented by organizations like APS, Psychonomics, Cognitive Science, etc. There is over a century of experimental psychology in the West and a millennium of less structured investigation in the East (e.g Ibn Al Haytham, 1024 AD) that we must build upon. We risk spending another century reinventing the wheel. Neither can we count on “off the shelf” psychology research to solve our problems, most psychologists are neither aware of nor interested in the special characteristics of the experiences that we create with our interactive visualization systems. The nature of those environments differs in fundamental ways from the natural environment that experimental psychologists implicitly gear their research and their laboratory and field experiment methods. The solution to this problem lies in the creation of a translational cognitive science of interaction, AKA a “cyberpsychology”. This begins with studies of "cognition in the wild", continues with laboratory studies of the ways in which the novel perceptual and interactive environments that are generated by complex visualization environments affect human perception, cognition, action and collaboration and ends with translational field experiments that integrate theory and methods from lab studies into semi-realistic tasks and settings to inform visualization design choices and evaluation metrics.
Visualization is impatient. Let's embrace that
A better understanding of the human side of the visualization equation, perception and cognition, is a critical foundation for visualization as a science (understanding how to communicate with images) and practice (building systems/displays that do so). However, the goals of visualization are quite different than the goals of traditional perceptual and cognitive science. In both cases, there is a goal of understanding people. But, visualization's specific need for understanding that helps us create better artifacts to be looked at provides a different focus and priorities. Perceptual (and cognitive) science can afford a cautious path of carefully controlled experiments that gradually builds an understanding of how human perception works; in visualization, we are impatient and want results that inform design decisions for challenging situations. Now. Our impatience is valuable: the urgent needs for visualization amongst its potential users drives research and practice. The pressure is on the visualization community to learn to build from the foundations provided by the perceptual and cognitive science communities: we cannot expect them to adapt to our goals and impatience. Impatience also makes it important not to waste time re-inventing their work. As a community, visualization must: learn and embrace their methodology of careful empiricism; learn to extrapolate from phenomena seen in carefully controlled settings and assess the implications in context; be willing to fill holes in the literature for topics that are not their priority; and find ways to make sure that the dialog is valuable both directions.
Capturing the Perceptual Complexity in Visualization
Despite more than three decades of research in data visualization, to a large extent it is not possible to predict the outcome and the benefits of a certain visualization algorithm applied to a particular data set. Even when considering a specific application domain or task, it cannot be determined beforehand to which extent an applied visualization technique will suffice. Although, the knowledge about the human visual system advanced in the last decades, we are far away from having detailed computational models to predict the effectiveness of a visualization when viewed by a human observer. This observation is also underlined when looking back 30 years, when David Marr initially proposed the idea to formulate computational models of visual processing. Despite this intriguing idea and great enthusiasm spanning several research communities, it clearly showed the difficulties related to computational models in this area. While visualization researchers can learn a lot form perception researchers, also severe differences in the design and conduction of the studies exist. First, the simplicity of the visual stimuli investigated in perceptual research calls for new stimulus generation concepts in the visualization community. Second, visualization applications are usually not applied in a controlled environment, as it is usually the basis for experimental perception research. Third, when dealing with complex stimuli the influence of perception and cognition blurs, as these two factors are hard to separate. As a technology-driven community we could tackle this complexity by using technology. We could ask ourselves how we can benefit from data, in a similar way as modern biology is making sense of the complexity of living processes through high-throughput data acquisition technologies? While the perception community has already developed several frequently used technologies for study conduction, it is remarkable that such efforts are rare in the technology-affine data visualization community.
The Process of Thinking Visually: Why we need design patterns
The science of perception and cognition can inform visualization design in many ways, but it can rarely prescribe design because of the complexity of most visual analytic tasks. When we use interactive visualizations to analyze data we participate in a cognitive system, part of the thinking process happens in our heads and part in a computer, taking advantage of thinking processes captured in computer programs. The visualization is an internal interface in this human-computer system. Understanding perception can help designers make visualization that map data to display so that important patterns are seen. Understanding visual cognition can designers understand key bottlenecks, such as visual working memory this can help us design efficient interactive systems. But design is complex. Science cannot provide the answers only inform the designer. Empirical studies relating to portrayal and interaction must be linked to theory in order to make the problem tractable. Visual Thinking Design Patterns, can be a tool that allows designers to reason about the cognitive system.
Organizer and panelists are listed in alphabetical order.
Georges-Pierre Bonneau is a professor of Computer Science at the University of Grenoble. He is a member of the research group Maverick within INRIA. GP Bonneau received his MS degree in Mathematics from the Ecole Normale Superieure de Cachan, France, the DEA in Numerical Analysis at the University of Paris VI, France, and the PhD degree in Computer Science from the University of Kaiserslautern, Germany, in 1993. His research centres on Scientific Visualization, Computer Aided Geometric Design and Visual perception for Computer graphics. He has been co-organizer of two Dagstuhl seminars on Scientific Visualization and co-chair of a EuroVis Conference. He is a past member of the Editorial Board Committee of IEEE Transactions on Visualization and Computer Graphics.
Rita Borgo is a Lecturer in the Department of Computer Science, Swansea University, UK. She received her BSc and MSc (Laurea with commendation) from University of Bologna, IT, and obtained her PhD. in Computer Science from University of Pisa, IT. Her research interests include scientific visualization, information visualization, and visual analytics, human factors in visualization, multimedia processing and visualization. She has been a visiting researcher at several prestigious institutions including the Italian national Research Council and the Lawrence Livermore National Laboratory (US). She is a member of BCS Women in Computer Science and IEEE Computer Society.
Brian Fisher: A psychologist with a Erdos number of 2, Brian is a Fellow of the Psychonomics Society, member of the VAST steering committee, VIS Executive Committee, and Visualization Pioneers group. By day he is Associate Professor in the School of Interactive Arts and Technology and Program in Cognitive Science at Simon Fraser University. At the University of British Columbia he is Associate Director of the Media And Graphics Interdisciplinary Centre, and member of the Brain Research Centre and Institute for Computing, Information and Cognitive Systems. His research includes laboratory studies of human cognitive architecture and individual differences, and applications in emergency management, aircraft safety, and public health. In collaboration with VIS colleagues he has presented symposia on visual analytics to the Cognitive Science Society, Association for Psychological Science, and the Psychonomics society with the goal of building more effective interdisciplinary research in visual information systems and their use in analysis, collaboration, and interpersonal communication.
Michael Gleicher is a Professor in the Department of Computer Sciences at the University of Wisconsin, Madison. Prof. Gleicher is founder of the Department's Visual Computing Group. His research interests span the range of visual computing, including data visualization, image and video processing tools, virtual reality, and character animation techniques for films, games and robotics. Prior to joining the university, Prof. Gleicher was a researcher at The Autodesk Vision Technology Center and in Apple Computer's Advanced Technology Group. He earned his Ph. D. in Computer Science from Carnegie Mellon University, and holds a B.S.E. in Electrical Engineering from Duke University. For the 2013-2014 academic year, he was a visiting researcher at INRIA Rhone-Alpes. Prof. Gleicher is an ACM Distinguished Scientist.
Timo Ropinski is a professor in interactive visualization at Link\"oping University, Sweden. He holds a PhD in computer science from the University of Muenster, where he also finished his Habilitation. Today he is coordinating the visualization community within the Swedish e-Science Research Centre (SeRC), which is a cooperation between leading Swedish Universities and the National Super Computing Centers. His research interests lie in the areas of interactive visual problem solving, whereby he focuses on the algorithms behind visualization systems, as well as the combination of perceptual aspects and the possibilities provided by modern computing systems. The results of his scientific work have been published in various international journals and conferences including IEEE TVCG, IEEE Visualization, Eurographics, IEEE VR, and others. Furthermore, he serves regularly on the IPC of various international conferences in the field and has held and organized tutorials at Eurographics, SIGGRAPH and IEEE Visualization.
Colin Ware is the Director of the Data Visualization Research Lab which is part of the Center for Coastal and Ocean Mapping at the University of New Hampshire. He combines interests in both basic and applied research and he has advanced degrees in both computer science (MMath, Waterloo) and in the psychology of perception (PhD,Toronto). He is cross appointed between the Departments of Ocean Engineering and Computer Science. Ware specializes in advanced data visualization and has a special interest in applications of visualization to Ocean Mapping. Ware has published over 160 articles in scientific and technical journals and leading conference proceedings. Many of these articles relate to the use of color, texture, motion and 3D displays in information visualization. His approach is always to combine theory with practice and his publications range from rigorously scientific contributions to the Journal of Physiology and Vision Research to applications-oriented articles in ACM Transactions on Graphics and various visualization and human-computer Interaction Journals. Colin Ware's book Information Visualization: Perception for Design 2012 3rd Edition is a standard reference on what the science of perception can tell us about visualization design. His other book: Visual Thinking for Design is an up to date account of the psychology of how we think using graphic displays as tools.