V-FRAMER: Visualization Framework for Mitigating Reasoning Errors in Public Policy
Lily W. Ge, Matthew Easterday, Matthew Kay, Evanthia Dimara, Peter Cheng, and Steven L Franconeri
ACM Human Factors in Computing Systems (CHI) 2024
An example showing how a visualization could still lead to inaccurate conclusions about the data despite the visual components being grammatically correct (i.e., no visual distortions on scales, marks, or channels). To answer the policy-relevant question Did California have a higher risk of COVID compared to Washington around April, 2021?, one needs to consider the Comparison Basis and normalize COVID cases by population (real-world example map B [7, 68]). Not considering the comparison basis could lead to issues such as Missing Normalization (real-world example map A [41]) and result in completely different conclusions about the same data. This example focuses on one section of the V-FRAMER (one-pager in Figure 2).
Abstract
Existing data visualization design guidelines focus primarily on constructing grammatically-correct visualizations that faithfully convey the values and relationships in the underlying data. However, a designer may create a grammatically-correct visualization that still leaves audiences susceptible to reasoning misleaders, e.g. by failing to normalize data or using unrepresentative samples. Reasoning misleaders are especially pernicious when presenting public policy data, where data-driven decisions can affect public health, safety, and economic development. Through textual analysis, a formative evaluation, and iterative design with 19 policy communicators, we construct an actionable visualization design framework, V-FRAMER, that effectively synthesizes ways of mitigating reasoning misleaders. We discuss important design considerations for frameworks like V-FRAMER, including using concrete examples to help designers understand reasoning misleaders, and using a hierarchical structure to support example-based accessing. We further describe V-FRAMER's congruence with current practice and how practitioners might integrate the framework into their existing workflows. Related materials available at: https://osf.io/q3uta/.