Papers – Mu Collective

Papers

2024

Erie: A Declarative Grammar for Data Sonification

Hyeok Kim, Yea-Seul Kim, and Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Watching the Election Sausage Get Made: How Data Journalists Visualize the Vote Counting Process in U.S. Elections

Mandi Cai and Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

V-FRAMER: Visualization Framework for Mitigating Reasoning Errors in Public Policy

Lily W. Ge, Matthew Easterday, Matthew Kay, Evanthia Dimara, Peter Cheng, and Steven L Franconeri

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

Dongping Zhang, Angelos Chatzimparmpas, Negar Kamali, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Milliways: Taming Multiverses through Principled Evaluation of Data Analysis Paths

Abhraneel Sarma, Kyle Hwang, Jessica Hullman and Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

To Cut or Not To Cut: A Systematic Exploration of Y-Axis Truncation

Sheng Long, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Odds and Insights: Decision Quality in Visual Analytics Under Uncertainty

Abhraneel Sarma, Xiaoying Pu, Yuan Cui, Eli T Brown, Michael Correll and Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2024 | PDF

Designing Shared Information Displays for Agents of Varying Strategic Sophistication

Dongping Zhang, Jason Hartline, Jessica Hullman

ACM SIGCHI Conference on Computer-Supported Cooperative Work & Social Computing (CSCW) 2024 | PDF

2023

Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms

Fumeng Yang, Mandi Cai, Chloe Mortenson, Hoda Fakhari, Ayse D. Lokmanoglu, Jessica Hullman, Steven Franconeri, Nicholas Diakopoulos, Erik C. Nisbet, Matthew Kay

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | BEST PAPER AWARD | PDF

Adaptive Assessment of Visualization Literacy

Yuan Cui, Lily W. Ge, Yiren Ding, Fumeng Yang, Lane Harrison, Matthew Kay

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

ggdist: Visualizations of Distributions and Uncertainty in the Grammar of Graphics

Matthew Kay

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

Dupo: A Mixed-Initiative Authoring Tool for Responsive Visualization

Hyeok Kim, Ryan Rossi, Jessica Hullman, Jane Hoffswell

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

The Rational Agent Benchmark for Data Visualization

Yifan Wu, Ziyang Guo, Michails Mamakos, Jason Hartline, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

EVM: Incorporating Model Checking into Exploratory Visual Analysis

Alex Kale, Ziyang Guo, Xiao Li Qiao, Jeffrey Heer, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research

Hariharan Subramonyam, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

Causal Quartets: Different Ways to Obtain the Same Average Treatment Effect

Andrew Gelman, Jessica Hullman, Lauren Kennedy

American Statistician | PDF

CALVI: Critical Thinking Assessment for Literacy in Visualizations

Lily W. Ge, Yuan Cui, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2023 | BEST PAPER HONORABLE MENTION | PDF

multiverse: Multiplexing Alternative Data Analyses in R Notebooks

Abhraneel Sarma, Alex Kale, Michael Moon, Nathan Taback, Fanny Chevalier, Jessica Hullman, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2023 | BEST PAPER HONORABLE MENTION | PDF

How Data Analysts Use a Visualization Grammar in Practice

Xiaoying Pu, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2023 | PDF

MetaExplorer: Facilitating Reasoning with Epistemic Uncertainty in Meta-analysis

Alex Kale, Sarah Lee, Terrance Goan, Elizabeth Tipton, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2023 | PDF | GITHUB

Subjective Probability Correction for Uncertainty Representations

Fumeng Yang, Maryam Hedayati, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2023 | BEST PAPER HONORABLE MENTION | PDF

2022

What's Driving Conflicts Around Differential Privacy for the US Census

Priyanka Nanayakkara, Jessica Hullman

IEEE Security & Privacy Magazine | PDF

paper-thumb-perception-individual-differences

The Risks of Ranking: Revisiting Graphical Perception to Model Individual Differences in Visualization Performance

Russell Davis, Xiaoying Pu, Yiren Ding, Brian D. Hall, Karen Bonilla, Mi Feng, Matthew Kay, Lane Harrison

Russell Davis, Xiaoying Pu, Yiren Ding, Brian D. Hall, Karen Bonilla, Mi Feng, Matthew Kay, and Lane Harrison

IEEE Transactions on Visualization and Computer Graphics | PDF

paper-thumb-uncertainty-vis-for-imputations

Evaluating the Use of Uncertainty Visualisations for Imputations of Data Missing At Random in Scatterplots

Abhraneel Sarma, Shunan Guo, Jane Hoffswell, Ryan Rossi, Fan Du, Eunyee Koh, and Matthew Kay

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2022 | PDF

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning

Jessica Hullman, Sayash Kapoor, Priyanka Nanayakkara, Andrew Gelman, Arvind Narayanan

AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society | PDF

Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews

David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad

AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society | PDF

A Survey of Tasks and Visualizations in Multiverse Analysis Reports

Brian D. Hall, Yang Liu, Yvonne Jansen, Pierre Dragicevic, Fanny Chevalier, Matthew Kay

Computer Graphics Forum | PDF

Cicero: A Declarative Grammar for Responsive Visualization

Hyeok Kim, Ryan Rossi, Fan Du, Eunyee Koh, Shunan Guo, Jessica Hullman, Jane Hoffswell

ACM Human Factors in Computing Systems (CHI) 2022 | PDF

ACM Human Factors in Computing Systems (CHI) 2022 | PDF | OSF

Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases

Priyanka Nanayakkara, Johes Bater, Xi He, Jessica Hullman, and Jennie Rogers

Proceedings on Privacy Enhancing Technologies 2022 | PDF

2021

Designing for interactive exploratory data analysis requires theories of graphical inference

Jessica Hullman and Andrew Gelman

Harvard Data Science Review 2021 | PDF

Visualization Equilibrium

Paula Kayongo, Glenn Sun, Jason Hartline, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

Visualizing Uncertainty in Probabilistic Graphs with Network Hypothetical Outcome Plots (NetHOPs)

Dongping Zhang, Eytan Adar, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

Causal Support: Modeling Causal Inferences with Visualizations

Alex Kale, Yifan Wu, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

An Automated Approach to Reasoning About Task-Oriented Insights in Responsive Visualization

Hyeok Kim, Ryan Rossi, Abhraneel Sarma, Dominik Moritz, and Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

Unpacking the Expressed Consequences of AI Research in Broader Impact Statements

Priyanka Nanayakkara, Jessica Hullman, Nicholas Diakopoulos

AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society 2021 | PDF

Design Patterns and Trade-Offs in Responsive Visualization for Communication

Hyeok Kim, Dominik Moritz, Jessica Hullman

Computer Graphics Forum (Proc. EUROVIS 2021) | PDF

2020

Information, incentives, and goals in election forecasts

Andrew Gelman, Jessica Hullman, Christopher Wlezien, George Elliott Morris

Judgment and Decision Making 2020 | PDF

Revealing Perceptual Proxies with Adversarial Examples

Brian D. Ondov, Fumeng Yang, Matthew Kay, Niklas Elmqvist, Steven Franconeri

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | PDF

Bayesian-Assisted Inference from Visualized Data

Bayesian-Assisted Inference from Visualized Data

Bayesian-Assisted Inference from Visualized Data

Yea-Seul Kim, Paula Kayongo, Madeleine Grunde-McLaughlin, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | PDF

Visual Reasoning Strategies for Effect Size Judgments and Decisions

Alex Kale, Matthew Kay, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | BEST PAPER AWARD | PDF | GITHUB

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs

Sungsoo Ray Hong, Jessica Hullman, Enrico Bertini

ACM Computer Supported Coopoerative Work (CSCW) 2020 | PDF

Prior Setting in Practice: Strategies and Rationales Used in Choosing Prior Distributions for Bayesian Analysis

Prior Setting in Practice: Strategies and Rationales Used in Choosing Prior Distributions for Bayesian Analysis

Abhraneel Sarma, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2020 | PDF | GITHUB

A Probabilistic Grammar of Graphics

How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results

A Probabilistic Grammar of Graphics

Xiaoying Pu, Matthew Kay

Jake M. Hoffman, Daniel G. Goldstein, Jessica Hullman

Xiaoying Pu, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2020 | BEST PAPER HONORABLE MENTION | PDF

How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results

How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results

Jake M. Hofman, Daniel G. Goldstein, Jessica Hullman

Jake M. Hoffman, Daniel G. Goldstein, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2020 | BEST PAPER HONORABLE MENTION | PDF

Exploring the Effects of Aggregation Choices on Untrained Visualization Users’ Generalizations from Data

Exploring the Effects of Aggregation Choices on Untrained Visualization Users’ Generalizations from Data

Francis Nguyen, Xiaoli Qiao, Jeffrey Heer, Jessica Hullman

Computer Graphics Forum 2020 | PDF | GITHUB

2019

Why Authors Don't Visualize Uncertainty

Why Authors Don't Visualize Uncertainty

Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2019 | PDF

Illusion of Causality in Visualized Data

Illusion of Causality in Visualized Data

Cindy Xiong, Joel Shapiro, Jessica Hullman, Steve Franconeri

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2019 | PDF

Increasing the Transparency of Research Papers with Explorable Multiverse Analyses

Increasing the Transparency of Research Papers with Explorable Multiverse Analyses

Pierre Dragicevic, Yvonne Jansen, Abhraneel Sarma, Matthew Kay, and Fanny Chevalier

ACM Human Factors in Computing Systems (CHI) 2019 | BEST PAPER AWARD | PDF | WEBSITE | GITHUB

A Bayesian Cognition Approach to Improve Data Visualization

A Bayesian Cognition pproach to Improve Data Visualization

Yea-Seul Kim, Logan A Walls, Peter Krafft, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GITHUB

Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths

Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths

Alex Kale, Matthew Kay, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GITHUB

Vocal Shortcuts for Creative Experts

Vocal Shortcuts for Creative Experts

Yea-Seul Kim, Mira Dontcheva, Eytan Adar, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2019 | PDF

Some Prior(s) Experience Necessary

Some Prior(s) Experience Necessary

Chanda Phelan, Jessica Hullman, Matthew Kay, Paul Resnick

ACM Human Factors in Computing Systems (CHI) 2019 | PDF

EV-tootltip-line-diagrams_voting_social copy

Belief-Driven Data Journalism

Belief-Driven Data Journalism

Francis Nguyen, Samana Shrestha, Joe Germuska, Yea-Seul Kim, Jessica Hullman

Computation + Journalism Symposium 2019 | PDF | MEDIUM POST | CODE

2018

Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data

Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data

Alex Kale, Francis Nguyen, Matthew Kay, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2018 | PDF | GITHUB

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation

Jessica Hullman, Xiaoli Qiao, Michael Correll, Alex Kale, Matthew Kay

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2018 | PDF | GITHUB | WEBSITE

The Garden of Forking Paths in Visualization: A Design Space for Reliable Exploratory Visual Analytics

Xiaoying Pu, Matthew Kay

BELIV 2018 | PDF

Improving Comprehension of Measurements Using Concrete Re-expression Strategies

Jessica Hullman, Yea-Seul Kim, Francis Nguyen, Lauren Speers, Maneesh Agrawala

ACM Human Factors in Computing Systems (CHI) 2018 | PDF | WEBSITE | GITHUB

Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, Matthew Kay

ACM Human Factors in Computing Systems (CHI) 2018 | BEST PAPER HONORABLE MENTION | PDF

2017

Keeping Multiple Views Consistent: Constraints, Validations, and Exceptions in Visualization Authoring

Zening Qu, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2017 | BEST PAPER HONORABLE MENTION | PDF

Data Through Others' Eyes: The Impact of Visualizing Others' Expectations on Visualization Interpretation

Yea-Seul Kim, Katharina Reinecke, Jessica Hullman

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2017 | PDF

Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty

Jessica Hullman, Matthew Kay, Yea-Seul Kim, Samana Shrestha

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2017 | PDF | SUPPLEMENTAL MATERIALS

Finding a Clear Path: Structuring Strategies for Visualization Sequences

Jessica Hullman, Robert Kosara, Heidi Lam

Computer Graphics Forum (Proc. EUROVIS) 2017 | PDF

GraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing

Younghoon Kim, Kanit Wongsuphasawat, Jessica Hullman, Jeffrey Heer

ACM Human Factors in Computing Systems (CHI) 2017 | BEST PAPER HONORABLE MENTION | PDF | GITHUB | SUPPLEMENTAL MATERIALS

Explaining the Gap: Visualizing One's Predictions Improves Recall and Comprehension of Data

Yea-Seul Kim, Katharina Reinecke, Jessica Hullman

ACM Human Factors in Computing Systems (CHI) 2017 | BEST PAPER AWARD | PDF | MEDIUM POST

2016

Why Evaluating Uncertainty Visualization is Error Prone

Jessica Hullman

BELIV 2016 | PDF

Evaluating Visualization Sets: Trade-offs Between Local Effectiveness and Global Consistency

Zening Qu, Jessica Hullman

BELIV 2016 | PDF

Researcher-centered design of statistics: Why Bayesian statistics better fit the culture and incentives of HCI

Matthew Kay, Gregory Nelson, Eric Hekler

ACM Human Factors in Computing Systems (CHI) 2016 | BEST PAPER HONORABLE MENTION | PDF | GITHUB

Generating Personalized Spatial Analogies for Distances and Areas

Yea-Seul Kim, Jessica Hullman, Maneesh Agrawala

ACM Human Factors in Computing Systems (CHI) 2016 | PDF | SUPPLEMENTAL MATERIALS | WEBSITE | BLOG

When(ish) is My Bus? User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

Matthew Kay, Tara Kola, Jessica Hullman, Sean Munson

ACM Human Factors in Computing Systems (CHI) 2016 | PDF | GITHUB

2015

Content, Context, and Critique: Commenting on a Data Visualization Blog

Jessica Hullman, Nicholas Diakopoulos, Elaheh Momeni Roochi, Eytan Adar

ACM Computer-Supported Cooperative Work (CSCW) 2015 | PDF

Beyond Weber’s Law: A Second Look at Ranking Visualizations of Correlation

Matthew Kay, Jeffery Heer

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2015 | BEST PAPER HONORABLE MENTION | PDF | GITHUB

How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy

Matthew Kay, Shwetak N. Patel, Julie A. Kientz

ACM Human Factors in Computing Systems (CHI) 2015 | PDF | GITHUB

Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering

Jessica Hullman, Paul Resnick, Eytan Adar

PLOS ONE 2015 | PDF

2014 — Earlier

NewsViews: An Automated Pipeline for Creating Custom Geovisualizations for News

Tong Gao, Jessica Hullman, Eytan Adar, Brent Hecht, Nicholas Diakopoulos

ACM Human Factors in Computing Systems (CHI) 2014 | PDF

There’s No Such Thing As Gaining a Pound: Reconsidering the Bathroom Scale User Interface

Matthew Kay, Dan Morris, mc schraefel, Julie A. Kientz

ACM SIGCHI UbiComp 2013 | BEST PAPER AWARD | PDF

A Deeper Understanding of Sequence in Narrative Visualization

Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, Eytan Adar

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2013 | PDF

Contextifier: Automatic Generation of Annotated Stock Visualizations

Jessica Hullman, Nicholas Diakopoulos, Eytan Adar

ACM Human Factors in Computing Systems (CHI) 2013 | PDF

Benefitting InfoVis with Visual Difficulties

Jessica Hullman, Eytan Adar, Priti Shah

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2011 | BEST PAPER HONORABLE MENTION | PDF

Visualization Rhetoric: Framing Effects in Narrative Visualization

Jessica Hullman, Nicholas Diakopoulos

IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2011 | PDF

The Impact of Social Information on Visual Judgments

Jessica Hullman, Eytan Adar, Priti Shah

ACM Human Factors in Computing Systems (CHI) 2011 | PDF