Midwest Uncertainty Collective

Midwest Uncertainty Collective

 

Midwest Uncertainty Collective

 

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matthew Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

The MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

We are a research lab working at the intersection of visualization and data-driven inference and prediction. Our mission is to empower people to learn from data by developing visual representations, approaches to uncertainty quantification, and human-in-the-loop tools that align with downstream goals and how people think. Topics we like include sampling-oriented uncertainty visualizations, visualization literacy, trust and personalization of predictions, usable statistics, and Bayesian inference.

MU Collective is directed by Jessica Hullman and Matt Kay of Northwestern University.

ezgif.com-crop
home-page-grid-image-matt-2
quantileDotplot
hops-video

Recent Papers | See all

Recent Papers | See all

VILA

Promises and Pitfalls: Using Large Language Models to Generate Visualization Items
Yuan Cui, Lily W. Ge, Yiren Ding, Lane Harrison, Fumeng Yang, Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2024 | PDF

visualization literacy interview study

What University Students Learn In Visualization Classes
Maryam Hedayati and Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2024 | PDF

backstory

The Backstory to “Swaying the Public”: A Design Chronicle of Election Forecast Visualizations
Fumeng Yang, Mandi Cai, Chloe Mortenson, Hoda Fakhari, Ayse D. Lokmanoglu, Nicholas Diakopoulos, Erik C. Nisbet, Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2024 | PDF

VMC: A Grammar for Visualizing Statistical Model Checks

VMC: A Grammar for Visualizing Statistical Model Checks
Ziyang Guo, Alex Kale, Matthew Kay, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2024 | PDF | Package | Documentation

visual_lineups

Old Wine in a New Bottle? Analysis of Visual Lineups with Signal Detection Theory
Sheng Long, Matthew Kay
IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) 2024 | PDF

barriers_to_vis_experiments

Tasks and Telephones: Threats to Experimental Validity due to Misunderstandings of Visualisation Tasks and Strategies
Abhraneel Sarma, Sheng Long, Michael Correll, Matthew Kay
IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) 2024 | PDF

AI Reliance

A Decision Theoretic Framework for Measuring AI Reliance
Ziyang Guo, Yifan Wu, Jason Hartline, Jessica Hullman
ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT ’24) | PDF

Measure-Observe-Remeasure

Measure-Observe-Remeasure: An Interactive Paradigm for Differentially-Private Exploratory Analysis
Priyanka Nanayakkara, Hyeok Kim, Yifan Wu, Ali Sarvghad, Narges Mahyar, Gerome Miklau, Jessica Hullman
IEEE Symposium on Security & Privacy (SP) | PDF

Erie

Erie: A Declarative Grammar for Data Sonification
Hyeok Kim, Yea-Seul Kim, and Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2024 | PDF | Package | Documentation | Online editor

vote counting process

Watching the Election Sausage Get Made: How Data Journalists Visualize the Vote Counting Process in U.S. Elections
Mandi Cai and Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2024 |  BEST PAPER HONORABLE MENTION | PDF

V-FRAMER

V-FRAMER: Visualization Framework for Mitigating Reasoning Errors in Public Policy
Lily W. Ge, Matthew Easterday, Matthew Kay, Evanthia Dimara, Peter Cheng, and Steven L Franconeri
ACM Human Factors in Computing Systems (CHI) 2024 | PDF

In Dice We Trust

In Dice We Trust: Uncertainty Displays for Maintaining Trust in Election Forecasts Over Time
Fumeng Yang, Chloe Mortenson, Erik C. Nisbet, Nicholas Diakopoulos, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2024 | BEST PAPER AWARD | PDF

Conformal Prediction Sets

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling
Dongping Zhang, Angelos Chatzimparmpas, Negar Kamali, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2024 |  BEST PAPER HONORABLE MENTION | PDF

milliways

Milliways: Taming Multiverses through Principled Evaluation of Data Analysis Paths
Abhraneel Sarma, Kyle Hwang, Jessica Hullman and Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2024 | PDF

AI bridged CLA

Authors’ Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts
Taewook Kim, Hyomin Han, Eytan Adar, Matthew Kay, John Joon Young Chung
ACM Human Factors in Computing Systems (CHI) 2024 |  BEST PAPER HONORABLE MENTION | PDF

to cut or not to cut

To Cut or Not To Cut: A Systematic Exploration of Y-Axis Truncation
Sheng Long, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2024 | PDF

odds and insights

Odds and Insights: Decision Quality in Visual Analytics Under Uncertainty
Abhraneel Sarma, Xiaoying Pu, Yuan Cui, Eli T Brown, Michael Correll and Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2024 |  BEST PAPER HONORABLE MENTION | PDF

dementia

Opportunities in Mental Health Support for Informal Dementia Caregivers Suffering from Verbal Agitation
Taewook Kim, Hyeok Kim, Angela C. Roberts, Maia Jacobs, Matthew Kay
ACM Computer-Supported Cooperative Work (CSCW) 2024 | PDF

shared_info_displays

Designing Shared Information Displays for Agents of Varying Strategic Sophistication
Dongping Zhang, Jason Hartline, Jessica Hullman
ACM Computer-Supported Cooperative Work (CSCW) 2024 | PDF

US_midterms

Swaying the Public? Impacts of Election Forecast Visualizations on Emotion, Trust, and Intention in the 2022 U.S. Midterms
Fumeng Yang, Mandi Cai, Chloe Mortenson, Hoda Fakhari, Ayse D. Lokmanoglu, Jessica Hullman, Steven Franconeri, Nicholas Diakopoulos, Erik C. Nisbet, Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | BEST PAPER AWARD | PDF

adaptive assessments of visualization literacy

Adaptive Assessment of Visualization Literacy
Yuan Cui, Lily W. Ge, Yiren Ding, Fumeng Yang, Lane Harrison, Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

ggdist

ggdist: Visualizations of Distributions and Uncertainty in the Grammar of Graphics
Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

Dupo

Dupo: A Mixed-Initiative Authoring Tool for Responsive Visualization
Hyeok Kim, Ryan Rossi, Jessica Hullman, Jane Hoffswell
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

rational_agent

The Rational Agent Benchmark for Data Visualization
Yifan Wu, Ziyang Guo, Michails Mamakos, Jason Hartline, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

EVM

EVM: Incorporating Model Checking into Exploratory Visual Analysis
Alex Kale, Ziyang Guo, Xiao Li Qiao, Jeffrey Heer, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

vis4ml_gaps

Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research
Hariharan Subramonyam, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2023 | PDF

causal_quartets

Causal Quartets: Different Ways to Obtain the Same Average Treatment Effect
Andrew Gelman, Jessica Hullman, Lauren Kennedy
American Statistician | PDF

calvi

CALVI: Critical Thinking Assessment for Literacy in Visualizations
Lily W. Ge, Yuan Cui, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2023 |  BEST PAPER HONORABLE MENTION | PDF

multiverse

multiverse: Multiplexing Alternative Data Analyses in R Notebooks
Abhraneel Sarma, Alex Kale, Michael Moon, Nathan Taback, Fanny Chevalier, Jessica Hullman, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2023 |  BEST PAPER HONORABLE MENTION | PDF

vis-grammar-in-practice

How Data Analysts Use a Visualization Grammar in Practice
Xiaoying Pu, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2023 | PDF

metaexplorer

MetaExplorer: Facilitating Reasoning with Epistemic Uncertainty in Meta-analysis
Alex Kale, Sarah Lee, Terrance Goan, Elizabeth Tipton, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2023 | PDF | GITHUB

subjective-probability

Subjective Probability Correction for Uncertainty Representations
Fumeng Yang, Maryam Hedayati, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2023 |  BEST PAPER HONORABLE MENTION | PDF

dp-census

What’s Driving Conflicts Around Differential Privacy for the U.S. Census
Priyanka Nanayakkara, Jessica Hullman
IEEE Security & Privacy Magazine | PDF

perception-individual-differences

The Risks of Ranking: Revisiting Graphical Perception to Model Individual Differences in Visualization Performance
Russell Davis, Xiaoying Pu, Yiren Ding, Brian D. Hall, Karen Bonilla, Mi Feng, Matthew Kay, and Lane Harrison
IEEE Transactions on Visualization and Computer Graphics | PDF

uncertainty-vis-for-imputations

Evaluating the Use of Uncertainty Visualisations for Imputations of Data Missing At Random in Scatterplots
Abhraneel Sarma, Shunan Guo, Jane Hoffswell, Ryan Rossi, Fan Du, Eunyee Koh, and Matthew Kay
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2022 | PDF

learning-errors-psych-ml

The Worst of Both Worlds: A Comparative Analysis of Errors in Learning from Data in Psychology and Machine Learning
Jessica Hullman, Sayash Kapoor, Priyanka Nanayakkara, Andrew Gelman, Arvind Narayanan
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society | PDF

examining-responsibility-and-deliberation

Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews
David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society | PDF

survey-multiverse-analysis-reports

A Survey of Tasks and Visualizations in Multiverse Analysis Reports
Brian D. Hall, Yang Liu, Yvonne Jansen, Pierre Dragicevic, Fanny Chevalier, Matthew Kay
Computer Graphics Forum | PDF

cicero

Cicero: A Declarative Grammar for Responsive Visualization
Hyeok Kim, Ryan Rossi, Fan Du, Eunyee Koh, Shunan Guo, Jessica Hullman, Jane Hoffswell
ACM Human Factors in Computing Systems (CHI) 2022 | PDF

visualizing-privacy

Visualizing Privacy-Utility Trade-Offs in Differentially Private Data Releases
Priyanka Nanayakkara, Johes Bater, Xi He, Jessica Hullman, and Jennie Rogers
Proceedings on Privacy Enhancing Technologies 2022 | PDF

belief-update

Designing for interactive exploratory data analysis requires theories of graphical inference
Jessica Hullman, Andrew Gelman
Harvard Data Science Review 2021 | PDF

belief-update

Unpacking the Expressed Consequences of AI Research in Broader Impact Statements
Priyanka Nanayakkara, Jessica Hullman, Nicholas Diakopoulos
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society 2021 | PDF

belief-update

Visualization Equilibrium
Paula Kayongo, Glenn Sun, Jason Hartline, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

belief-update

Visualizing Uncertainty in Probabilistic Graphs with Network Hypothetical Outcome Plots (NetHOPs)
Dongping Zhang, Eytan Adar, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

belief-update

Causal Support: Modeling Causal Inferences with Visualizations
Alex Kale, Yifan Wu, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

belief-update

An Automated Approach to Reasoning About Task-Oriented Insights in Responsive Visualization
Hyeok Kim, Ryan Rossi, Abhraneel Sarma, Dominik Moritz, and Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. VIS) 2021 | PDF

belief-update

Design Patterns and Trade-Offs in Responsive Visualizationfor Communication
Hyeok Kim, Dominik Moritz, Jessica Hullman
EUROVIS 2021 | PDF

belief-update

Information, incentives, and goals in election forecasts
Andrew Gelman, Jessica Hullman, Christopher Wlezien, George Elliott Morris
Judgment and Decision Making 2020 | PDF

belief-update

Revealing Perceptual Proxies with Adversarial Examples
Brian D. Ondov, Fumeng Yang, Matthew Kay, Niklas Elmqvist, Steven Franconeri
IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | PDF

belief-update

Bayesian-Assisted Inference from Visualized Data
Yea-Seul Kim, Paula Kayongo, Madeleine Grunde-McLaughlin, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | PDF

vis-reasoning

Visual Reasoning Strategies for Effect Size Judgments and Decisions
Alex Kale, Matthew Kay, Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2020 | BEST PAPER AWARD | PDF

cscw-intertability

Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
Sungsoo Ray Hong, Jessica Hullman, Enrico Bertini
ACM Computer Supported Coopoerative Work (CSCW) 2020 | PDF

visualiing-inferential-uncertainty

How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results
Jake M. Hofman, Dan Goldstein, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2020 |  BEST PAPER HONORABLE MENTION |  PDF

uncertainty-bus

Prior Setting in Practice: Strategies and Rationales Used in Choosing Prior Distributions for Bayesian Analysis
Abhraneel Sarma, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2020 | PDF | GITHUB

uncertainty-bus

A Probabilistic Grammar of Graphics
Xiaoying Pu, Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2020 |  BEST PAPER HONORABLE MENTION |  PDF | GITHUB

aggregation effects

Exploring the Effects of Aggregation Choices on Untrained Visualization Users’ Generalizations from Data
Francis Nguyen, Xiaoli Qiao, Jeffrey Heer, Jessica Hullman
Computer Graphics Forum 2020 | PDF |   GITHUB

Value Uncertainty Thumbnail

Why Authors Don't Visualize Uncertainty
Jessica Hullman
IEEE Trans. Visualization & Comp. Graphics (Proc. INFOVIS) 2019 |  PDF

bayesian-vis

Increasing the Transparency of Research Papers with Explorable Multiverse Analyses
Pierre Dragicevic, Yvonne Jansen, Abhraneel Sarma, Matthew Kay, and Fanny Chevalier
ACM Human Factors in Computing Systems (CHI) 2019 |  BEST PAPER AWARD  | PDF | WEBSITE | GITHUB

bayesian-vis

A Bayesian Cognition Approach to Improve Data Visualization
Yea-Seul Kim, Logan A Walls, Peter Krafft, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GITHUB

analysis-paths-research-synthesis

Decision-Making Under Uncertainty in Research Synthesis: Designing for the Garden of Forking Paths
Alex Kale, Matthew Kay, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2019 | PDF | GITHUB

voice-cuts

Vocal Shortcuts for Creative Experts
Yea-Seul Kim, Mira Dontcheva, Eytan Adar, Jessica Hullman
ACM Human Factors in Computing Systems (CHI) 2019 | PDF