More Forecasts, More (Decision) Problems: How Uncertainty Representations for Multiple Forecasts Impact Decision-Making
Abhraneel Sarma, Maryam Hedayati, and Matthew Kay
ACM Human Factors in Computing Systems (CHI) 2025
Illustration of the stimuli used in the two experiments. Both experiments consisted of two blocks. The first block in both experiments required participants to perform the binary decision making task based on a single forecast, visualised as CDFs. In block 2 of experiment 1, participants were presented with either p-boxes or ensembles. In block 2 of experiment 2, participants were presented with either ensembles skewed right or skewed left. The actual stimuli used in the experiment can be found in supplement ▶ stimuli.
Abstract
Users often have access to multiple forecasts regarding an event. Different forecasts incorporate different assumptions and epistemic information. A growing body of work argues against decisionmaking solely based on expected utility maximisation strategies in multiple forecasts scenarios, in favour of other strategies such as the maximin expected utility. In this work, we compare two different approaches for depicting epistemic uncertainty—ensembles (a direct representation of multiple forecasts) and p-boxes (a representation which only communicates the bounds of epistemic uncertainty)—in plots where individual distributions are represented as cumulative distribution plots (CDFs). We conduct three experiments to investigate the impact of the visual representation on the decision-making strategies that people adopt. Our results suggest that participants adopt conservative decision-making strategies (i.e. place greater weight on the worst-case forecast than the best-case forecast) for both p-boxes and ensembles if the set of forecasts are uniformly distributed. However, if a majority of the forecasts are clustered near one of the bounds, participants may discount the forecast which appears as a visual outlier.