Underspecified Human Decision Experiments Considered Harmful
Jessica Hullman, Alex Kale, Jason Hartline
ACM Human Factors in Computing Systems (CHI) 2025
Diagram depicting normative decision for example AI-assisted flight booking scenario. From left to right: The agent is informed of the decision problem, including the action, scoring rule, and prior information about the data-generating model. They next view a signal generated by the data-generating model, which is correlated with the state. The agent updates their beliefs about the state, then chooses the score-maximizing action (in this case, to not book the flight).
Abstract
Decision-making with information displays is a key focus of research in areas like human-AI collaboration and data visualization. However, what constitutes a decision problem, and what is required for an experiment to conclude that decisions are flawed, remain imprecise. We present a widely applicable definition of a decision problem synthesized from statistical decision theory and information economics. We claim that to attribute loss in human performance to bias, an experiment must provide the information that a rational agent would need to identify the normative decision. We evaluate whether recent empirical research on AI-assisted decisions achieves this standard. We find that only 10 (26%) of 39 studies that claim to identify biased behavior presented participants with sufficient information to make this claim in at least one treatment condition. We motivate the value of studying well-defined decision problems by describing a characterization of performance losses they allow to be conceived.