Machine Predictions and Human Decisions with Variation in Payoffs and Skills

Discussion Papers 1911, 42 S.

Michael Allan Ribers, Hannes Ullrich


get_appDownload (PDF  2.99 MB)


Human decision-making differs due to variation in both incentives and available information. This generates substantial challenges for the evaluation of whether and how machine learning predictions can improve decision outcomes. We propose a framework that incorporates machine learning on large-scale administrative data into a choice model featuring heterogeneity in decision maker payoff functions and predictive skill. We apply our framework to the major health policy problem of improving the efficiency in antibiotic prescribing in primary care, one of the leading causes of antibiotic resistance. Our analysis reveals large variation in physicians’ skill to diagnose bacterial infections and in how physicians trade off the externality inherent in antibiotic use against its curative benefit. Counterfactual policy simulations show the combination of machine learning predictions with physician diagnostic skill achieves a 25.4 percent reduction in prescribing.

Michael Ribers

Research Associate in the Firms and Markets Department

Hannes Ullrich

Research Associate in the Firms and Markets Department

JEL-Classification: C10;C55;I11;I18;Q28
Keywords: Prediction policy, expert decision-making, machine learning, antibiotic prescribing