Decision support tools: can they mislead us?

Decision support systems enhanced by Artificial Intelligence (AI) are increasingly being used in high-stakes scenarios where errors or biased outcomes can have significant consequences.In this work, we explore to what extent AIs can increase or decrease human accuracy in a face matching task. As other social animals, humans have evolved an innate capacity to recognize and keep track of other people's faces. However, changes in lighting conditions, face orientation, or the well-known "other-race effect" can reduce our accuracy at this task.We provide various levels of AI support, from a high-accuracy system to a simulation of an AI malfunction in which we suggest to a human annotator the exact opposite of the AI's output.Results indicate that a well-functioning AI can have a positive effect in human performance at this task, but a malfunctioning AI can have a comparatively larger negative effect, particularly in cases where the task difficulty is high. This has important implications for the design and analysis of decision support systems in high-stakes scenarios.