Stakeholder-dependent views on biases of human- and machine-based judging systems

Abstract

Motivated by recent controversy over biases associated with algorithmic decision-making, we embarked on studying various stakeholders’ perceptions related to potential biases in verdicts from human-based and algorithm-based judging. In an empirical study conducted in the domain of gymnastics judging, we found that, while our informants viewed both human- and AI-based judging systems as being subject to biases (of different types), they were quite welcoming of a shift from human-based judging to machine-based judging. Our findings show that the athletes trusted strongly in unknown, “magic” capabilities of AI, thought to be more objective and impartial. This, in turn, encouraged potential acceptance of new technology. While the gymnasts saw AI-based systems in a positive light, judges demonstrated less favorable perceptions overall and less acceptance of AI technology, expressing concern about possible challenges of AI.