Algorithm aversion is "biased assessment of an algorithm which manifests in negative behaviours and attitudes towards the algorithm compared to a human agent."[1] It describes a phenomenon where humans reject advice from an algorithm in a case where they would accept the same advice if they thought it was coming from another human.
Algorithms, such as those employing machine learning methods or various forms of artificial intelligence, are commonly used to provide recommendations or advice to human decisionmakers. For example, recommender systems are used in E-commerce to identify products a customer might like, and artificial intelligence is used in healthcare to assist with diagnosis and treatment decisions. However, humans sometimes appear to resist or reject these algorithmic recommendations more than if the recommendation had come from a human. Notably, algorithms are often capable of outperforming humans, so rejecting algorithmic advice can result in poor performance or suboptimal outcomes.
This is an emerging topic and it is not completely clear why or under what circumstances people will display algorithm aversion. In some cases, people seem to be more likely to take recommendations from an algorithm than from a human, a phenomenon called algorithm appreciation.[2]
Examples of Algorithm Aversion
Algorithm aversion has been studied in a wide variety of contexts. For example, people seem to prefer recommendations for jokes from a human rather than from an algorithm,[3] and would rather rely on a human to predict the number of airline passengers from each US state instead of an algorithm.[4] People also seem to prefer medical recommendations from human doctors instead of an algorithm.[citation needed]
Proposed Factors Influencing Algorithm Aversion
Various frameworks have been proposed to explain the causes for algorithm aversion and techniques or system features that might help reduce aversion.[1][5]
Decision control (Role of the "Human in the Loop")
Algorithms may either be used in an advisory role (providing advice to a human who will make the final decision) or in an delegatory role (where the algorithm makes a decision without human supervision). A movie recommendation system providing a list of suggestions would be in an advisory role, whereas the human driver delegates the task of steering the car to Tesla's Autopilot. Generally, a lack of decision control tends to increase algorithm aversion.
Perceptions about Algorithm Capabilities and Performance
Overall, people tend to judge machines more critically than they do humans.[6] Several system characteristics or factors have been shown to influence how people evaluate algorithms.
Algorithm Process and the role of System Transparency
One reason people display resistance to algorithms is a lack of understanding about how the algorithm is arriving at its recommendation.[3] People also seem to have a better intuition for how another human would make recommendations. Whereas people assume that other humans will account for unique differences between situations, they sometimes perceive algorithms as incapable of considering individual differences and resist the algorithms accordingly.[7] Providing explanations about how algorithms work has been shown to reduce aversion. These explanations can take a variety of forms, including about how the algorithm as a whole works, about why it is making a particular recommendation in a specific case, or how confident it is in its recommendation.
Decision Domain
People are generally skeptical that algorithms can make accurate predictions in certain areas, particularly if task involves a seemingly human characteristic like morals or empathy. Algorithm aversion tends to be higher when the task is more subjective and lower on tasks that are objective or quantifiable.[1]
Human characteristics
Domain expertise
Expertise in a particular field has been shown to increase algorithm aversion[2] and reduce use of algorithmic decision rules.[8] Overconfidence may partially explain this effect; experts might feel that an algorithm is not capable of the types of judgments they make. Compared to non-experts, experts also have more knowledge of the field and therefore may be more critical of a recommendation. Where a non-expert might accept a recommendation ("The algorithm must know something I don't.") the expert might find specific fault with the algorithm's recommendation ("This recommendation does not account for a particular factor").
Decision-making research has shown that experts in a given field tend to think about decisions differently than a non-expert.[9] Experts chunk and group information; for example, expert chess players will see opening positions (e.g., the Queen's Gambit) instead of individual pieces on the board. Experts may see a situation as a functional representation (e.g., a doctor could see a trajectory and predicted outcome for a patient instead of a list of medications and symptoms). These differences may also partly account for the increased algorithm aversion seen in experts.
Algorithm appreciation
Studies do not consistently show people demonstrating bias against algorithms. Results are mixed, showing that people sometimes seem to prefer advice that comes from an algorithm instead of a human which is termed algorithm appreciation.[2]
References
- ^ a b c Jussupow, Ekaterina; Benbasat, Izak; Heinzl, Armin (2020). "Why Are We Averse Towards Algorithms ? A Comprehensive Literature Review on Algorithm Aversion". Twenty-Eighth European Conference on Information Systems (ECIS2020): 1–16.
- ^ a b c Logg, Jennifer M.; Minson, Julia A.; Moore, Don A. (2019-03-01). "Algorithm appreciation: People prefer algorithmic to human judgment". Organizational Behavior and Human Decision Processes. 151: 90–103. doi:10.1016/j.obhdp.2018.12.005. ISSN 0749-5978.
- ^ a b Yeomans, Michael; Shah, Anuj; Mullainathan, Sendhil; Kleinberg, Jon (2019). "Making sense of recommendations". Journal of Behavioral Decision Making. 32 (4): 403–414. doi:10.1002/bdm.2118. ISSN 1099-0771.
- ^ Dietvorst, Berkeley J.; Simmons, Joseph P.; Massey, Cade (2015). "Algorithm aversion: People erroneously avoid algorithms after seeing them err". Journal of Experimental Psychology: General. 144 (1): 114–126. doi:10.1037/xge0000033. ISSN 1939-2222. PMID 25401381.
- ^ Burton, Jason W.; Stein, Mari-Klara; Jensen, Tina Blegind (2020). "A systematic review of algorithm aversion in augmented decision making". Journal of Behavioral Decision Making. 33 (2): 220–239. doi:10.1002/bdm.2155. ISSN 1099-0771.
- ^ Hidalgo, Cesar (2021). How Humans Judge Machines. Cambridge, MA: MIT Press. ISBN 978-0-262-04552-0.
- ^ Longoni, Chiara; Bonezzi, Andrea; Morewedge, Carey K (2019-05-03). "Resistance to Medical Artificial Intelligence". Journal of Consumer Research. 46 (4): 629–650. doi:10.1093/jcr/ucz013. ISSN 0093-5301.
- ^ Arkes, Hal R.; Dawes, Robyn M.; Christensen, Caryn (1986-02-01). "Factors influencing the use of a decision rule in a probabilistic task". Organizational Behavior and Human Decision Processes. 37 (1): 93–110. doi:10.1016/0749-5978(86)90046-4. ISSN 0749-5978.
- ^ Feltovich, Paul J.; Prietula, Michael J.; Ericsson, K. Anders (2006), Ericsson, K. Anders; Charness, Neil; Feltovich, Paul J.; Hoffman, Robert R. (eds.), "Studies of Expertise from Psychological Perspectives", The Cambridge Handbook of Expertise and Expert Performance, Cambridge Handbooks in Psychology, Cambridge: Cambridge University Press, pp. 41–68, doi:10.1017/cbo9780511816796.004, ISBN 978-1-107-81097-6, retrieved 2021-09-08