Hansen, Sakina and Loftus, Joshua ORCID: 0000-0002-2905-1632 (2023) Model-agnostic auditing: a lost cause? CEUR Workshop Proceedings, 3442. ISSN 1613-0073
Text (Model-Agnostic Auditing: A Lost Cause?)
- Published Version
Available under License Creative Commons Attribution. Download (1MB) |
Abstract
Tools for interpretable machine learning (IML) or explainable artificial intelligence (xAI) can be used to audit algorithms for fairness or other desiderata. In a black-box setting without access to the algorithm’s internal structure an auditor may be limited to methods that are model-agnostic. These methods have severe limitations with important consequences for outcomes such as fairness. Among model-agnostic IML methods, visualizations such as the partial dependence plot (PDP) or individual conditional expectation (ICE) plots are popular and useful for displaying qualitative relationships. Although we focus on fairness auditing with PDP/ICE plots, the consequences we highlight generalize to other auditing or IML/xAI applications. This paper questions the validity of auditing in high-stakes settings with contested values or conflicting interests if the audit methods are model-agnostic.
Item Type: | Article |
---|---|
Additional Information: | © 2023 The Author(s) |
Divisions: | Statistics |
Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science H Social Sciences > HA Statistics |
Date Deposited: | 01 Sep 2023 09:27 |
Last Modified: | 07 Oct 2024 16:04 |
URI: | http://eprints.lse.ac.uk/id/eprint/120114 |
Actions (login required)
View Item |