Library Header Image
LSE Research Online LSE Library Services

Model-agnostic auditing: a lost cause?

Hansen, Sakina and Loftus, Joshua ORCID: 0000-0002-2905-1632 (2023) Model-agnostic auditing: a lost cause? CEUR Workshop Proceedings, 3442. ISSN 1613-0073

[img] Text (Model-Agnostic Auditing: A Lost Cause?) - Published Version
Available under License Creative Commons Attribution.

Download (1MB)


Tools for interpretable machine learning (IML) or explainable artificial intelligence (xAI) can be used to audit algorithms for fairness or other desiderata. In a black-box setting without access to the algorithm’s internal structure an auditor may be limited to methods that are model-agnostic. These methods have severe limitations with important consequences for outcomes such as fairness. Among model-agnostic IML methods, visualizations such as the partial dependence plot (PDP) or individual conditional expectation (ICE) plots are popular and useful for displaying qualitative relationships. Although we focus on fairness auditing with PDP/ICE plots, the consequences we highlight generalize to other auditing or IML/xAI applications. This paper questions the validity of auditing in high-stakes settings with contested values or conflicting interests if the audit methods are model-agnostic.

Item Type: Article
Additional Information: © 2023 The Author(s)
Divisions: Statistics
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
H Social Sciences > HA Statistics
Date Deposited: 01 Sep 2023 09:27
Last Modified: 02 Jun 2024 06:15

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics