Model-agnostic auditing: a lost cause?

Hansen, Sakina and Loftus, Joshua ORCID: 0000-0002-2905-1632 (2023) Model-agnostic auditing: a lost cause? CEUR Workshop Proceedings, 3442. ISSN 1613-0073

Text (Model-Agnostic Auditing: A Lost Cause?) - Published Version Available under License Creative Commons Attribution. Download (1MB)

• Scopus publication
• Author
• Publisher

Abstract

Tools for interpretable machine learning (IML) or explainable artificial intelligence (xAI) can be used to audit algorithms for fairness or other desiderata. In a black-box setting without access to the algorithm’s internal structure an auditor may be limited to methods that are model-agnostic. These methods have severe limitations with important consequences for outcomes such as fairness. Among model-agnostic IML methods, visualizations such as the partial dependence plot (PDP) or individual conditional expectation (ICE) plots are popular and useful for displaying qualitative relationships. Although we focus on fairness auditing with PDP/ICE plots, the consequences we highlight generalize to other auditing or IML/xAI applications. This paper questions the validity of auditing in high-stakes settings with contested values or conflicting interests if the audit methods are model-agnostic.

Item Type:	Article
Additional Information:	© 2023 The Author(s)
Divisions:	Statistics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science H Social Sciences > HA Statistics
Date Deposited:	01 Sep 2023 09:27
Last Modified:	25 Apr 2024 17:30
URI:	http://eprints.lse.ac.uk/id/eprint/120114

Actions (login required)

View Item