Active learning approaches for labeling text: review and assessment of the performance of active learning approaches

Miller, Blake ORCID: 0000-0002-4707-0984, Linder, Fridolin and Mebane, Walter R. (2020) Active learning approaches for labeling text: review and assessment of the performance of active learning approaches. Political Analysis, 28 (4). 532 - 551. ISSN 1047-1987

Text (Paper_Active_Learning_Approaches) - Accepted Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (6MB)

• Author

Abstract

Supervised machine learning methods are increasingly employed in political science. Such models require costly manual labeling of documents. In this paper, we introduce active learning, a framework in which data to be labeled by human coders are not chosen at random but rather targeted in such a way that the required amount of data to train a machine learning model can be minimized. We study the benefits of active learning using text data examples. We perform simulation studies that illustrate conditions where active learning can reduce the cost of labeling text data. We perform these simulations on three corpora that vary in size, document length, and domain. We find that in cases where the document class of interest is not balanced, researchers can label a fraction of the documents one would need using random sampling (or “passive” learning) to achieve equally performing classifiers. We further investigate how varying levels of intercoder reliability affect the active learning procedures and find that even with low reliability, active learning performs more efficiently than does random sampling.

Item Type:	Article
Official URL:	https://www.cambridge.org/core/journals/political-...
Additional Information:	© 2020 The Authors
Divisions:	Methodology
Subjects:	J Political Science > JA Political science (General) H Social Sciences > H Social Sciences (General)
Date Deposited:	12 Oct 2020 11:24
Last Modified:	16 Apr 2024 22:18
URI:	http://eprints.lse.ac.uk/id/eprint/106964

Actions (login required)

View Item