Cookies?
Library Header Image
LSE Research Online LSE Library Services

A new approach to inter-rater agreement through stochastic orderings: the discrete case

Giovagnoli, Alessandra, Marzialetti, Johnny and Wynn, Henry P. (2008) A new approach to inter-rater agreement through stochastic orderings: the discrete case. Metrika, 67 (3). pp. 349-370. ISSN 0026-1335

Full text not available from this repository.

Abstract

We wish to study inter-rater agreement comparing groups of observers who express their ratings on a discrete or ordinal scale. The starting point is that of defining what we mean by “agreement”. Given d observers, let the scores they assign to a given statistical unit be expressed as a d-vector in the real space. We define a deterministic ordering among these vectors, which expresses the degree of the raters’ agreement. The overall scoring of the raters on the sample space will be a d-dimensional random vector. We then define an associated partial ordering among the random vectors of the ratings, illustrate a number of its properties, and look at order-preserving functions (agreement measures). In this paper we also show how to test the hypothesis of greater agreement against the unrestricted hypothesis, and the hypothesis of equal agreement against the hypothesis that an agreement ordering holds. The test is applied to real data on two medical observers rating clinical guidelines.

Item Type: Article
Official URL: http://link.springer.com/journal/184
Additional Information: © 2008 Springer-Verlag
Library of Congress subject classification: Q Science > QA Mathematics
Sets: Research centres and groups > Centre for the Analysis of Time Series (CATS)
Research centres and groups > Decision Support and Risk Group (DSRG)
Rights: http://www.lse.ac.uk/library/usingTheLibrary/academicSupport/OA/depositYourResearch.aspx
Date Deposited: 26 Feb 2014 11:23
URL: http://eprints.lse.ac.uk/55871/

Actions (login required)

Record administration - authorised staff only Record administration - authorised staff only