Document Server@UHasselt >
Research >
Research publications >

Please use this identifier to cite or link to this item: http://hdl.handle.net/1942/763

Title: A universal method of information retrieval evaluation: the "missing" link M and the universal IR surface
Authors: EGGHE, Leo
Issue Date: 2004
Citation: Information Processing & Management, 40(1). p. 21-30
Abstract: The paper shows that the present evaluation methods in information retrieval (basically recall R and precision P and in some cases fallout F) lack universal comparability in the sense that their values depend on the generality of the IR problem. A solution is given by using all "parts" of the database, including the non-relevant documents and also the notretrieved documents. It turns out that the solution is given by introducing the measure M being the fraction of the not-retrieved documents that are relevant (hence the "miss" measure). We prove that - independent of the IR problem or of the IR action - the quadruple (P,R,F,M) belongs to a universal IR surface, being the same for &l IR-activities. This universality is then exploited by defining a new measure for evaluation in IR allowing for unbiased comparisons of all IR results. We also show that only using 1, 2 or even 3 measures from the set {P,R,F,M} necessary leads to evaluation measures that are nonuniversal and hence not capable of comparing different IR situations.
URI: http://hdl.handle.net/1942/763
DOI: 10.1016/S0306-4573(02)00094-8
ISI #: 000187738300002
ISSN: 0306-4573
Category: A1
Type: Journal Contribution
Validation: ecoom, 2005
Appears in Collections: Research publications

Files in This Item:

Description SizeFormat
Published version256.97 kBAdobe PDF
Non peer-reviewed author version364.52 kBAdobe PDF

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.