www.uhasselt.be
DSpace

Document Server@UHasselt >
Research >
Research publications >

Please use this identifier to cite or link to this item: http://hdl.handle.net/1942/19828

Title: From a Ranking System to a Confidence Aware Semi-Automatic Classifier
Authors: Malherbe, Emmanuel
Vanrompay, Yves
Aufaure, Marie-Aude
Issue Date: 2015
Publisher: ELSEVIER SCIENCE BV
Citation: Ding, L.; Pang, C.; Kew, L.M.; Jain, L.C.; Howlett, R.J. (Ed.). KNOWLEDGE-BASED AND INTELLIGENT INFORMATION & ENGINEERING SYSTEMS 19TH ANNUAL CONFERENCE, KES-2015, p. 73-82
Series/Report: Procedia Computer Science
Series/Report no.: 60
Abstract: Many systems rank outcomes before suggesting them to a user, such as Recommender Systems or Information Retrieval Algorithms. These systems require manual validation, which is time consuming and costly in industrial context. As it is the case in our industrial applications, we assume that the user's needs can be fulfilled by only one relevant outcome. We thus consider an algorithm that systematically selects the top ranked outcome. This approach requires to compute a correctness, estimating the confidence of the automatic decision, or equivalently how likely the first outcome of the ranking system is to be correct. Based on this estimation, we can apply a threshold on the correctness, above which no manual action is required; the system avoids human validation in many cases. This paper proposes a novel method to estimate this correctness based on a supervised classification approach using the manual validations available in the base coupled with a representation of the system's scores. We conducted experiments on Multiposting real-world datasets generated by algorithms used in the industry; the first algorithm categorizes a job offer, the second recommends semantic equivalents for a given expression in a nomenclature. Our approach has thereby been evaluated and compared, and showed good results on our datasets, even with a limited training base. Moreover, in our experiments, for a given threshold, the better is the correctness estimation, the more performant is the semi-automatic system, showing that the correctness estimation leads thus to a crucial efficiency gain. (C) 2015 The Authors. Published by Elsevier B.V.
Notes: [Malherbe, Emmanuel] Multiposting, F-75009 Paris, France. [Malherbe, Emmanuel; Aufaure, Marie-Aude] CentraleSupelec, F-92290 Chetenay Malabry, France. [Vanrompay, Yves] Hasselt Univ, IMOB, B-3590 Diepenbeek, Belgium.
URI: http://hdl.handle.net/1942/19828
DOI: 10.1016/j.procs.2015.08.106
ISI #: 000360571700007
ISSN: 1877-0509
Category: C1
Type: Proceedings Paper
Validation: ecoom, 2016
Appears in Collections: Research publications

Files in This Item:

Description SizeFormat
published version685.35 kBAdobe PDF

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.