Influence of data distribution in missing data imputation

• Type: Article in International Conference Proceedings Book
• Date: 2017

Authors

Santos M.S. (Author) Other; Soares J.P. (Author) Other; Pedro Henriques Abreu (Author) Other; Araújo H. (Author) Other; Santos J. (Author) Other

Conference proceedings International

Title: Artificial Intelligence in Medicine - 16th Conference on Artificial Intelligence in Medicine, AIME 2017, Vienna, Austria, June 21-24, 2017, Proceedings

Pages: 285-294

16th Conference on Artificial Intelligence in Medicine, AIME 2017

21 June 2017 through 24 June 2017

Indexing

Scopus - 35 Citations

Other information

Authenticus ID: P-00Z-SMQ

DOI: 10.1007/978-3-319-59758-4_33

Abstract (EN): Dealing with missing data is a crucial step in the preprocessing stage of most data mining projects. Especially in healthcare contexts, addressing this issue is fundamental, since it may result in keeping or loosing critical patient information that can help physicians in their daily clinical practice. Over the years, many researchers have addressed this problem, basing their approach on the implementation of a set of imputation techniques and evaluating their performance in classification tasks. These classic approaches, however, do not consider some intrinsic data information that could be related to the performance of those algorithms, such as features¿ distribution. Establishing a correspondence between data distribution and the most proper imputation method avoids the need of repeatedly testing a large set of methods, since it provides a heuristic on the best choice for each feature in the study. The goal of this work is to understand the relationship between data distribution and the performance of well-known imputation techniques, such as Mean, Decision Trees, k-Nearest Neighbours, Self-Organizing Maps and Support Vector Machines imputation. Several publicly available datasets, all complete, were selected attending to several characteristics such as number of distributions, features and instances. Missing values were artificially generated at different percentages and the imputation methods were evaluated in terms of Predictive and Distributional Accuracy. Our findings show that there is a relationship between features¿ distribution and algorithms¿ performance, although some factors must be taken into account, such as the number of features per distribution and the missing rate at state.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 9