Exploring the effects of data distribution in missing data imputation

• Type: Article in International Conference Proceedings Book
• Date: 2018

Authors

Pompeu Soares, J (Author) Other; Seoane Santos, M (Author) Other; Pedro Henriques Abreu (Author) Other; Araújo, H (Author) Other; Santos, J (Author) Other

Conference proceedings International

Title: Advances in Intelligent Data Analysis XVII - 17th International Symposium, IDA 2018, 's-Hertogenbosch, The Netherlands, October 24-26, 2018, Proceedings

Pages: 251-263

17th International Symposium on Intelligent Data Analysis, IDA 2018

24 October 2018 through 26 October 2018

Indexing

Scopus - 7 Citations

Other information

Authenticus ID: P-00P-TN0

DOI: 10.1007/978-3-030-01768-2_21

Abstract (EN): In data imputation problems, researchers typically use several techniques, individually or in combination, in order to find the one that presents the best performance over all the features comprised in the dataset. This strategy, however, neglects the nature of data (data distribution) and makes impractical the generalisation of the findings, since for new datasets, a huge number of new, time consuming experiments need to be performed. To overcome this issue, this work aims to understand the relationship between data distribution and the performance of standard imputation techniques, providing a heuristic on the choice of proper imputation methods and avoiding the needs to test a large set of methods. To this end, several datasets were selected considering different sample sizes, number of features, distributions and contexts and missing values were inserted at different percentages and scenarios. Then, different imputation methods were evaluated in terms of predictive and distributional accuracy. Our findings show that there is a relationship between features¿ distribution and algorithms¿ performance, and that their performance seems to be affected by the combination of missing rate and scenario at state and also other less obvious factors such as sample size, goodness-of-fit of features and the ratio between the number of features and the different distributions comprised in the dataset. © Springer Nature Switzerland AG 2018.

Language: English

Type (Professor's evaluation): Scientific