mdatagen: A python library for the artificial generation of missing data

• Type: Article in International Scientific Journal
• Date: 2025

Authors

Mangussi, AD (Author) Other; Santos, MS (Author) FCUP; Lopes, FL (Author) Other; Pereira, RC (Author) Other; Lorena, AC (Author) Other; Pedro Henriques Abreu (Author) Other

Journal

Title: Neurocomputing

Vol. 625

ISSN: 0925-2312

Publisher: Elsevier

Indexing

ISI Web of Knowledge - 1 Citation

Scopus - 1 Citation

Other information

Authenticus ID: P-017-ZVT

DOI: 10.1016/j.neucom.2025.129478

Abstract (EN): Missing data is characterized by the presence of absent values in data (i.e., missing values) and it is currently categorized into three different mechanisms: Missing Completely at Random, Missing At Random, and Missing Not At Random. When performing missing data experiments and evaluating techniques to handle absent values, these mechanisms are often artificially generated (a process referred to as data amputation) to assess the robustness and behavior of the used methods. Due to the lack of a standard benchmark for data amputation, different implementations of the mechanisms are used in related research (some are often not disclaimed), preventing the reproducibility of results and leading to an unfair or inaccurate comparison between existing and new methods. Moreover, for users outside the field, experimenting with missing data or simulating the appearance of missing values in real-world domains is unfeasible, impairing stress testing in machine learning systems. This work introduces mdatagen, an open source Python library for the generation of missing data mechanisms across 20 distinct scenarios, following different univariate and multivariate implementations of the established missing mechanisms. The package therefore fosters reproducible results across missing data experiments and enables the simulation of artificial missing data under flexible configurations, making it very versatile to mimic several real-world applications involving missing data. The source code and detailed documentation for mdatagen are available at https://github.com/ArthurMangussi/pymdatagen.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 10