Probabilistic change detection and visualization methods for the assessment of temporal stability in biomedical data quality

• Tipo: Artigo em Revista Científica Internacional
• Data: 2015

Autores

Carlos Saez (Autor) Outra; Pedro Pereira Rodrigues (Autor) FMUP; João Gama (Autor) FEP; Montserrat Robles (Autor) Outra; Juan M Garcia Gomez (Autor) Outra

Revista

Título: Data Mining and Knowledge Discovery

Vol. 29

Páginas: 950-975

ISSN: 1384-5810

Editora: Springer Nature

Indexação

ISI Web of Knowledge - 16 Citações

Scopus - 18 Citações

Classificação Científica

FOS: Ciências exactas e naturais > Ciências da computação e da informação

Outras Informações

ID Authenticus: P-00G-6BX

DOI: 10.1007/s10618-014-0378-6

Abstract (EN): Knowledge discovery on biomedical data can be based on on-line, data-stream analyses, or using retrospective, timestamped, off-line datasets. In both cases, changes in the processes that generate data or in their quality features through time may hinder either the knowledge discovery process or the generalization of past knowledge. These problems can be seen as a lack of data temporal stability. This work establishes the temporal stability as a data quality dimension and proposes new methods for its assessment based on a probabilistic framework. Concretely, methods are proposed for (1) monitoring changes, and (2) characterizing changes, trends and detecting temporal subgroups. First, a probabilistic change detection algorithm is proposed based on the Statistical Process Control of the posterior Beta distribution of the Jensen-Shannon distance, with a memoryless forgetting mechanism. This algorithm (PDF-SPC) classifies the degree of current change in three states: In-Control, Warning, and Out-of-Control. Second, a novel method is proposed to visualize and characterize the temporal changes of data based on the projection of a non-parametric information-geometric statistical manifold of time windows. This projection facilitates the exploration of temporal trends using the proposed IGT-plot and, by means of unsupervised learning methods, discovering conceptually-related temporal subgroups. Methods are evaluated using real and simulated data based on the National Hospital Discharge Survey (NHDS) dataset.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 26