The search of conditional outliers

• Tipo: Artigo em Revista Científica Internacional
• Data: 2019

Autores

Portela, E (Autor) Outra; Rita Ribeiro (Autor) FCUP; João Gama (Autor) FEP

Revista

Título: Intelligent Data Analysis

Vol. 23

Páginas: 23-39

ISSN: 1088-467X

Editora: IOS PRESS

Indexação

ISI Web of Knowledge - 5 Citações

Scopus - 5 Citações

Outras Informações

ID Authenticus: P-00Q-917

DOI: 10.3233/ida-173619

Abstract (EN): There is no standard definition of outliers, but most authors agree that outliers are points far from other data points. Several outlier detection techniques have been developed mainly for two different purposes. On one hand, outliers are considered error measurement observations that should be removed from the analysis, e.g. robust statistics. On the other hand, outliers are the interesting observations, like in fraud detection, and should be modelled by some learning method. In this work, we start from the observation that outliers are affected by the so-called simpson paradox: a trend that appears in different groups of data but disappears or reverses when these groups are combined. Given a data set, we learn a regression tree. The tree grows by partitioning the data into groups more and more homogeneous of the target variable. At each partition defined by the tree, we apply a box plot on the target variable to detect outliers. We would expect that the deeper nodes of the tree would contain less and less outliers. We observe that some points previously signalled as outliers are no more signalled as such, but new outliers appear.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 17