Estimating reliability for assessing and correcting individual streaming predictions

• Tipo: Capítulo ou Parte de Livro
• Data: 2012

Autores

Pedro Pereira Rodrigues (Autor) FMUP; Bosni¿, Z (Autor) Outra; João Gama (Autor) FEP; Kononenko, I (Autor) Outra

Livro

Título: Reliable Knowledge Discovery

Páginas: 29-49

ISBN: 9781461419037; 9781461419020

Indexação

Scopus - 4 Citações

Outras Informações

ID Authenticus: P-00K-DXF

DOI: 10.1007/978-1-4614-1903-7

Abstract (EN): Several predictive systems are nowadays vital for operations and decision support. The quality of these systems is most of the time defined by their average accuracy which has low or no information at all about the estimated error of each individual prediction. In these cases, users should be allowed to associate a measure of reliability to each prediction. However, with the advent of data streams, batch state-of-the-art reliability estimates need to be redefined. In this chapter we adapt and evaluate five empirical measures for online reliability estimation of individual predictions: similarity-based (k-NN) error, local sensitivity (bias and variance) and online bagging predictions (bias and variance). Evaluation is performed with a neural network base model on two different problems, with results showing that online bagging and k-NN estimates are consistently correlated with the error of the base model. Furthermore, we propose an approach for correcting individual predictions based on the CNK reliability estimate. Evaluation is done on a real-world problem (prediction of the electricity load for a selected European geographical region), using two different regression models: neural network and the k nearest neighbors algorithm. Comparison is performed with corrections based on the Kalman filter. The results show that our method performs better than the Kalman filter, significantly improving the original predictions to more accurate values.

Idioma: Inglês

Tipo (Avaliação Docente): Científica