A Comparative Study of Performance Estimation Methods for Time Series Forecasting

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2017

Autores

Cerqueira, V (Autor) Outra; Torgo, L (Autor) FCUP; Smailovic, J (Autor) Outra; Mozetic, I (Autor) Outra

Ata de Conferência Internacional

Título: 2017 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2017, Tokyo, Japan, October 19-21, 2017

Páginas: 529-538

Indexação

ISI Web of Knowledge - 29 Citações

Scopus - 36 Citações

Outras Informações

ID Authenticus: P-00N-H1B

DOI: 10.1109/dsaa.2017.7

Abstract (EN): Performance estimation denotes a task of estimating the loss that a predictive model will incur on unseen data. These procedures are part of the pipeline in every machine learning task and are used for assessing the overall generalisation ability of models. In this paper we address the application of these methods to time series forecasting tasks. For independent and identically distributed data the most common approach is cross-validation. However, the dependency among observations in time series raises some caveats about the most appropriate way to estimate performance in these datasets and currently there is no settled way to do so. We compare different variants of cross-validation and different variants of out-of-sample approaches using two case studies: One with 53 real-world time series and another with three synthetic time series. Results show noticeable differences in the performance estimation methods in the two scenarios. In particular, empirical experiments suggest that cross-validation approaches can be applied to stationary synthetic time series. However, in real-world scenarios the most accurate estimates are produced by the out-of-sample methods, which preserve the temporal order of observations.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 10