Model Selection for Time Series Forecasting An Empirical Analysis of Multiple Estimators

• Tipo: Artigo em Revista Científica Internacional
• Data: 2023

Autores

Cerqueira, V (Autor) FEUP; Torgo, L (Autor) FCUP; Carlos Soares (Autor) FEUP

Revista

A Revista está pendente de validação pelos Serviços Administrativos.

Título: NEURAL PROCESSING LETTERS

Vol. 55

ISSN: 1370-4621

Indexação

ISI Web of Knowledge - 3 Citações

Scopus - 5 Citações

Outras Informações

ID Authenticus: P-00T-RXM

DOI: 10.1007/s11063-023-11239-8

Abstract (EN): Evaluating predictive models is a crucial task in predictive analytics. This process is especially challenging with time series data because observations are not independent. Several studies have analyzed how different performance estimation methods compare with each other for approximating the true loss incurred by a given forecasting model. However, these studies do not address how the estimators behave for model selection: the ability to select the best solution among a set of alternatives. This paper addresses this issue. The goal of this work is to compare a set of estimation methods for model selection in time series forecasting tasks. This objective is split into two main questions: (i) analyze how often a given estimation method selects the best possible model; and (ii) analyze what is the performance loss when the best model is not selected. Experiments were carried out using a case study that contains 3111 time series. The accuracy of the estimators for selecting the best solution is low, despite being significantly better than random selection. Moreover, the overall forecasting performance loss associated with the model selection process ranges from 0.28 to 0.58%. Yet, no considerable differences between different approaches were found. Besides, the sample size of the time series is an important factor in the relative performance of the estimators.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 19