RHiOTS: A Framework for Evaluating Hierarchical Time Series Forecasting Algorithms

• Type: Article in International Conference Proceedings Book
• Date: 2024

Authors

Roque, L (Author) Other; Carlos Soares (Author) FEUP; Torgo, L (Author) Other

Conference proceedings International

Title: PROCEEDINGS OF THE 30TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, KDD 2024

Pages: 2491-2499

30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Barcelona, SPAIN, AUG 25-29, 2024

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 1 Citation

Other information

Authenticus ID: P-016-X25

DOI: 10.1145/3637528.3672062

Abstract (EN): We introduce the Robustness of Hierarchically Organized Time Series (RHiOTS) framework, designed to assess the robustness of hierarchical time series forecasting models and algorithms on real-world datasets. Hierarchical time series, where lower-level forecasts must sum to upper-level ones, are prevalent in various contexts, such as retail sales across countries. Current empirical evaluations of forecasting methods are often limited to a small set of benchmark datasets, offering a narrow view of algorithm behavior. RHiOTS addresses this gap by systematically altering existing datasets and modifying the characteristics of individual series and their interrelations. It uses a set of parameterizable transformations to simulate those changes in the data distribution. Additionally, RHiOTS incorporates an innovative visualization component, turning complex, multidimensional robustness evaluation results into intuitive, easily interpretable visuals. This approach allows an in-depth analysis of algorithm and model behavior under diverse conditions. We illustrate the use of RHiOTS by analyzing the predictive performance of several algorithms. Our findings show that traditional statistical methods are more robust than state-of-the-art deep learning algorithms, except when the transformation effect is highly disruptive. Furthermore, we found no significant differences in the robustness of the algorithms when applying specific reconciliation methods, such as MinT. RHiOTS provides researchers with a comprehensive tool for understanding the nuanced behavior of forecasting algorithms, offering a more reliable basis for selecting the most appropriate method for a given problem.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 9