Evaluation Procedures for Forecasting with Spatiotemporal Data

• Type: Article in International Scientific Journal
• Date: 2021

Authors

Oliveira, M (Author) Other; Torgo, L (Author) FCUP; Costa, VS (Author) FCUP

Journal

Title: Mathematics

Vol. 9 No. 12

Final page: 691

Publisher: MDPI

Indexing

ISI Web of Knowledge - 17 Citations

Scopus - 18 Citations

Other information

Authenticus ID: P-00T-P3N

DOI: 10.3390/math9060691

Abstract (EN): The increasing use of sensor networks has led to an ever larger number of available spatiotemporal datasets. Forecasting applications using this type of data are frequently motivated by important domains such as environmental monitoring. Being able to properly assess the performance of different forecasting approaches is fundamental to achieve progress. However, traditional performance estimation procedures, such as cross-validation, face challenges due to the implicit dependence between observations in spatiotemporal datasets. In this paper, we empirically compare several variants of cross-validation (CV) and out-of-sample (OOS) performance estimation procedures, using both artificially generated and real-world spatiotemporal datasets. Our results show both CV and OOS reporting useful estimates, but they suggest that blocking data in space and/or in time may be useful in mitigating CV's bias to underestimate error. Overall, our study shows the importance of considering data dependencies when estimating the performance of spatiotemporal forecasting models.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 27