Adaptive windowing for online learning from multiple inter-related data streams

• Type: Article in International Conference Proceedings Book
• Date: 2011

Authors

Ikonomovska, E (Author) Other; Driessensy, K (Author) Other; Dzeroski, S (Author) Other; João Gama (Author) FEP

Conference proceedings International

Title: Proceedings - IEEE International Conference on Data Mining, ICDM

Pages: 697-704

Data Mining Workshops (ICDMW), 2011 IEEE 11th International Conference on, Vancouver, BC, Canada, December 11, 2011

Indexing

Scopus - 4 Citations

Other information

Authenticus ID: P-008-2KY

DOI: 10.1109/icdmw.2011.22

Abstract (EN): Relational reinforcement learning is a promising branch of reinforcement learning research that deals with structured environments. In these environments, states and actions are differentiated by the presence of certain types of objects and the relations between them and the objects that are involved in the actions. This makes it ultimately suited for tasks that require the manipulation of multiple, interacting objects, such as tasks that a future house-holding robot can be expected to perform like cleaning up a dinner table or storing away done dishes. However, the application of relational reinforcement learning to robotics has been hindered by assumptions such as discrete and atomic state observations. Typical robotic observation systems work in a streaming setup, where objects are discovered and recognized and their placement within their surroundings is determined in a quasi continuous manner instead of a state based one. The resulting information stream can be compared to a set of multiple inter-related data streams. In this paper, we propose an adaptive windowing strategy for generating a stream of learning examples and enabling relational learning from this kind of data. Our approach is independent from the learning algorithm and is based on a gradient search over the space of parameter values, i.e., window sizes, guided by the estimation of the testing error. The proposed algorithm performs online and is data driven and flexible. To the best of our knowledge, this is the first work addressing this problem. Our ideas are empirically supported by an extensive experimental evaluation in a controlled setup using artificial data. © 2011 IEEE.

Language: English

Type (Professor's evaluation): Scientific