How to Correctly Evaluate an Automatic Bioacoustics Classification Method

• Type: Article in International Conference Proceedings Book
• Date: 2016

Authors

Colonna, JG (Author) Other; João Gama (Author) FEP; Nakamura, EF (Author) Other

Conference proceedings International

Title: ADVANCES IN ARTIFICIAL INTELLIGENCE, CAEPIA 2016

Pages: 37-47

17th Conference of the Spanish-Association-for-Artificial-Intelligence (CAEPIA)

Salamanca, SPAIN, SEP 14-16, 2016

Indexing

ISI Web of Knowledge - 11 Citations

Scopus - 13 Citations

Other information

Authenticus ID: P-00K-VAX

DOI: 10.1007/978-3-319-44636-3_4

Abstract (EN): In this work, we introduce a more appropriate (or alternative) approach to evaluate the performance and the generalization capabilities of a framework for automatic anuran call recognition. We show that, by using the common k-folds Cross-Validation (k-CV) procedure to evaluate the expected error in a syllable-based recognition system the recognition accuracy is overestimated. To overcome this problem, and to provide a fair evaluation, we propose a new CV procedure in which the specimen information is considered during the split step of the k-CV. Therefore, we performed a k-CV by specimens (or individuals) showing that the accuracy of the system decrease considerably. By introducing the specimen information, we are able to answer a more fundamental question: Given a set of syllables that belongs to a specific group of individuals, can we recognize new specimens of the same species? In this article, we go deeper into the reviews and the experimental evaluations to answer this question.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 11