Exploring Transformers for Multi-Label Classification of Java Vulnerabilities

• Type: Article in International Conference Proceedings Book
• Date: 2022

Authors

Mamede, C (Author) Other; Pinconschi, E (Author) Other; Rui Abreu (Author) FEUP; José Campos (Author) FEUP

Conference proceedings International

Title: IEEE International Conference on Software Quality, Reliability and Security, QRS

Pages: 43-52

22nd IEEE International Conference on Software Quality, Reliability and Security, QRS 2022

Virtual, Online, 5 December 2022 through 9 December 2022

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-00Y-8M0

DOI: 10.1109/qrs57517.2022.00015

Abstract (EN): Deep learning (DL) techniques have demonstrated potential in reasoning complex patterns of vulnerable code from high-level abstractions. Recent advancements in the area, such as the introduction of transformer-based models, like BERT, help overcome the problem of the available vulnerability detection datasets being too small to enable most DL models to capture all relevant patterns. They mitigate the challenge by leveraging knowledge from a general domain to solve problems in specific domains. In this paper, we explore different BERT-based models for multi-label classification of vulnerabilities in Java on a synthetic dataset. The models yield up to 99% in accuracy and 94% in f1-score. We remove biases in the training dataset and observe drops of up to 13% of the f1-score. We further assess the generalizability of the models on realistic samples and notice that one model, in particular, predicted unknown vulnerabilities with an f1-score of nearly 85%.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 10