GPT-3-Powered Type Error Debugging: Investigating the Use of Large Language Models for Code Repair

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2023

Autores

Ribeiro, F (Autor) Outra; de Macedo, JNC (Autor) Outra; Tsushima, K (Autor) Outra; Rui Abreu (Autor) FEUP; Saraiva, J (Autor) Outra

Ata de Conferência Internacional

Título: PROCEEDINGS OF THE 16TH ACM SIGPLAN INTERNATIONAL CONFERENCE ON SOFTWARE LANGUAGE ENGINEERING, SLE 2023

Páginas: 111-124

16th ACM SIGPLAN International Conference on Software Language Engineering (SLE) as part of SPLASH Conference

Cascais, PORTUGAL, OCT 23-24, 2023

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 0 Citações

Outras Informações

ID Authenticus: P-00Z-B11

DOI: 10.1145/3623476.3623522

Abstract (EN): Type systems are responsible for assigning types to terms in programs. That way, they enforce the actions that can be taken and can, consequently, detect type errors during compilation. However, while they are able to flag the existence of an error, they often fail to pinpoint its cause or provide a helpful error message. Thus, without adequate support, debugging this kind of errors can take a considerable amount of effort. Recently, neural network models have been developed that are able to understand programming languages and perform several downstream tasks. We argue that type error debugging can be enhanced by taking advantage of this deeper understanding of the language's structure. In this paper, we present a technique that leverages GPT-3's capabilities to automatically fix type errors in OCaml programs. We perform multiple source code analysis tasks to produce useful prompts that are then provided to GPT-3 to generate potential patches. Our publicly available tool, Mentat, supports multiple modes and was validated on an existing public dataset with thousands of OCaml programs. We automatically validate successful repairs by using Quickcheck to verify which generated patches produce the same output as the user-intended fixed version, achieving a 39% repair rate. In a comparative study, Mentat outperformed two other techniques in automatically fixing ill-typed OCaml programs.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 14