Optimizing Medical Image Captioning with Conditional Prompt Encoding

• Type: Article in International Conference Proceedings Book
• Date: 2026

Authors

Fernandes, RF (Author) Other; Oliveira, HS (Author) Other; Pedro Ribeiro (Author) FCUP; Oliveira, HP (Author) FCUP

Conference proceedings International

Title: PATTERN RECOGNITION AND IMAGE ANALYSIS, IBPRIA 2025, PT II

Pages: 196-207

12th Iberian Conference on Pattern Recognition and Image Analysis-IbPRIA

Coimbra, PORTUGAL, JUN 30-JUL 03, 2025

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-019-TXA

DOI: 10.1007/978-3-031-99568-2_16

Abstract (EN): Medical image captioning is an essential tool to produce descriptive text reports of medical images. One of the central problems of medical image captioning is their poor domain description generation because large pre-trained language models are primarily trained in non-medical text domains with different semantics of medical text. To overcome this limitation, we explore improvements in contrastive learning for X-ray images complemented with soft prompt engineering for medical image captioning and conditional text decoding for caption generation. The main objective is to develop a softprompt model to improve the accuracy and clinical relevance of the automatically generated captions while guaranteeing their complete linguistic accuracy without corrupting the models' performance. Experiments on the MIMIC-CXR and ROCO datasets showed that the inclusion of tailored soft-prompts improved accuracy and efficiency, while ensuring a more cohesive medical context for captions, aiding medical diagnosis and encouraging more accurate reporting.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 12