End-to-End Deep Sketch-to-Photo Matching Enforcing Realistic Photo Generation

• Type: Article in International Conference Proceedings Book
• Date: 2021

Authors

Capozzi, L (Author) Other; Pinto, JR (Author) Other; Jaime S Cardoso (Author) FEUP; Rebelo, A (Author) Other

Conference proceedings International

Title: CIARP

Pages: 451-460

Indexing

Scopus - 0 Citations

Other information

Authenticus ID: P-00V-YE0

DOI: 10.1007/978-3-030-93420-0_42

Abstract (EN): The traditional task of locating suspects using forensic sketches posted on public spaces, news, and social media can be a difficult task. Recent methods that use computer vision to improve this process present limitations, as they either do not use end-to-end networks for sketch recognition in police databases (which generally improve performance) or/and do not offer a photo-realistic representation of the sketch that could be used as alternative if the automatic matching process fails. This paper proposes a method that combines these two properties, using a conditional generative adversarial network (cGAN) and a pre-trained face recognition network that are jointly optimised as an end-to-end model. While the model can identify a short list of potential suspects in a given database, the cGAN offers an intermediate realistic face representation to support an alternative manual matching process. Evaluation on sketch-photo pairs from the CUFS, CUFSF and CelebA databases reveal the proposed method outperforms the state-of-the-art in most tasks, and that forcing an intermediate photo-realistic representation only results in a small performance decrease.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 9