Deformable convolutions in multi-view stereo

• Type: Article in International Scientific Journal
• Date: 2022

Authors

Masson, JEN (Author) Other; Marcelo Petry (Author) FEUP; Coutinho, DF (Author) Other; Honorio, LD (Author) Other

Journal

Title: Image and Vision Computing

Vol. 118

ISSN: 0262-8856

Publisher: Elsevier

Indexing

ISI Web of Knowledge - 5 Citations

Scopus - 7 Citations

Other information

Authenticus ID: P-00V-YE6

DOI: 10.1016/j.imavis.2021.104369

Abstract (EN): The Multi-View Stereo (MVS) is a key process in the photogrammetry workflow. It is responsible for taking the camera's views and finding the maximum number of matches between the images yielding a dense point cloud of the observed scene. Since this process is based on the matching between images it greatly depends on the abil-ity of features matching throughout different images. To improve the matching performance several researchers have proposed the use of Convolutional Neural Networks (CNNs) to solve the MVS problem. Despite the progress in the MVS problem with the usage of CNNs, the Video RAM (VRAM) consumption within these approaches is usually far greater than classical methods, that rely more on RAM, which is cheaper to expand than VRAM. This work then follows the progress made in CasMVSNet in the reduction of GPU memory usage, and further study the changes in the feature extraction process. The Average Group-wise Correlation is used in the cost vol-ume generation, to reduce the number of channels in the cost volume, yielding a reduction in GPU memory usage without noticeable penalties in the result. The deformable convolutions are applied in the feature extraction net -work to augment the spatial sampling locations with learning offsets, without additional supervision, to further improve the network's ability to model transformations. The impact of these changes is measured using quanti-tative and qualitative tests using the DTU and the Tanks and Temples datasets. The modifications reduced the GPU memory usage by 32% and improved the completeness by 9% with a penalty of 6.6% in accuracy on the DTU dataset.(c) 2021 Published by Elsevier B.V.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 7