基于少量图像的三维重建综述

于航; 付彦伟; 姜柏言; 薛向阳

doi:10.11959/j.issn.2096-6652.202240

您当前的位置：

首页 >

文章列表页 >

基于少量图像的三维重建综述

专栏：人工智能3.0中的机器学习方法 | 更新时间：2024-06-05

- 基于少量图像的三维重建综述
- A survey of image-based few-shot 3D reconstruction
- 智能科学与技术学报 2022年4卷第4期页码：544-559
- 作者机构：
  
  1. 复旦大学工程与应用技术研究院，上海 200082
  2. 复旦大学大数据学院，上海 200082
  3. 复旦大学计算机科学技术学院，上海 200082
- 作者简介：
  
  [ "于航（1991− ），男，复旦大学工程与应用技术研究院博士生，主要研究方向为3D视觉、人体重建" ]
  [ "付彦伟（1986− ），男，博士，复旦大学大数据学院研究员、博士生导师，主要研究方向为少量样本学习、稀疏化学习算法、图像修复等" ]
  [ "姜柏言（1996− ），男，复旦大学计算机科学技术学院博士生，主要研究方向为三维/四维人体表示与重建" ]
  [ "薛向阳（1968− ），男，博士，复旦大学计算机科学技术学院教授、博士生导师，主要研究方向为计算机视觉、多媒体内容识别和类脑智能系统等" ]
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202240
  中图分类号： TP391.41
- 网络出版日期：2022-12，
  
  纸质出版日期：2022-12-15
- 稿件说明：
移动端阅览
于航, 付彦伟, 姜柏言, 等. 基于少量图像的三维重建综述[J]. 智能科学与技术学报, 2022,4(4):544-559.

Hang YU, Yanwei FU, Boyan JIANG, et al. A survey of image-based few-shot 3D reconstruction[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 544-559.
于航, 付彦伟, 姜柏言, 等. 基于少量图像的三维重建综述[J]. 智能科学与技术学报, 2022,4(4):544-559. DOI： 10.11959/j.issn.2096-6652.202240.

Hang YU, Yanwei FU, Boyan JIANG, et al. A survey of image-based few-shot 3D reconstruction[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 544-559. DOI： 10.11959/j.issn.2096-6652.202240.

摘要

基于少量图像的三维重建被认为是第三代人工智能的经典应用之一。在计算机图形学和计算机视觉领域，基于少量图像的三维重建任务因具有广泛的应用场景和很高的研究价值，长期以来吸引着众多学者的目光。引入深度学习方法后，该领域于近年来得到了长足发展。对此类基于少量图像的三维重建任务进行了全面阐述，并介绍了本研究组在该方面的系列工作，对其中涉及的数据类型进行分析，阐明其适用性和一般处理方法。此外，对常见的数据集进行分析、整理，针对不同重建方法，归纳出其基本框架、思路。最后，展示了一些常见三维重建的代表性实验结果，并提出了未来可能的研究方向。

Abstract

Few-shot 3D reconstruction is considered one of the classic applications of the third generation of artificial intelligence.In the area of computer graphics and computer vision

few-shot 3D reconstruction has attracted the attention of many researchers during the past several decades because of its wide application scenarios and high research value.The area has grown significantly in recent years after the introduction of deep learning methods.The state-of-the-art methods in image-based few-shot 3D reconstruction were reviewed comprehensively and the series of works of our research group were introduced.The various 3D data types were introduced

and their applicability and general processing procedures in 3D reconstruction were discussed.Furthermore

the most widely used datasets were categorized.Finally

some representative experimental results of common 3D reconstructions were presented

and potential future research directions were proposed.

关键词

Keywords

references

CHANG A X , FUNKHOUSER T , GUIBAS L , et al . ShapeNet:an information-rich 3D model repository [J ] . arXiv preprint , 2015 ,arXiv:1512.03012.

TATARCHENKO M , RICHTER S R , RANFTL R , et al . What do single-view 3D reconstruction networks learn? [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 3400 - 3409 .

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2009 : 248 - 255 .

WANG W , YANG Y J , WANG X , et al . Development of convolutional neural network and its application in image classification:a survey [J ] . Optical Engineering , 2019 ,58:040901.

BARRON J L , FLEET D J , BEAUCHEMIN S S . Performance of optical flow techniques [J ] . International Journal of Computer Vision , 1994 , 12 ( 1 ): 43 - 77 .

ULLMAN S . The interpretation of structure from motion [J ] . The Interpretation of Visual Motion , 1979 : 133 - 175 .

LORENSEN W E , CLINE H E . Marching cubes:a high resolution 3D surface construction algorithm [J ] . ACM SIGGRAPH Computer Graphics , 1987 , 21 ( 4 ): 163 - 169 .

LEWINER T , LOPES H , VIEIRA A W , et al . Efficient implementation of marching cubes’ cases with topological guarantees [J ] . Journal of Graphics Tools , 2003 , 8 ( 2 ): 1 - 15 .

PARK J J , FLORENCE P , STRAUB J , et al . DeepSDF:learning continuous signed distance functions for shape representation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 165 - 174 .

CHOY C B , XU D F , GWAK J , et al . 3D-R2N2:an unified approach for single and multi-view 3D object reconstruction [C ] // Proceedings of 2016 European Conference on Computer Vision - ECCV 2016 . Cham:Springer , 2016 : 628 - 644 .

FAN H Q , SU H , GUIBAS L . A point set generation network for 3D object reconstruction from a single image [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 2463 - 2471 .

CHARLES R Q , HAO S , MO K C , et al . PointNet:deep learning on point sets for 3D classification and segmentation [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 77 - 85 .

QI C R , YI L , SU H , et al . PointNet++:deep hierarchical feature learning on point sets in a metric space [J ] . arXiv preprint , 2017 ,arXiv:1706

WANG N Y , ZHANG Y D , LI Z W , et al . Pixel2Mesh:generating 3D mesh models from single RGB images [C ] // Proceedings of 2018 European Conference on Computer Vision - ECCV 2018 . Cham:Springer , 2018 : 55 - 71 .

WEN C , ZHANG Y D , LI Z W , et al . Pixel2Mesh++:multi-view 3D mesh generation via deformation [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 1042 - 1051 .

KATO H , USHIKU Y , HARADA T . Neural 3D mesh renderer [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 3907 - 3916 .

GROUEIX T , FISHER M , KIM V G , et al . A papier-mache approach to learning 3D surface generation [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 216 - 224 .

MESCHEDER L , OECHSLE M , NIEMEYER M , et al . Occupancy networks:learning 3D reconstruction in function space [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4455 - 4465 .

XIANG Y , MOTTAGHI R , SAVARESE S . Beyond PASCAL:a benchmark for 3D object detection in the wild [C ] // Proceedings of IEEE Winter Conference on Applications of Computer Vision . Piscataway:IEEE Press , 2014 : 75 - 82 .

SUN X Y , WU J J , ZHANG X M , et al . Pix3D:dataset and methods for single-image 3D shape modeling [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 2974 - 2983 .

XIANG Y , KIM W , CHEN W , et al . ObjectNet3D:a large scale database for 3D object recognition [C ] // Proceedings of 2016 European Conference on Computer Vision - ECCV 2016 . Cham:Springer , 2016 : 160 - 176 .

GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite [C ] // Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2012 : 3354 - 3361 .

KANAZAWA A , BLACK M J , JACOBS D W , et al . End-to-end recovery of human shape and pose [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 7122 - 7131 .

ZHU H , ZUO X X , WANG S , et al . Detailed human shape estimation from a single image by hierarchical mesh deformation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4486 - 4495 .

KOLOTOUROS N , PAVLAKOS G , DANIILIDIS K . Convolutional mesh regression for single-image human shape reconstruction [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4496 - 4505 .

SAITO S , HUANG Z , NATSUME R , et al . PIFu:pixel-aligned implicit function for high-resolution clothed human digitization [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 2304 - 2314 .

LIANG J B , LIN M . Shape-aware human pose and shape reconstruction using multi-view images [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 4351 - 4361 .

YU H , CHEANG C , FU Y W , et al . Multi-view shape generation for 3D human-like body [J ] . ACM Transactions on Multimedia Computing,Communications,and Applications , 2022 .

WANG J S , WEN C , FU Y W , et al . Neural pose transfer by spatially adaptive instance normalization [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 5830 - 5838 .

XU C M , FU Y W , WEN C , et al . Pose-guided person image synthesis in the non-iconic views [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 9060 - 9072 .

JIANG B Y , ZHANG Y D , WEI X K , et al . Learning compositional representation for 4D captures with neural ODE [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 5336 - 5346 .

CHEN R T Q , RUBANOVA Y , BETTENCOURT J , et al . Neural ordinary differential equations [J ] . arXiv preprint , 2018 ,arxiv:18066.

JIANG B Y , ZHANG Y D , WEI X K , et al . H4D:human 4D modeling by learning neural compositional representation [J ] . arXiv preprint , 2022 ,arXiv:2203.01247.

CHO K , MERRIENBOER B V , BAHDANAU D , et al . On the properties of neural machine translation:encoder-decoder approaches [C ] // Proceedings of the 8th Workshop on Syntax,Semantics and Structure in Statistical Translation .[S.l.:s.n. ] , 2014 : 103 - 111 .

JOHNSON S , EVERINGHAM M . Clustered pose and nonlinear appearance models for human pose estimation [C ] // Proceedings of the British Machine Vision Conference 2010 .[S.l.:s.n. ] , 2010 .

SIGAL L , BALAN A O , BLACK M J . HumanEva:synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion [J ] . International Journal of Computer Vision , 2009 , 87 ( 1/2 ): 4 - 27 .

IONESCU C , PAPAVA D , OLARU V , et al . Human3.6M:large scale datasets and predictive methods for 3D human sensing in natural environments [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 36 ( 7 ): 1325 - 1339 .

GUO K W , XU F , YU T , et al . Real-time geometry,albedo and motion reconstruction using a single RGBD camera [J ] . ACM Transactions on Graphics , 2017 , 36 ( 4 ).

XIAO C W , YANG D W , LI B , et al . MeshAdv:adversarial meshes for visual recognition [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 6891 - 6900 .

浏览量

623

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于触觉感知和伺服的物体三维边缘重建方法