A survey of image-based few-shot 3D reconstruction

Hang YU; Yanwei FU; Boyan JIANG; Xiangyang XUE

doi:10.11959/j.issn.2096-6652.202240

您当前的位置：

首页 >

文章列表页 >

A survey of image-based few-shot 3D reconstruction

Special Column: Machine Learning Methods in AI 3.0 | 更新时间：2024-06-05

- A survey of image-based few-shot 3D reconstruction
- Chinese Journal of Intelligent Science and Technology Vol. 4, Issue 4, Pages: 544-559(2022)
- 作者机构：
  
  1. 复旦大学工程与应用技术研究院，上海 200082
  2. 复旦大学大数据学院，上海 200082
  3. 复旦大学计算机科学技术学院，上海 200082
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202240
  CLC： TP391.41
- Published Online：2022-12，
  
  Published：15 December 2022
- 稿件说明：
移动端阅览
Hang YU, Yanwei FU, Boyan JIANG, et al. A survey of image-based few-shot 3D reconstruction[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 544-559.
DOI：

Hang YU, Yanwei FU, Boyan JIANG, et al. A survey of image-based few-shot 3D reconstruction[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 544-559. DOI： 10.11959/j.issn.2096-6652.202240.

摘要

基于少量图像的三维重建被认为是第三代人工智能的经典应用之一。在计算机图形学和计算机视觉领域，基于少量图像的三维重建任务因具有广泛的应用场景和很高的研究价值，长期以来吸引着众多学者的目光。引入深度学习方法后，该领域于近年来得到了长足发展。对此类基于少量图像的三维重建任务进行了全面阐述，并介绍了本研究组在该方面的系列工作，对其中涉及的数据类型进行分析，阐明其适用性和一般处理方法。此外，对常见的数据集进行分析、整理，针对不同重建方法，归纳出其基本框架、思路。最后，展示了一些常见三维重建的代表性实验结果，并提出了未来可能的研究方向。

Abstract

Few-shot 3D reconstruction is considered one of the classic applications of the third generation of artificial intelligence.In the area of computer graphics and computer vision

few-shot 3D reconstruction has attracted the attention of many researchers during the past several decades because of its wide application scenarios and high research value.The area has grown significantly in recent years after the introduction of deep learning methods.The state-of-the-art methods in image-based few-shot 3D reconstruction were reviewed comprehensively and the series of works of our research group were introduced.The various 3D data types were introduced

and their applicability and general processing procedures in 3D reconstruction were discussed.Furthermore

the most widely used datasets were categorized.Finally

some representative experimental results of common 3D reconstructions were presented

and potential future research directions were proposed.

关键词

Keywords

references

CHANG A X , FUNKHOUSER T , GUIBAS L , et al . ShapeNet:an information-rich 3D model repository [J ] . arXiv preprint , 2015 ,arXiv:1512.03012.

TATARCHENKO M , RICHTER S R , RANFTL R , et al . What do single-view 3D reconstruction networks learn? [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 3400 - 3409 .

DENG J , DONG W , SOCHER R , et al . ImageNet:a large-scale hierarchical image database [C ] // Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2009 : 248 - 255 .

WANG W , YANG Y J , WANG X , et al . Development of convolutional neural network and its application in image classification:a survey [J ] . Optical Engineering , 2019 ,58:040901.

BARRON J L , FLEET D J , BEAUCHEMIN S S . Performance of optical flow techniques [J ] . International Journal of Computer Vision , 1994 , 12 ( 1 ): 43 - 77 .

ULLMAN S . The interpretation of structure from motion [J ] . The Interpretation of Visual Motion , 1979 : 133 - 175 .

LORENSEN W E , CLINE H E . Marching cubes:a high resolution 3D surface construction algorithm [J ] . ACM SIGGRAPH Computer Graphics , 1987 , 21 ( 4 ): 163 - 169 .

LEWINER T , LOPES H , VIEIRA A W , et al . Efficient implementation of marching cubes’ cases with topological guarantees [J ] . Journal of Graphics Tools , 2003 , 8 ( 2 ): 1 - 15 .

PARK J J , FLORENCE P , STRAUB J , et al . DeepSDF:learning continuous signed distance functions for shape representation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 165 - 174 .

CHOY C B , XU D F , GWAK J , et al . 3D-R2N2:an unified approach for single and multi-view 3D object reconstruction [C ] // Proceedings of 2016 European Conference on Computer Vision - ECCV 2016 . Cham:Springer , 2016 : 628 - 644 .

FAN H Q , SU H , GUIBAS L . A point set generation network for 3D object reconstruction from a single image [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 2463 - 2471 .

CHARLES R Q , HAO S , MO K C , et al . PointNet:deep learning on point sets for 3D classification and segmentation [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 77 - 85 .

QI C R , YI L , SU H , et al . PointNet++:deep hierarchical feature learning on point sets in a metric space [J ] . arXiv preprint , 2017 ,arXiv:1706

WANG N Y , ZHANG Y D , LI Z W , et al . Pixel2Mesh:generating 3D mesh models from single RGB images [C ] // Proceedings of 2018 European Conference on Computer Vision - ECCV 2018 . Cham:Springer , 2018 : 55 - 71 .

WEN C , ZHANG Y D , LI Z W , et al . Pixel2Mesh++:multi-view 3D mesh generation via deformation [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 1042 - 1051 .

KATO H , USHIKU Y , HARADA T . Neural 3D mesh renderer [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 3907 - 3916 .

GROUEIX T , FISHER M , KIM V G , et al . A papier-mache approach to learning 3D surface generation [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 216 - 224 .

MESCHEDER L , OECHSLE M , NIEMEYER M , et al . Occupancy networks:learning 3D reconstruction in function space [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4455 - 4465 .

XIANG Y , MOTTAGHI R , SAVARESE S . Beyond PASCAL:a benchmark for 3D object detection in the wild [C ] // Proceedings of IEEE Winter Conference on Applications of Computer Vision . Piscataway:IEEE Press , 2014 : 75 - 82 .

SUN X Y , WU J J , ZHANG X M , et al . Pix3D:dataset and methods for single-image 3D shape modeling [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 2974 - 2983 .

XIANG Y , KIM W , CHEN W , et al . ObjectNet3D:a large scale database for 3D object recognition [C ] // Proceedings of 2016 European Conference on Computer Vision - ECCV 2016 . Cham:Springer , 2016 : 160 - 176 .

GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite [C ] // Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2012 : 3354 - 3361 .

KANAZAWA A , BLACK M J , JACOBS D W , et al . End-to-end recovery of human shape and pose [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 7122 - 7131 .

ZHU H , ZUO X X , WANG S , et al . Detailed human shape estimation from a single image by hierarchical mesh deformation [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4486 - 4495 .

KOLOTOUROS N , PAVLAKOS G , DANIILIDIS K . Convolutional mesh regression for single-image human shape reconstruction [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 4496 - 4505 .

SAITO S , HUANG Z , NATSUME R , et al . PIFu:pixel-aligned implicit function for high-resolution clothed human digitization [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 2304 - 2314 .

LIANG J B , LIN M . Shape-aware human pose and shape reconstruction using multi-view images [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 4351 - 4361 .

YU H , CHEANG C , FU Y W , et al . Multi-view shape generation for 3D human-like body [J ] . ACM Transactions on Multimedia Computing,Communications,and Applications , 2022 .

WANG J S , WEN C , FU Y W , et al . Neural pose transfer by spatially adaptive instance normalization [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 5830 - 5838 .

XU C M , FU Y W , WEN C , et al . Pose-guided person image synthesis in the non-iconic views [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 9060 - 9072 .

JIANG B Y , ZHANG Y D , WEI X K , et al . Learning compositional representation for 4D captures with neural ODE [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 5336 - 5346 .

CHEN R T Q , RUBANOVA Y , BETTENCOURT J , et al . Neural ordinary differential equations [J ] . arXiv preprint , 2018 ,arxiv:18066.

JIANG B Y , ZHANG Y D , WEI X K , et al . H4D:human 4D modeling by learning neural compositional representation [J ] . arXiv preprint , 2022 ,arXiv:2203.01247.

CHO K , MERRIENBOER B V , BAHDANAU D , et al . On the properties of neural machine translation:encoder-decoder approaches [C ] // Proceedings of the 8th Workshop on Syntax,Semantics and Structure in Statistical Translation .[S.l.:s.n. ] , 2014 : 103 - 111 .

JOHNSON S , EVERINGHAM M . Clustered pose and nonlinear appearance models for human pose estimation [C ] // Proceedings of the British Machine Vision Conference 2010 .[S.l.:s.n. ] , 2010 .

SIGAL L , BALAN A O , BLACK M J . HumanEva:synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion [J ] . International Journal of Computer Vision , 2009 , 87 ( 1/2 ): 4 - 27 .

IONESCU C , PAPAVA D , OLARU V , et al . Human3.6M:large scale datasets and predictive methods for 3D human sensing in natural environments [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 36 ( 7 ): 1325 - 1339 .

GUO K W , XU F , YU T , et al . Real-time geometry,albedo and motion reconstruction using a single RGBD camera [J ] . ACM Transactions on Graphics , 2017 , 36 ( 4 ).

XIAO C W , YANG D W , LI B , et al . MeshAdv:adversarial meshes for visual recognition [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 6891 - 6900 .

Views

623

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

3D edge reconstruction method based on tactile sensing and servo

Related Author

Jingyi HU

Shaowei CUI

Chaofan ZHANG

Boyue ZHANG

Shuo WANG

Related Institution

The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

School of Future Technology, University of Chinese Academy of Sciences

School of Artificial Intelligence, University of Chinese Academy of Sciences

Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35702 Visits Today：177

⁰