基于半监督生成对抗网络的三维重建云工作室

余翀

doi:10.11959/j.issn.2096-6652.201909

您当前的位置：

首页 >

文章列表页 >

基于半监督生成对抗网络的三维重建云工作室

学术论文 | 更新时间：2024-06-05

- 基于半监督生成对抗网络的三维重建云工作室
- Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks
- 智能科学与技术学报 2019年1卷第1期页码：70-82
- 作者机构：
- 作者简介：
  
  [ "余翀（1989- ），男，复旦大学硕士研究生，英伟达半导体科技有限公司高级架构师，英特尔亚太研发有限公司高级工程师，主要研究领域为人工智能与机器学习、机器视觉与图像处理、无人机与导航系统、智能控制理论与机器人系统、物联网技术与可穿戴设备等。" ]
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.201909
  中图分类号： TP18,TP39
- 网络出版日期：2019-03，
  
  纸质出版日期：2019-03-20
- 稿件说明：
移动端阅览
余翀. 基于半监督生成对抗网络的三维重建云工作室[J]. 智能科学与技术学报, 2019,1(1):70-82.

Chong YU. Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks[J]. Chinese journal of intelligent science and technology, 2019, 1(1): 70-82.
余翀. 基于半监督生成对抗网络的三维重建云工作室[J]. 智能科学与技术学报, 2019,1(1):70-82. DOI： 10.11959/j.issn.2096-6652.201909.

Chong YU. Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks[J]. Chinese journal of intelligent science and technology, 2019, 1(1): 70-82. DOI： 10.11959/j.issn.2096-6652.201909.

摘要

由于固有的问题复杂性和计算复杂度，三维重建是计算机视觉研究和应用领域非常重要且富有挑战性的课题。目前已有的三维重建算法往往会导致重建的三维模型上存在着明显的空洞、扭曲失真或者模糊不清的部分，而基于机器学习的三维重建算法往往又只能重建简单的分离物体，并表示成三维体元形式。所以这些算法框架对于实际应用来说都还远远不够。从 2014 年起，生成对抗网络被广泛应用于半监督学习，以及产生非真实数据集的应用中。所以本文的重点是采用生产对抗网络原理，来获得高质量的三维重建效果。提出了一种新颖的半监督三维重建算法架构，命名为 SS-GAN-3D。该算法通过训练生成对抗网络模型，使其达到收敛状态，以此来迭代式地提高原始三维重建模型的质量。SS-GAN-3D 只需要将事先观测的二维图像作为弱监督样本，对于三维结构外形的先验知识或者参考观测都没有任何依赖。最终通过定性和定量实验，以及对实验结果的分析，该算法框架在 Tanks ＆amp; Temples 和 ETH3D 标准三维重建测试集上，比目前最先进的三维重建方法有明显优势。基于SS-GAN-3D算法，又提出了三维重建云工作室解决方案。

Abstract

Because of the intrinsic complexity in computation

three-dimensional (3D) reconstruction is an essential and challenging topic in computer vision research and applications.The existing methods for 3D reconstruction often produce holes

distortions and obscure parts in the reconstructed 3D models.While the 3D reconstruction algorithms based on machine learning can only reconstruct voxelized 3D models for simple isolated objects

they are not adequate for real usage.From 2014

the generative adversarial network (GAN) is widely used in generating unreal dataset and semi-supervised learning.So the focus of this paper is to achieve high quality 3D reconstruction performance by adopting GAN principle.A novel semi-supervised 3D reconstruction framework

namely SS-GAN-3D was proposed

which can iteratively improve any raw 3D reconstruction models by training the GAN models to converge.This new model only takes 2D observation images as the weak supervision

and doesn’t rely on prior knowledge of shape models or any referenced observations.Finally

through qualitative and quantitative experiments and analysis

this new method shows compelling advantages over the current state-of-the-art methods on Tanks ＆amp; Temples and ETH3D reconstruction benchmark datasets.Based on SS-GAN-3D

the 3D reconstruction studio solution was proposed.

关键词

Keywords

references

DAI A , NIEßNER M , ZOLLHÖFER M , et al . BundleFusion:real-time globally consistent 3D reconstruction using on-the-fly surface reintegration [J ] . ACM Transactions on Graphics (TOG) , 2017 , 36 ( 4 ):76a.

NEX F , REMONDINO F . UAV for 3D mapping applications:a review [J ] . Applied Geomatics , 2014 , 6 ( 1 ): 1 - 15 .

JOHNSON-ROBERSON M , BRYSON M , FRIEDMAN A , et al . High-resolution underwater robotic vision-based mapping and three‐dimensional reconstruction for archaeology [J ] . Journal of Field Robotics , 2017 , 34 ( 4 ): 625 - 643 .

GOTARDO P F U , SIMON T , SHEIKH Y , et al . Photogeometric scene flow for high-detail dynamic 3D reconstruction [C ] // Proceedings of the IEEE International Conference on Computer Vision , 2015 : 846 - 854 .

SRA M , GARRIDO-JURADO S , SCHMANDT C , et al . Procedurally generated virtual reality from 3D reconstructed physical space [C ] // Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology,ACM , 2016 : 191 - 200 .

CHOY C B , XU D , GWAK J Y , et al . 3D-R2N2:a unified approach for single and multi-view 3D object reconstruction [C ] // European conference on computer vision,Springer,Cham , 2016 : 628 - 644 .

WU J , ZHANG C , XUE T , et al . Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling [C ] // Advances in Neural Information Processing Systems , 2016 : 82 - 90 .

WHELAN T , SALAS-MORENO R F , GLOCKER B , et al . ElasticFusion:real-time dense SLAM and light source estimation [J ] . The International Journal of Robotics Research , 2016 , 35 ( 14 ): 1697 - 1716 .

ZHOU Q Y , MILLER S , KOLTUN V . Elastic fragments for dense scene reconstruction [C ] // Proceedings of the IEEE International Conference on Computer Vision , 2013 : 473 - 480 .

WU C , . Towards linear-time incremental structure from motion [C ] // 2013 International Conference on 3D Vision,IEEE , 2013 : 127 - 134 .

WU C . Visual SFM:a visual structure from motion system [J ] . 2011 .

NEWCOMBE R A , IZADI S , HILLIGES O , et al . KinectFusion:real-time dense surface mapping and tracking [C ] // Mixed and augmented reality (ISMAR),2011 10th IEEE international symposium on . IEEE , 2011 : 127 - 136 .

PILLAI S , RAMALINGAM S , LEONARD J J . High-performance and tunable stereo reconstruction [C ] // Robotics and Automation (ICRA),2016 IEEE International Conference on . IEEE , 2016 : 3188 - 3195 .

LOPER M M , BLACK M J . OpenDR:an approximate differentiable renderer [C ] // European Conference on Computer Vision,Springer,Cham , 2014 : 154 - 169 .

YU C . Steganography of digital watermark based on artificial neural networks in image communication and intellectual property protection [J ] . Neural Processing Letters , 2016 , 44 ( 2 ): 307 - 316 .

WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment:from error visibility to structural similarity [J ] . IEEE transactions on image processing , 2004 , 13 ( 4 ): 600 - 612 .

HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // Proceedings of the IEEE conference on computer vision and pattern recognition , 2016 : 770 - 778 .

GULRAJANI I , AHMED F , ARJOVSKY M , et al . Improved training of Wasserstein GANs [C ] // Advances in Neural Information Processing Systems , 2017 : 5767 - 5777 .

BOGO F , ROMERO J, , LOPERM , et al . FAUST:dataset and evaluation for 3D mesh registration [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2014 : 3794 - 3801 .

KNAPITSCH A , PARK J , ZHOU Q Y , et al . Tanks and temples:benchmarking large-scale scene reconstruction [J ] . ACM Transactions on Graphics (ToG) , 2017 , 36 ( 4 ):78.

FURUKAWA Y , PONCE J . Accurate,dense,and robust multiview stereopsis [J ] . IEEE transactions on pattern analysis and machine intelligence , 2010 , 32 ( 8 ): 1362 - 1376 .

FUHRMANN S , LANGGUTH F , GOESELE M . MVE-a multi-view reconstruction environment [C ] // GCH . 2014 : 11 - 18 .

GALLIANI S , LASINGER K , SCHINDLER K . Massively parallel multiview stereopsis by surface normal diffusion [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2015 : 873 - 881 .

SCHONBERGER J L , FRAHM J M . Structure-from-motion revisited [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 4104 - 4113 .

MOULON P , MONASSE P , PERROT R , et al . OpenMVG:open multiple view geometry [C ] // International Workshop on Reproducible Research in Pattern Recognition . Springer,Cham , 2016 : 60 - 74 .

LANGGUTH F , SUNKAVALLI K , HADAP S , et al . Shading-aware multi-view stereo [C ] // European Conference on Computer Vision . Springer,Cham , 2016 : 469 - 485 .

浏览量

269

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于半监督学习的人脸识别反欺骗方法研究

基于细节增强的MR序列转换方法