- 个人中心
- 退出登录
浏览全部资源
扫码关注微信
Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks
- Chinese Journal of Intelligent Science and Technology Vol. 1, Issue 1, Pages: 70-82(2019)
- 作者机构：
- 作者简介：
- 基金信息：
DOI：10.11959/j.issn.2096-6652.201909
CLC： TP18,TP39-
Published Online：2019-03，
Published：20 March 2019
- 稿件说明：
移动端阅览
Chong YU. Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks[J]. Chinese journal of intelligent science and technology, 2019, 1(1): 70-82.
Chong YU. Three-dimensional reconstruction cloud studio based on semi-supervised generative adversarial networks[J]. Chinese journal of intelligent science and technology, 2019, 1(1): 70-82. DOI： 10.11959/j.issn.2096-6652.201909.
摘要
由于固有的问题复杂性和计算复杂度，三维重建是计算机视觉研究和应用领域非常重要且富有挑战性的课题。目前已有的三维重建算法往往会导致重建的三维模型上存在着明显的空洞、扭曲失真或者模糊不清的部分，而基于机器学习的三维重建算法往往又只能重建简单的分离物体，并表示成三维体元形式。所以这些算法框架对于实际应用来说都还远远不够。从 2014 年起，生成对抗网络被广泛应用于半监督学习，以及产生非真实数据集的应用中。所以本文的重点是采用生产对抗网络原理，来获得高质量的三维重建效果。提出了一种新颖的半监督三维重建算法架构，命名为 SS-GAN-3D。该算法通过训练生成对抗网络模型，使其达到收敛状态，以此来迭代式地提高原始三维重建模型的质量。SS-GAN-3D 只需要将事先观测的二维图像作为弱监督样本，对于三维结构外形的先验知识或者参考观测都没有任何依赖。最终通过定性和定量实验，以及对实验结果的分析，该算法框架在 Tanks ＆amp; Temples 和 ETH3D 标准三维重建测试集上，比目前最先进的三维重建方法有明显优势。基于SS-GAN-3D算法，又提出了三维重建云工作室解决方案。
Abstract
Because of the intrinsic complexity in computation
three-dimensional (3D) reconstruction is an essential and challenging topic in computer vision research and applications.The existing methods for 3D reconstruction often produce holes
distortions and obscure parts in the reconstructed 3D models.While the 3D reconstruction algorithms based on machine learning can only reconstruct voxelized 3D models for simple isolated objects
they are not adequate for real usage.From 2014
the generative adversarial network (GAN) is widely used in generating unreal dataset and semi-supervised learning.So the focus of this paper is to achieve high quality 3D reconstruction performance by adopting GAN principle.A novel semi-supervised 3D reconstruction framework
namely SS-GAN-3D was proposed
which can iteratively improve any raw 3D reconstruction models by training the GAN models to converge.This new model only takes 2D observation images as the weak supervision
and doesn’t rely on prior knowledge of shape models or any referenced observations.Finally
through qualitative and quantitative experiments and analysis
this new method shows compelling advantages over the current state-of-the-art methods on Tanks ＆amp; Temples and ETH3D reconstruction benchmark datasets.Based on SS-GAN-3D
the 3D reconstruction studio solution was proposed.
关键词
Keywords
references
DAI A , NIEßNER M , ZOLLHÖFER M , et al . BundleFusion:real-time globally consistent 3D reconstruction using on-the-fly surface reintegration [J ] . ACM Transactions on Graphics (TOG) , 2017 , 36 ( 4 ):76a.
NEX F , REMONDINO F . UAV for 3D mapping applications:a review [J ] . Applied Geomatics , 2014 , 6 ( 1 ): 1 - 15 .
JOHNSON-ROBERSON M , BRYSON M , FRIEDMAN A , et al . High-resolution underwater robotic vision-based mapping and three‐dimensional reconstruction for archaeology [J ] . Journal of Field Robotics , 2017 , 34 ( 4 ): 625 - 643 .
GOTARDO P F U , SIMON T , SHEIKH Y , et al . Photogeometric scene flow for high-detail dynamic 3D reconstruction [C ] // Proceedings of the IEEE International Conference on Computer Vision , 2015 : 846 - 854 .
SRA M , GARRIDO-JURADO S , SCHMANDT C , et al . Procedurally generated virtual reality from 3D reconstructed physical space [C ] // Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology,ACM , 2016 : 191 - 200 .
CHOY C B , XU D , GWAK J Y , et al . 3D-R2N2:a unified approach for single and multi-view 3D object reconstruction [C ] // European conference on computer vision,Springer,Cham , 2016 : 628 - 644 .
WU J , ZHANG C , XUE T , et al . Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling [C ] // Advances in Neural Information Processing Systems , 2016 : 82 - 90 .
WHELAN T , SALAS-MORENO R F , GLOCKER B , et al . ElasticFusion:real-time dense SLAM and light source estimation [J ] . The International Journal of Robotics Research , 2016 , 35 ( 14 ): 1697 - 1716 .
ZHOU Q Y , MILLER S , KOLTUN V . Elastic fragments for dense scene reconstruction [C ] // Proceedings of the IEEE International Conference on Computer Vision , 2013 : 473 - 480 .
WU C , . Towards linear-time incremental structure from motion [C ] // 2013 International Conference on 3D Vision,IEEE , 2013 : 127 - 134 .
WU C . Visual SFM:a visual structure from motion system [J ] . 2011 .
NEWCOMBE R A , IZADI S , HILLIGES O , et al . KinectFusion:real-time dense surface mapping and tracking [C ] // Mixed and augmented reality (ISMAR),2011 10th IEEE international symposium on . IEEE , 2011 : 127 - 136 .
PILLAI S , RAMALINGAM S , LEONARD J J . High-performance and tunable stereo reconstruction [C ] // Robotics and Automation (ICRA),2016 IEEE International Conference on . IEEE , 2016 : 3188 - 3195 .
LOPER M M , BLACK M J . OpenDR:an approximate differentiable renderer [C ] // European Conference on Computer Vision,Springer,Cham , 2014 : 154 - 169 .
YU C . Steganography of digital watermark based on artificial neural networks in image communication and intellectual property protection [J ] . Neural Processing Letters , 2016 , 44 ( 2 ): 307 - 316 .
WANG Z , BOVIK A C , SHEIKH H R , et al . Image quality assessment:from error visibility to structural similarity [J ] . IEEE transactions on image processing , 2004 , 13 ( 4 ): 600 - 612 .
HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition [C ] // Proceedings of the IEEE conference on computer vision and pattern recognition , 2016 : 770 - 778 .
GULRAJANI I , AHMED F , ARJOVSKY M , et al . Improved training of Wasserstein GANs [C ] // Advances in Neural Information Processing Systems , 2017 : 5767 - 5777 .
BOGO F , ROMERO J, , LOPERM , et al . FAUST:dataset and evaluation for 3D mesh registration [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2014 : 3794 - 3801 .
KNAPITSCH A , PARK J , ZHOU Q Y , et al . Tanks and temples:benchmarking large-scale scene reconstruction [J ] . ACM Transactions on Graphics (ToG) , 2017 , 36 ( 4 ):78.
FURUKAWA Y , PONCE J . Accurate,dense,and robust multiview stereopsis [J ] . IEEE transactions on pattern analysis and machine intelligence , 2010 , 32 ( 8 ): 1362 - 1376 .
FUHRMANN S , LANGGUTH F , GOESELE M . MVE-a multi-view reconstruction environment [C ] // GCH . 2014 : 11 - 18 .
GALLIANI S , LASINGER K , SCHINDLER K . Massively parallel multiview stereopsis by surface normal diffusion [C ] // Proceedings of the IEEE International Conference on Computer Vision . 2015 : 873 - 881 .
SCHONBERGER J L , FRAHM J M . Structure-from-motion revisited [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2016 : 4104 - 4113 .
MOULON P , MONASSE P , PERROT R , et al . OpenMVG:open multiple view geometry [C ] // International Workshop on Reproducible Research in Pattern Recognition . Springer,Cham , 2016 : 60 - 74 .
LANGGUTH F , SUNKAVALLI K , HADAP S , et al . Shading-aware multi-view stereo [C ] // European Conference on Computer Vision . Springer,Cham , 2016 : 469 - 485 .
0
Views
269
下载量
0
CSCD
- L-PDFDownload
- XMLDownload
- Meta-XMLDownload
- BibTeXDownload
- EndnoteDownload
- RefMan Download
- NoteExpressDownload
- NoteFirstDownload
Publicity Resources
Related Articles
Related Author
Related Institution
- Postal code：100079
- Tel：（010）53879206 Email：cjist@bjxintong.com.cn
- Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19
京公网安备11010802024621 - It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
- Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
- Total Visits：35641 Visits Today：116
