Data augmentation method based on diffusion model for domain generalization

Yujun TONG; Heqing WANG; Yueheng LUO; Wenxin NING; Mandan GUAN; Wenqing YU; Keyan HUANG; Jiaxun ZHANG; Zhanyu MA

doi:10.11959/j.issn.2096-6652.202334

您当前的位置：

首页 >

文章列表页 >

Data augmentation method based on diffusion model for domain generalization

Special Topic: Diffusion Model and Artificial Intelligence Generated Content | 更新时间：2024-04-04

- Data augmentation method based on diffusion model for domain generalization
- Chinese Journal of Intelligent Science and Technology Vol. 5, Issue 3, Pages: 380-388(2023)
- 作者机构：
  
  1. 北京邮电大学人工智能学院，北京 100876
  2. 北京空间飞行器总体设计部，北京 100094
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202334
  CLC： TP39
- Revised：2023-08-10，
  
  Online First：2023-09，
  
  Published：15 September 2023
- 稿件说明：
移动端阅览
Yujun TONG, Heqing WANG, Yueheng LUO, et al. Data augmentation method based on diffusion model for domain generalization[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(3): 380-388.
DOI：

Yujun TONG, Heqing WANG, Yueheng LUO, et al. Data augmentation method based on diffusion model for domain generalization[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(3): 380-388. DOI： 10.11959/j.issn.2096-6652.202334.

摘要

域泛化是计算机视觉领域中一个重要且具有挑战性的问题，该问题源于现实场景中的数据分布偏移。在实际应用中，通常会遇到训练数据和测试数据来自不同的数据域的情况，这种数据分布的差异会导致测试时准确率下降。因此，提出了一种基于隐空间数据增广的域泛化方法，与传统图像级数据增广方法不同，该方法在隐空间中引入扩散模型，以实现对特征的精细控制和多样性生成，从而提升模型在目标域上的泛化能力。具体来说，基于分类器的隐式扩散模型在隐空间训练后可以条件生成准确且丰富的源域特征，并利用高效的采样方法加速生成增广特征。实验结果表明，新提出的方法在各种域泛化任务上取得了显著的性能提升，在真实场景中有较好的有效性和鲁棒性。该方法的创新点在于将数据增广焦点转移到隐空间级别，并引入扩散模型进行增广，为解决域泛化问题提供了一种新的思路。

Abstract

Domain generalization is an important and challenging problem in computer vision

arising from the distribution shift of real-world data.In practical applications

it is common to encounter training and testing data from different domains

and the difference in data distribution can lead to performance degradation during testing.In this paper

we propose a domain generalization method based on latent space data augmentation.Unlike traditional image-level data augmentation approaches

the method introduces a diffusion model in the latent space to achieve fine control and diversity generation of features

thereby achieving feature level data augmentation and enhancing the model's generalization ability in the target domain.Specifically

the classifier-based implicit diffusion model

trained within the latent space

can conditionally generate accurate and rich source domain features.It leverages efficient sampling techniques to expedite the generation of augmented features.Experimental results show that the method has achieved significant performance improvement in various domain generalization tasks

and has good effectiveness and robustness in real scenarios.The key innovation of this paper lies in shifting data augmentation to the latent space level and introducing the diffusion model for augmentation

providing a novel approach to address the domain generalization problem.

关键词

Keywords

references

LI D , ZHANG J S , YANG Y X , et al . Episodic training for domain generalization [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway:IEEE Press , 2019 : 1446 - 1455 .

LI D , YANG Y X , SONG Y Z , et al . Deeper,broader and artier domain generalization [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision(ICCV) . Piscataway:IEEE Press , 2017 : 5543 - 5551 .

MUANDET K , BALDUZZI D , SCHÖLKOPF B . Domain generalization via invariant feature representation [C ] // Proceedings of the 30th International Conference on Machine Learning . Piscataway:IEEE Press , 2013 .

TORRALBA A , EFROS A A . Unbiased look at dataset bias [C ] // Proceedings of the IEEE International Conference on Computer Vision . Piscataway:IEEE Press , 2011 : 1521 - 1528 .

SUN B C , SAENKO K . Deep CORAL:correlation alignment for deep domain adaptation [C ] // Proceedings of the 2016 Computer Vision–ECCV Workshops . Piscataway:IEEE Press , 2016 : 443 - 450 .

LI H L , PAN S J , WANG S Q , et al . Domain generalization with adversarial feature learning [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 5400 - 5409 .

BALAJI Y , SANKARANARAYANAN S , CHELLAPPA R . MetaReg:towards domain generalization using meta-regularization [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . New York:ACM , 2018 : 1006 - 1016 .

HO J , JAIN A , ABBEEL P . Denoising diffusion probabilistic models [C ] // Proceedings of the 34th International Conference on Neural Information Processing Systems . New York:ACM , 2020 : 6840 - 6851 .

ZHOU K Y , YANG Y X , HOSPEDALES T , et al . Deep domainadversarial image generation for domain generalisation [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 13025 - 13032 .

ZHOU K , YANG Y , QIAO Y , et al . Domain adaptive ensemble learning [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 8008 - 8018 .

VOLPI R , NAMKOONG H , SENER O , et al . Generalizing to unseen domains via adversarial data augmentation [EB ] . arXiv preprint , 2018 ,arXiv:1805.12018.

LI P , LI D , LI W , et al . A simple feature augmentation for domain generalization [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway:IEEE Press , 2021 : 8886 - 8895 .

SHORTEN C , KHOSHGOFTAAR T M . A survey on image data augmentation for deep learning [J ] . Journal of Big Data , 2019 , 6 ( 1 ): 1 - 48 .

HONARVAR NAZARI N , KOVASHKA A . Domain generalization using shape representation [C ] // Proceedings of the Computer Vision–ECCV 2020 Workshops . Cham:Springer , 2020 : 666 - 670 .

SONG Y , ERMON S . Generative modeling by estimating gradients of the data distribution [EB ] . arXiv preprint , 2019 ,arXiv:1907.05600.

ROMBACH R , BLATTMANN A , LORENZ D , et al . High-resolution image synthesis with latent diffusion models [C ] // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2022 : 10684 - 10695 .

GHIFARY M , KLEIJN W B , ZHANG M J , et al . Domain generalization for object recognition with multi-task autoencoders [C ] // Proceedings of 2015 IEEE international Conference on Computer Vision(ICCV) . Piscataway:IEEE Press , 2015 : 2551 - 2559 .

LI D , YANG Y X , SONG Y Z , et al . Learning to generalize:metalearning for domain generalization [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Piscataway:IEEE Press , 2018 .

DOU Q , CASTRO D C , KAMNITSAS K , et al . Domain generalization via model-agnostic learning of semantic features [EB ] . arXiv preprint , 2019 ,arXiv:1910.13580.

GAO B , GOUK H , YANG Y , et al . Loss function learning for domain generalization by implicit gradient [C ] // Proceedings of the 39nd International Conference on Machine Learning .[S.l.:s.n. ] , 2022 : 7002 - 7016 .

LI Y , YANG Y , ZHOU W , et al . Feature-critic networks for heterogeneous domain generalization [C ] // Proceedings of the 36nd International Conference on Machine Learning .[S.l.:s.n. ] , 2019 : 3915 - 3924 .

KHOSLA A , ZHOU T H , MALISIEWICZ T , et al . Undoing the damage of dataset bias [C ] // Proceedings of the European Conference on Computer Vision . Heidelberg:Springer , 2012 : 158 - 171 .

CARLUCCI F M , D'INNOCENTE A , BUCCI S , et al . Domain generalization by solving jigsaw puzzles [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway:IEEE Press , 2019 : 2229 - 2238 .

SOHL-DICKSTEIN J , WEISS E A , MAHESWARANATHAN N , et al . Deep unsupervised learning using nonequilibrium thermodynamics [C ] // Proceedings of the 32nd International Conference on Machine Learning . New York:ACM , 2015 : 2256 - 2265 .

KINGMA D , SALIMANS T , POOLE B , et al . Variational diffusion models [EB ] . arXiv preprint , 2021 ,arXiv:2107.00630.

KONG Z , PING W , HUANG J , et al . Diffwave:a versatile diffusion model for audio synthesis [EB ] . arXiv preprint , 2020 ,arXiv:2009.09761.

SAHARIA C , HO J , CHAN W , et al . Image super-resolution via iterative refinement [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 45 ( 4 ): 4713 - 4726 .

SINHA A , SONG J , MENG C , et al . D2c:diffusion-denoising models for few-shot conditional generation [EB ] . arXiv preprint , 2021 ,arXiv:2106.06819.

HO J , SAHARIA C , CHAN W , et al . Cascaded diffusion models for high fidelity image generation [EB ] . The Journal of Machine Learning Research . 2022 , 3 ( 1 ): 2249 - 2281 .

SONG Y , SOHL-DICKSTEIN J , KINGMA D P , et al , et al . Score-based generative modeling through stochastic differential equations [EB ] . arXiv preprint , 2020 ,arXiv:2011.13456.

DHARIWAL P , NICHOL A . Diffusion models beat GANs on image synthesis [EB ] . arXiv preprint , 2021 ,arXiv:2105.05233.

SAHARIA C , CHAN W , CHANG H W , et al . Palette:image-to-image diffusion models [C ] // ACM SIGGRAPH 2022 Conference Proceedings . New York:ACM , 2022 : 1 - 10 .

SONG J , MENG C , ERMON S , et al . Denoising diffusion implicit models [EB ] . arXiv preprint , 2020 ,arXiv:2010.02502.

LU C , ZHOU Y , BAO F , et al . DPM-solver:a fast ODE solver for diffusion probabilistic model sampling in around 10 steps [EB ] . arXiv preprint , 2022 ,arXiv:2206.00927.

MATSUURA T , HARADA T . Domain generalization using a mixture of multiple latent domains [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Piscataway:IEEE Press , 2020 : 11749 - 11756 .

CHEN K Y , ZHUANG D , CHANG J M . Discriminative adversarial domain generalization with meta-learning based cross-domain validation [J ] . Neurocomputing , 2022 , 467 : 418 - 426 .

SEGU M , TONIONI A , TOMBARI F . Batch normalization embeddings for deep domain generalization [J ] . Pattern Recognition . 2023 , 135 : 109 - 115 .

Views

647

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Cerberus: a cross-site social bot detection system based on deep learning

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：162003 Visits Today：166

⁰