Research on method and architecture for defense assessment of artificial intelligence backdoors

XIE Tian; LI Qiang; JU Zhuoya; HAN Jiaqi; YI Ping

doi:10.11959/j.issn.2096-6652.202430

您当前的位置：

首页 >

文章列表页 >

Research on method and architecture for defense assessment of artificial intelligence backdoors

Special Column: Artificial Intelligence Trustworthiness and Governance | 更新时间：2024-11-04

- Research on method and architecture for defense assessment of artificial intelligence backdoors
- Chinese Journal of Intelligent Science and Technology Vol. 6, Issue 3, Pages: 381-393(2024)
- 作者机构：
  
  1.上海交通大学网络空间安全学院，上海 200240
  2.32178部队科技创新中心，北京，100012
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62202290)
- DOI：10.11959/j.issn.2096-6652.202430
  CLC： TP18;TP309
- Received：18 April 2024，
  
  Revised：2024-08-19，
  
  Published：15 September 2024
- 稿件说明：
移动端阅览
谢天,李强,鞠卓亚等.人工智能后门防御评估方法及其架构研究[J].智能科学与技术学报,2024,06(03):381-393.

XIE Tian,LI Qiang,JU Zhuoya,et al.Research on method and architecture for defense assessment of artificial intelligence backdoors[J].Chinese Journal of Intelligent Science and Technology,2024,06(03):381-393.
谢天,李强,鞠卓亚等.人工智能后门防御评估方法及其架构研究[J].智能科学与技术学报,2024,06(03):381-393. DOI： 10.11959/j.issn.2096-6652.202430.

XIE Tian,LI Qiang,JU Zhuoya,et al.Research on method and architecture for defense assessment of artificial intelligence backdoors[J].Chinese Journal of Intelligent Science and Technology,2024,06(03):381-393. DOI： 10.11959/j.issn.2096-6652.202430.

摘要

为了应对人工智能系统可能面临的后门攻击风险，研究人员已经开发了一系列后门防御策略。现有防御方法评估标准的多样性，使得跨方法比较成为一大挑战，因此提出了一种人工智能后门防御统一评估框架。该框架旨在为不同层面（包括数据集级别和模型级别等）的防御策略，提供一个公共的评价标准。在数据集级别，主要通过准确率来评估后门检测的有效性；而在模型级别，则主要关注攻击成功率等指标。人工智能后门防御统一评估框架，能够在相同的评价标准下，对比和分析不同后门防御方法的性能。这不仅有助于识别各方法的优势和不足，还能够提出针对性改进建议。结果表明，人工智能后门防御统一评估框架能有效地评估不同防御策略的性能，为进一步提升人工智能系统的安全性提供重要的参考依据。

Abstract

In response to the potential risk of backdoor attacks faced by artificial intelligence systems

a range of backdoor defense strategies are developed. The diversity of the evaluation criteria for existing defense method

makes cross-method comparisons a significant challenge. Hence

a unified evaluation framework base on artificial intelligence backdoors was proposed. This framework aimed to provide a common standard for evaluating different levels of defense strategies

including dataset-level and model-level defenses. Regarding the dataset-level defense strategies

the effectiveness of backdoor detection was primarily assessed through accuracy. Regarding the model-level defense strategies

focus was mainly placed on metrics such as attack success rate. By implementing unified evaluation framework

the performance of various backdoor defense methods under the same standards were compared and analyzed. This not only aids in identifying the strengths and weaknesses of each method

but also proposes targeted suggestions for improvements. The results indicate that unified evaluation framework can effectively measure the performance of different defense strategies

providing an important reference for further enhancing the security of artificial intelligence systems.

关键词

Keywords

references

LI Y J, LI Y Q, DAI X L, et al. Physical-world optical adversarial attacks on 3D face recognition[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2023: 24699-24708.

KIM S, GHOLAMI A, SHAW A, et al. Squeezeformer: an efficient Transformer for automatic speech recognition[J]. Advances in Neural Information Processing Systems, 2022, 35: 9361-9373.

WANG L Y, LYU C Y, JI T B, et al. Document-level machine translation with large language models[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2023: 16646-16661.

DU Y L, YANG M J, DAI B, et al. Learning universal policies via text-guided video generation[EB ] . arXiv preprint, 2023, arXiv: 2302.00111 .

TOUVRON H, MARTIN L, STONE K, et al. Llama 2: open foundation and fine-tuned chat models[EB ] . arXiv preprint, 2023, arXiv: 2307.09288 .

LI Y M, JIANG Y, LI Z F, et al. Backdoor learning: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(1): 5-22.

GAO Y S, DOAN B G, ZHANG Z, et al. Backdoor attacks and countermeasures on deep learning: a comprehensive review[EB ] . arXiv preprint, 2020, arXiv: 2007.10760 .

GU T Y, DOLAN-GAVITT B, GARG S. BadNets: identifying vulnerabilities in the machine learning model supply chain[EB ] . arXiv preprint, 2017, arXiv: 1708.06733 .

WANG Z T, ZHAI J, MA S Q. BppAttack: stealthy and efficient Trojan attacks against deep neural networks via image quantization and contrastive adversarial learning[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 15054-15063.

TURNER A, TSIPRAS D, MADRY A. Label-consistent backdoor attacks[EB ] . arXiv preprint, 2019, arXiv: 1912.02771 .

DAI J Z, CHEN C S, LI Y F. A backdoor attack against LSTM-based text classification systems[J]. IEEE Access, 2019, 7: 138872-138878.

QI F C, CHEN Y Y, ZHANG X R, et al. Mind the style of text! adversarial and backdoor attacks based on text style transfer[EB ] . arXiv preprint, 2021, arXiv: 2110.07139 .

KOFFAS S, XU J, CONTI M, et al. Can you hear it? backdoor attacks via ultrasonic triggers[C]//Proceedings of the 2022 ACM Workshop on Wireless Security and Machine Learning. New York: ACM Press, 2022: 57-62.

ZHAO S H, MA X J, ZHENG X, et al. Clean-label backdoor attacks on video recognition models[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 14431-14440.

XI Z H, PANG R, JI S L, et al. Graph backdoor[EB ] . arXiv preprint, 2021, arXiv: 2006.11890 .

CHEN B, CARVALHO W, BARACALDO N, et al. Detecting backdoor attacks on deep neural networks by activation clustering[EB ] . arXiv preprint, 2018, arXiv: 1811.03728 .

GAO Y S, KIM Y, DOAN B G, et al. Design and evaluation of a multi-domain Trojan detection method on deep neural networks[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 19(4): 2349-2364.

LI Y, LYU X, KOREN N, et al. Anti-backdoor learning: training clean models on poisoned data[J]. Advances in Neural Information Processing Systems, 2021, 34: 14900-14912.

LIU K, DOLAN-GAVITT B, GARG S. Fine-pruning: defending against backdooring attacks on deep neural networks[M]. Cham: Springer International Publishing, 2018: 273-294.

ZHENG R K, TANG R J, LI J Z, et al. Data-free backdoor removal based on Channel lipschitzness[M]. Berlin: Springer, 2022: 175-191.

PANG R, ZHANG Z, GAO X S, et al. TrojanZoo: towards unified, holistic, and practical evaluation of neural backdoors[EB ] .arXiv preprint, 2020, arXiv: 2012.09302 .

WU B, CHEN H, ZHANG M, et al. Backdoorbench: a comprehensive benchmark of backdoor learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 10546-10559.

CUI G, YUAN L, HE B, et al. A unified evaluation of textual backdoor learning: frameworks and benchmarks[J]. Advances in Neural Information Processing Systems, 2022, 35: 5009-5023.

谭清尹, 曾颖明, 韩叶, 等. 神经网络后门攻击研究[J]. 网络与信息安全学报, 2021, 7(3): 46-58.

TAN Q Y, ZENG Y M, HAN Y, et al. Survey on backdoor attacks targeted on neural network[J]. Chinese Journal of Network and Information Security, 2021, 7(3): 46-58.

ZENG Y, PARK W, MAO Z M, et al. Rethinking the backdoor attacks' triggers: a frequency perspective[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2021: 16453-16461.

BARNI M, KALLAS K, TONDI B. A new backdoor attack in CNNS by training set corruption without label poisoning[C]//Proceedings of the 2019 IEEE International Conference on Image Processing(ICIP). Piscataway: IEEE Press, 2019: 101-105.

LI Y Z, LI Y M, WU B Y, et al. Invisible backdoor attack with sample-specific triggers[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2021: 16443-16452.

NGUYEN A, TRAN A. WaNet: imperceptible warping-based backdoor attack[EB ] . arXiv preprint, 2021, arXiv: 2102.10369 .

KRIZHEVSKY A , HINTON G .Learning multiple layers of features from tiny images[J]. Handbook of Systemic Autoimmune Diseases, 2009, 1(4): 1-60.

STALLKAMP J, SCHLIPSING M, SALMEN J, et al. The German traffic sign recognition benchmark: a multi-class classification competition[C]//Proceedings of the 2011 International Joint Conference on Neural Networks. Piscataway: IEEE Press, 2011: 1453-1460.

HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2016: 770-778.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB ] . arXiv preprint, 2014, arXiv: 1409.1556 .

WARDEN P. Speech commands: a dataset for limited-vocabulary speech recognition[EB ] . arXiv preprint, 2018, arXiv: 1804.03209 .

ABDOLI S, CARDINAL P, KOERICH A L. End-to-end environmental sound classification using a 1D convolutional neural network[J]. Expert Systems with Applications, 2019, 136: 252-263.

CHUNG J S, NAGRANI A, ZISSERMAN A. VoxCeleb2: deep speaker recognition[EB ] . arXiv preprint, 2018, arXiv: 1806.05622 .

SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Washington: Association for Computational Linguistics, 2013: 1631-1642.

ZHANG X, ZHAO J B, LECUN Y. Character-level convolutional networks for text classification[J]. Advances in Neural Information Processing Systems, 2015(1): 649-657.

DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB ] . arXiv preprint, 2018, arXiv: 1810.04805 .

CHEN X Y, LIU C, LI B, et al. Targeted backdoor attacks on deep learning systems using data poisoning[EB ] . arXiv preprint, 2017, arXiv: 1712.05526 .

LIU Q, ZHOU T Q, CAI Z P, et al. Opportunistic backdoor attacks: exploring human-imperceptible vulnerabilities on speech recognition systems[C]//Proceedings of the 30th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2022: 2390-2398.

KOFFAS S, PAJOLA L, PICEK S, et al. Going in style: audio backdoors through stylistic transformations[C]//Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway: IEEE Press, 2023: 1-5.

LI L Y, SONG D M, LI X N, et al. Backdoor attacks on pre-trained models by layerwise weight poisoning[EB ] . arXiv preprint, 2021, arXiv: 2108.13888 .

QI F C, LI M K, CHEN Y Y, et al. Hidden killer: invisible textual backdoor attacks with syntactic trigger[EB ] . arXiv preprint, 2021, arXiv: 2105.12400 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Cerberus: a cross-site social bot detection system based on deep learning

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：161982 Visits Today：145

⁰