DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证

娄云洁; 艾明飞; 庄术洁; 于海; 王鑫; 滕出; 王江成; 沈甜雨; 郝坤坤; 崔文

doi:10.11959/j.issn.2096-6652.202509

您当前的位置：

首页 >

文章列表页 >

DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证

专栏：具身智能与特种机器人 | 更新时间：2025-06-10

- DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证
- DK-Port: construction and validation of port autonomous driving simulation environment based on large language models and reinforcement learning
- 智能科学与技术学报 2025年7卷第1期页码：98-113
- 作者机构：
  
  1.青岛前湾联合集装箱码头有限责任公司，山东青岛 266520
  2.安徽深信科创信息技术有限公司，安徽合肥 230088
  3.北京化工大学信息科学与技术学院，北京 100029
  4.交叉信息核心技术研究院（西安）有限公司，陕西西安 710077
  5.西安电子科技大学前沿交叉研究院，陕西西安 710071
- 作者简介：
  
  [ "娄云洁（1971- ），男，青岛前湾联合集装箱码头有限责任公司总经理，主要研究方向为集装箱码头的运营和管理。" ]
  [ "艾明飞（1980- ），男，青岛前湾联合集装箱码头有限责任公司副总经理，主要研究方向为码头大型设备管理、绿色港口建设等。" ]
  [ "庄术洁（1992- ），男，青岛前湾联合集装箱码头有限责任公司自动化研创中心数字化部主任，主要研究方向为智慧港口建设、智能驾驶及自动化等。" ]
  [ "于海（1972- ），男，青岛前湾联合集装箱码头有限责任公司自动化研创中心主任，主要研究方向为传统集装箱码头转型升级全场景自动化、无人化关键技术研究与应用等。" ]
  [ "王鑫（1991- ），男，青岛前湾联合集装箱码头有限责任公司自动化研创中心智能设备部主任，主要研究方向为电控系统。" ]
  [ "滕出（2001- ），男，青岛前湾联合集装箱码头有限责任公司自动化研创中心数字化工程师，主要研究方向为自动化轨道吊、无人集卡技术等。" ]
  [ "王江成（1997- ），男，安徽深信科创信息技术有限公司算法工程师，主要研究方向为自动驾驶仿真测试、对抗场景生成、智能体训练等。" ]
  [ "沈甜雨（1996- ），女，博士，北京化工大学信息科学与技术学院副教授，主要研究方向为机器视觉、智能感知、智能机器人系统等。" ]
  [ "郝坤坤（1991- ），男，安徽深信科创信息技术有限公司高级算法工程师，主要研究方向为自动驾驶预期功能安全场景生成、对抗场景生成、仿真交通流构建等。" ]
  [ "崔文（1989- ），男，博士，安徽深信科创信息技术有限公司博士后，主要研究方向为智能网联技术，如安全关键测试场景生成、车辆通信网络优化和车辆状态感知等方向。" ]
- 基金信息：
  
  西安交通大学人机混合增强智能全国重点实验室开放课题(HMHAI-202416)
- DOI：10.11959/j.issn.2096-6652.202509
  中图分类号： TP39
- 收稿日期：2024-08-05，
  
  修回日期：2024-10-22，
  
  纸质出版日期：2025-03-15
- 稿件说明：
移动端阅览
娄云洁,艾明飞,庄术洁等.DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证[J].智能科学与技术学报,2025,07(01):98-113.

LOU Yunjie,AI Mingfei,ZHUANG Shujie,et al.DK-Port: construction and validation of port autonomous driving simulation environment based on large language models and reinforcement learning[J].Chinese Journal of Intelligent Science and Technology,2025,07(01):98-113.
娄云洁,艾明飞,庄术洁等.DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证[J].智能科学与技术学报,2025,07(01):98-113. DOI： 10.11959/j.issn.2096-6652.202509.

LOU Yunjie,AI Mingfei,ZHUANG Shujie,et al.DK-Port: construction and validation of port autonomous driving simulation environment based on large language models and reinforcement learning[J].Chinese Journal of Intelligent Science and Technology,2025,07(01):98-113. DOI： 10.11959/j.issn.2096-6652.202509.

摘要

与常规驾驶环境相比，港口环境具有车辆作业繁忙、道路定制化以及人机车辆混行等特点。为了解决港口自动驾驶数据缺乏和算法泛化性问题，缩短算法开发周期和降低开发成本，以具身智能设计理念为依托，以提供逼真可控环境、车辆交互过程为目标，提出了DK-Port港口自动驾驶仿真环境的构建与验证方法。首先，基于调查问卷和专家经验，采用零次提示和思路链等技术，使通用大语言模型参与奖励函数的设计过程；然后，基于复杂道路上车辆的行驶数据，构建丰富且逼真的人机混行仿真交互场景，并利用PPO深度强化学习算法训练对抗驾驶员模型，以揭示自动驾驶算法的安全隐患；最后在直道和交叉路口等4种典型场景下进行了对比实验。结果表明DK-Port能够有效生成更符合港口实际特性的多类型对抗驾驶行为，如危险超车、紧急切入、抢占路口交汇点等。在保证关键指标分布合理的前提下，直道场景下的变道次数是基准方法的2.9倍，交叉路口场景下的紧急制动率提升了46.7%。

Abstract

Port environments differ significantly from conventional driving environments

characterized by high vehicle activity

customized roadways

and mixed traffic involving human-operated and autonomous vehicles. To address the lack of port-specific autonomous driving data and improve algorithm generalization while reducing development costs and time

this paper proposes a simulation environment for port autonomous driving called DK-Port. Leveraging embodied intelligence principles

DK-Port provides a realistic and controllable environment for vehicle interactions. This paper incorporates survey-based expert input

employing zero-shot prompting and chain-of-thought reasoning techniques to enable large language models to assist in designing reward functions efficiently. Then

human-machine mixed-traffic scenarios are constructed using complex road data

and adversarial driver models are trained with the PPO reinforcement learning algorithm to identify safety vulnerabilities in autonomous driving systems. Comparative experiments in four typical scenarios

including straight roads and intersections

demonstrate that DK-Port generates diverse adversarial driving behaviors aligned with real port characteristics

such as dangerous overtaking and abrupt lane cutting. Under the premise of ensuring a reasonable distribution of key metrics

the number of lane changes in the straight road scenario is 2.9 times that of the baseline method

and the emergency braking rate in the intersection scenario is increased by 46.7%.

关键词

Keywords

references

The United Nations Conference on Trade and Development. Review of maritime transport 2023[R]. 2023.

程国政. 我国港口集装箱运输发展现状与趋势[J]. 中国航务周刊, 2024(30): 45-47.

CHENG G Z. Development status and trends of container transportation in China's ports[J]. China Shipping Gazette, 2024(30): 45-47.

柳长满, 张传捷, 陈微波, 等. 国内沿海自动化集装箱码头关键技术发展趋势[J]. 中国港口, 2021(1): 17-23.

LIU C M, ZHANG C J, CHEN W B, et al. Key technology development trends of domestic coastal automated container terminal[J]. China Ports, 2021(1): 17-23.

CHU F, GAILUS S, LIU L, et al. The future of automated ports[R]. 2018.

RODRIGUE J P. The geography of transport systems[M]. New York: Routledge, 2020.

DE LA PEÑA ZARZUELO I, FREIRE SOEANE M J, LÓPEZ BERMÚDEZ B. Industry 4.0 in the port and maritime industry: a literature review[J]. Journal of Industrial Information Integration, 2020, 20: 100173.

CHEN X Q, LIU S H, ZHAO J S, et al. Autonomous port management based AGV path planning and optimization via an ensemble reinforcement learning framework[J]. Ocean & Coastal Management, 2024, 251: 107087.

LI X. Trends and prospects of port digital transformation[J]. Highlights in Business, Economics and Management, 2024, 24: 227-233.

林维猛. 我国港口自动驾驶的趋势和挑战[J]. 中国港口, 2022(11): 1-5.

LIN W M. Trends and challenges of autonomous driving in China's ports [J]. China Ports, 2022(11): 1-5.

王媛, 席芳, 汤伊琼. 新形势下我国智慧港口建设现状与发展趋势[J]. 水运工程, 2024(8): 224-228, 287.

WANG Y, XI F, TANG Y Q. Current situation and development trends of smart port construction in China[J]. Port & Waterway Engineering, 2024(8): 224-228, 287.

VAQUERO V, REPISO E, SANDELIU A, et al. Low cost, robust and real time system for detecting and tracking moving objects to automate cargo handling in port terminals[C]//Proceedings of Robot 2015: Second Iberian Robotics Conference: Advances in Robotics. New York: Springer International Publishing, 2016, 2: 491-502.

CHEN X Q, WANG Z C, HUA Q Z, et al. AI-empowered speed extraction via port-like videos for vehicular trajectory analysis[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(4): 4541-4552.

HU Y J, DONG L C, XU L. Multi-AGV dispatching and routing problem based on a three-stage decomposition method[J]. Mathematical Biosciences and Engineering, 2020, 17(5): 5150-5172.

HU H T, YANG X R, XIAO S C, et al. Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning[J]. International Journal of Production Research, 2023: 61: 65-80.

QIN K, WANG B, ZHANG H, et al. Research on application and testing of autonomous driving in ports[J]. SAE Technical Paper, 2020.

KIM E, KIM Y, PARK J. The necessity of introducing autonomous trucks in logistics 4.0[J]. Sustainability, 2022, 14(7): 3978.

CHOTTANI A, HASTINGS G, MURNANE J, et al. Distraction or disruption? Autonomous trucks gain ground in US logistics[R]. 2018

MOORE D, REBECCA C, MICHAEL S, et al. Defense against the dark cars: design principles for griefing of autonomous vehicles[C]//Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. [S.l.:s.n.], 2020: 201-209.

LEE Y C, MOMEN A, LAFRENIERE J. Attributions of social interactions: driving among self-driving vs . conventional vehicles[J ] . Technology in Society, 2021, 66: 101631.

YAM K C, TANG P M, JACKSON J C, et al. The rise of robots increases job insecurity and maladaptive workplace behaviors: multimethod evidence[J]. Journal of Applied Psychology, 2023, 108(5): 850-870.

International Transport Forum. Container port automation: impacts and implications[R]. 2021.

和福建, 马文博, 田晓笛. 港口自动驾驶现状及典型场景测试技术研究[J]. 汽车电器, 2022(8): 1-2, 5.

HE F J, MA W B, TIAN X D. Research on the status of port autonomous driving and testing technology of typical scenes[J]. Auto Electric Parts, 2022(8): 1-2, 5.

王飞跃, 王雨桐. 数字科学家与平行科学: AI4S和S4AI的本源与目标[J]. 中国科学院院刊, 2024, 39(1): 27-33.

WANG F Y, WANG Y T. Digital scientists and parallel sciences: the origin and goal of AI for science and science for AI[J]. Bulletin of Chinese Academy of Sciences, 2024, 39(1): 27-33.

WANG X, HUANG J, TIAN Y L , et al. Parallel driving with big models and foundation intelligence in cyber-physical-social spaces[J]. Research. 2024, 7(3): 1-16.

YAN X T, ZOU Z X, FENG S, et al. Learning naturalistic driving environment with statistical realism[J]. Nature Communications, 2023, 14: 2037.

CHEN D, ZHU M X, YANG H, et al. Data-driven traffic simulation: a comprehensive review[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(4): 4730-4748.

HAO K K, CUI W, LUO Y G, et al. Adversarial safety-critical scenario generation using naturalistic human driving priors[J]. IEEE Transactions on Intelligent Vehicles, 2024: 1-16.

HAO K K, LIU L, CUI W, et al. Bridging data-driven and knowledge-driven approaches for safety-critical scenario generation in automated vehicle validation[EB]. arXiv preprint, 2023: 2311.109371.

王祥,谭国真. 基于知识与大语言模型的高速环境自动驾驶决策研究[J]. 系统仿真学报, 已录用, doi: 10.16182/j.issn1004731x.joss.24-0065.

WANG X, TAN G Z. Research on decision-making of autonomous driving in highway environment based on knowledge and large language model[J]. Journal of System Simulation, accepted, doi: 10.16182/j.issn1004731x.joss.24-0065.

时高松, 赵清海, 董鑫, 等. 基于PPO算法的自动驾驶人机交互式强化学习方法[J]. 计算机应用研究, 2024, 41(9): 2732-2736.

SHI G S, ZHAO Q H, DONG X, et al. Human-machine interactive reinforcement learning method for autonomous driving based on PPO algorithm[J]. Application Research of Computers, 2024, 41(9): 2732-2736.

SHINN N, CASSANO F, GOPINATH A, et al. Reflexion: language agents with verbal reinforcement learning[EB ] . arXiv preprint, 2023, arXiv: 2303.11366v4 .

MADAAN A, TANDON N, GUPTA P, et al. Self-refine: iterative refinement with self-feedback[EB ] . arXiv preprint, 2023, arXiv: 2303.17651v2 .

XIE T B, ZHAO S H, WU C H, et al. Text2Reward: reward shaping with language models for reinforcement learning[C]//Proceedings of the Twelfth International Conference on Learning Representations. [S.l.:s.n.], 2024.

KWON M, XIE S M, BULLARD K, et al. Reward design with language models[EB ] . arXiv preprint, 2023, arXiv: 2303.00001 .

LI Y J, CHOI D, CHUNG J, et al. Competition-level code generation with AlphaCode[J]. Science, 2022, 378(6624): 1092-1097.

XUE B Z, PARK N K. A study on the efficiency of container terminals in Korea and China[J]. The Asian Journal of Shipping and Logistics, 2016, 32(4): 213-220.

COIFMAN B, Li L Z. A critical evaluation of the next generation simulation (NGSIM) vehicle trajectory dataset[J]. Transportation Research Part B: Methodological, 2017, 105: 362-377.

ZHAN W, SUN L T, WANG D, et al. INTERACTION dataset: an international, adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps[EB ] . arXiv preprint, 2019, arXiv: 1910.03088 .

HO J, ERMON S. Generative adversarial imitation learning[EB ] . arXiv preprint, 2016, arXiv: 1606.03476 .

AMODEI D, OLAH C, STEINHARDT J, et al. Concrete problems in AI safety[EB ] . arXiv preprint, 2016, arXiv: 1606.06565 .

ARORA S, DOSHI P. A survey of inverse reinforcement learning: challenges, methods and progress[J]. Artificial Intelligence, 2021, 297: 103500.

HUANG L, YU W J, MA W T, et al. A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions[EB]. arXiv preprint, 2023, arXiv: 2311.05232v2

沈甜雨, 李志伟, 范丽丽, 等. 具身智能驾驶: 概念、方法、现状与展望[J]. 智能科学与技术学报, 2024, 6(1): 17-32.

SHEN T Y, LI Z W, FAN L L, et al. Embodied intelligent driving: concept, methods, the state of the art and beyond[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 17-32.

SPECTOR P E. Do not cross me: optimizing the use of cross-sectional designs[J]. Journal of Business and Psychology, 2019, 34(2): 125-137.

JOSHI A, KALE S, CHANDEL S, et al. Likert scale: explored and explained[J]. British Journal of Applied Science & Technology, 2015, 7(4): 396-403.

KOJJMA T, GU S S, REID M, et al. Large language models are zero-shot reasoners[J]. Advances in Neural Information Processing Systems, 2022, 35: 22199-22213.

ZHOU X, KIM K, XU B W, et al. The devil is in the tails: how long-tailed code distributions impact large language models[C]//Proceedings of the 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). Piscataway: IEEE Press, 2023: 40-52.

JIANG Z, XU F F, GAO L, et al. Active retrieval augmented generation[EB ] . arXiv preprint, 2023, arXiv: 2305.06983 .

XIE Z K. Order matters in hallucination: reasoning order as benchmark and reflexive prompting for large-language-models[EB ] . arXiv preprint, 2024, arXiv: 2408.05093 .

SKALSE J, HOWE N, KRASHENINNIKOV D, et al. Defining and characterizing reward gaming[J]. Advances in Neural Information Processing Systems, 2022, 35: 9460-9471.

SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[EB ] . arXiv preprint, 2017, arXiv: 1707.06347 .

LEURENT E. An environment for autonomous driving decision-making[R]. 2018.

KESTING A, TREIBER M, HELBING D. Enhanced intelligent driver model to access the impact of driving strategies on traffic capacity[J]. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2010, 368(1928): 4585-4605.

KESTING A, TREIBER M, HELBING D. General lane-changing model MOBIL for car-following models[J]. Transportation Research Record: Journal of the Transportation Research Board, 2007, 1999(1): 86-94.

MCINNES L, HEALY J, ASTELS S. Hdbscan: Hierarchical density based clustering[J]. The Journal of Open Source Software, 2017, 2(11): 205.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于ACP和LLM犯罪学研究进展：综述与展望

生成式大模型引发的隐私风险及治理路径

基于改进DQN算法的复杂海战场路径规划方法

机器人自动轴孔装配研究进展

基于动态自选择参数共享的合作多智能体强化学习算法