publications

2026

1. preprint

   

   Topology Enhanced MARL for Multi-Agent Cooperative Decision-Making of CAVs

   Ye Han, Lijun Zhang, Dejian Meng, and 1 more author

   arXiv preprint, 2026

   Abs DOI HTML PDF

   Decentralized multi-agent cooperative decision-making in continuous environments is fundamentally bottlenecked by the curse of dimensionality, where undirected exploration typically converges to conservative local optima. We propose Topology-Enhanced Multi-Agent Reinforcement Learning (TPE-MARL) to reformulate multi-agent exploration as a structured topological traversal. We introduce the Game Topology Tensor, utilizing locality-sensitive hashing to project the continuous physical manifold into a discrete quotient space. This abstraction operates as an adversarial Information Bottleneck, decoupling strategic coordination intents from environmental noise. Within this space, a dual intrinsic reward mechanism drives exploration: a novelty reward maximizes the marginal entropy of visited topologies, while a collaboration reward, optimized via a variational Evidence Lower Bound (ELBO), minimizes conditional entropy to exploit cooperative joint configurations. Evaluations demonstrate that TPE-MARL achieves near-optimal decision distributions, closely approximating the theoretical bounds established by a Monte Carlo Tree Search (MCTS) oracle. Furthermore, physical testbed experiments validate the framework’s zero-shot out-of-distribution (OOD) generalization. Supported by a spatial relaxation mechanism, the learned representations reliably execute dynamic negotiations, such as cooperative zipper-merging, exhibiting inherent robustness against real-world covariate shifts and actuation latencies.

2. preprint

   

   Hybrid Differential Reward: Combining Temporal Difference and Action Gradients for Efficient Multi-Agent Reinforcement Learning in Cooperative Driving

   Ye Han, Lijun Zhang, Dejian Meng, and 1 more author

   arXiv preprint, 2026

   Abs DOI HTML

   In multi-vehicle cooperative driving tasks involving high-frequency continuous control, traditional state-based reward functions suffer from the issue of vanishing reward differences. This phenomenon results in a low signal-to-noise ratio (SNR) for policy gradients, significantly hindering algorithm convergence and performance improvement. To address this challenge, this paper proposes a novel Hybrid Differential Reward (HDR) mechanism. We first theoretically elucidate how the temporal quasi-steady nature of traffic states and the physical proximity of actions lead to the failure of traditional reward signals. Building on this analysis, the HDR framework innovatively integrates two complementary components: (1) a Temporal Difference Reward (TRD) based on a global potential function, which utilizes the evolutionary trend of potential energy to ensure optimal policy invariance and consistency with long-term objectives; and (2) an Action Gradient Reward (ARG), which directly measures the marginal utility of actions to provide a local guidance signal with a high SNR. Furthermore, we formulate the cooperative driving problem as a Multi-Agent Partially Observable Markov Game (POMDPG) with a time-varying agent set and provide a complete instantiation scheme for HDR within this framework. Extensive experiments conducted using both online planning (MCTS) and Multi-Agent Reinforcement Learning (QMIX, MAPPO, MADDPG) algorithms demonstrate that the HDR mechanism significantly improves convergence speed and policy stability. The results confirm that HDR guides agents to learn high-quality cooperative policies that effectively balance traffic efficiency and safety.

3. preprint

   

   Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality

   Ye Han, Lijun Zhang, Dejian Meng, and 1 more author

   arXiv preprint, 2026

   Abs DOI HTML

   Evaluating Multi-Agent Reinforcement Learning (MARL) policies in autonomous driving fundamentally relies on extrinsic statistical indicators (e.g., reward curves and success rates), which often mask intrinsic policy degradation and algorithmic blind spots. To break this black-box evaluation, this letter proposes a novel information-theoretic diagnostic framework. By leveraging a fully converged Monte Carlo Tree Search (MCTS) as an asymptotic oracle, we establish a theoretical ground-truth baseline distribution. We formulate a bounded policy optimality score (\mathcalM_opt) using the forward KL divergence to rigorously penalize fatal collaborative omissions. Crucially, we semantically decouple this metric into lateral and longitudinal dimensions, creating a granular “semantic microscope”. Extensive spatial and temporal diagnostics on state-of-the-art MARL architectures and exploration mechanisms demonstrate that our framework conclusively exposes hidden directional biases, identifies temporal average-policy traps, and transforms heuristic hyperparameter tuning into a visually trackable trajectory optimization. This framework establishes a rigorous, model-agnostic standard for benchmarking intrinsic multi-agent policy quality.

4. Auto. Inno.

   

   A Geometry and Semantics-Based Progressive Generation Approach for Traffic Sign Adversarial Patches

   Zhuang Zhang, Lijun Zhang, Dejian Meng, and 4 more authors

   Automotive Innovation. More Information can be found here , 2026

   Abs DOI HTML Video

   Traffic sign detection systems help intelligent vehicles to perceive and adhere to traffic regulations, which is crucial for ensuring driving safety. Recent studies reveal that attackers can generate adversarial examples by attaching carefully designed patches to traffic signs, which can mislead detection results and produce potential collision risks. In order to enhance the success rate of adversarial attacks and fully reveal the threat of adversarial examples, researchers simulate the changes of traffic signs in driving scenarios by transforming the size and position of adversarial patches when generating them. However, when performing such transformations, most studies did not take into account the long-tailed distribution of traffic sign sizes in driving scenarios. Additionally, optimization mechanisms overly rely on semantic information and ignore geometric information, which limits the applicability of adversarial examples. To address these issues, this paper proposes a progressive sampling strategy-guided image transformation that adjusts the size distribution of traffic signs based on the number of attack rounds, thereby fitting the long-tailed distribution through an “easy-to-hard" progressive attack. Additionally, an optimization metric is proposed to adjust the semantic confidence based on the intersection over union between the ground truth and predicted boxes, thereby enabling a focus on boxes that exhibit both high geometric and semantic consistency. Experiments conducted on the TT100KD and COCO datasets, as well as on a dataset collected by a test vehicle, demonstrate that the proposed method achieves at least a 3.5% improvement in attack success rates compared to existing methods.

5. ESWA

   

   Hybrid reinforcement learning-model predictive control framework for full-band (0–500 Hz) vibro-acoustic optimization in active suspension systems

   Zhehui Zhu, Dejian Meng, Ye Han, and 2 more authors

   Expert Systems with Applications, 2026

   Abs DOI HTML

   Electronically Controlled Suspension (ECS) systems primarily aim to improve low-frequency vibration comfort (0–20 Hz), whereas Active Road Noise Control (ARNC) systems focus on mitigating mid- to high-frequency structure-borne noise (20–500 Hz). However, ECS discrete control signals may excite high-frequency structural vibrations that couple with the tire cavity mode, thereby deteriorating in-cabin acoustic comfort. To investigate how ECS influences in-cabin noise, this study proposes a high-frequency quarter-car model incorporating tire acoustic cavity coupling dynamics (HFM-ACO). Model parameters are identified and validated utilizing vehicle road-test data, and the vibration-to-interior-noise time-domain transfer functions are obtained through operational transfer path analysis (OTPA), serving as prior knowledge for control design and performance evaluation. To suppress high-frequency vibration and noise, this study develops an intelligent decision-making module based on a Hybrid Reinforcement Learning-Model Predictive Control (Hybrid RL-MPC) architecture, in which a learned value function encapsulates long-term control knowledge and is synthesized as the MPC terminal cost, while the terminal states predicted by MPC are incorporated as additional observations for the RL agent. By framing the control task as a multi-band vibration optimization problem within the learning-based decision framework, the RL agent further compensates the MPC-generated force to refine full-band performance. Simulation results indicate that the proposed controller improves the root-mean-square of body acceleration by 26.98% in 0–20 Hz and by 63.6% in 50–500 Hz compared with baseline controllers, with notable reductions in in-cabin noise levels. Comparative analyses of various RL-MPC fusion architectures verify that the proposed framework achieves the best vibro-acoustic performance. This study provides theoretical guidance for the integrated optimization of ECS and ARNC, offering new insights for enhancing ride and acoustic comfort in intelligent vehicles.

2025

1. IEEE T-ITS

   

   A Value Based Parallel Update MCTS Method for Multi-Agent Cooperative Decision Making of Connected and Automated Vehicles

   Ye Han, Lijun Zhang, Dejian Meng, and 3 more authors

   IEEE Transactions on Intelligent Transportation Systems, 2025

   Abs DOI HTML

   This paper addresses the problem of long term joint lateral and longitudinal decision-making for multi-vehicle cooperative driving among connected and automated vehicles (CAVs). We propose a Monte Carlo tree search (MCTS) method featuring parallel update mechanism and experiential action preference, with the problem modeled as a finite-horizon, time-discounted multi-agent Markov game. By enabling a single simulation rollout to prune entire families of hazardous actions simultaneously, parallel update mechanism dramatically accelerates the convergence of the search within the vast joint action space. The search is further focused by an experiential action preference heuristic that biases exploration towards efficient maneuvers. Evaluated in randomly generated mixed-traffic scenarios, our method significantly outperforms standard MCTS and reinforcement learning baselines in both safety and traffic efficiency. Crucially, in-depth analysis reveals the discovery of emergent, non-myopic cooperative strategies that are inaccessible to myopic planners. This work provides a computationally tractable solution for complex, multi-agent MCTS, achieving a higher level of strategic cooperative decision-making.

2. CSAE

   

   A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms

   Ye Han, Lijun Zhang, Dejian Meng, and 1 more author

   In Proceedings of the 32nd China-SAE Congress: Selected Papers, 2025

   Abs DOI HTML

   Reinforcement learning (RL) shows great potential for optimizing multi-vehicle cooperative driving strategies through the state-action-reward feedback loop, but it still faces challenges such as low sample efficiency. This paper proposes a differentiated reward method based on steady-state transition systems, which incorporates state transition gradient information into the reward design by analyzing traffic flow characteristics, aiming to optimize action selection and policy learning in multi-vehicle cooperative decision-making. The performance of the proposed method is validated in RL algorithms such as MAPPO, MADQN, and QMIX under varying autonomous vehicle penetration. The results show that the differentiated reward method significantly accelerates training convergence and outperforms centering reward and others in terms of traffic efficiency, safety, and action rationality. Additionally, the method demonstrates strong scalability and environmental adaptability, providing a novel approach for multi-agent cooperative decision-making in complex traffic scenarios.

3. ISoIRS

   

   LLMCTS: Large Language Model Prior Guided MCTS for Autonomous Vehicle Decision-Making

   Xingyu Hu, Lijun Zhang, Dejian Meng, and 3 more authors

   In 2025 International Symposium on Intelligent Robotics and Systems (ISoIRS), 2025

   Abs DOI HTML

   Although the technique of combining Deep Reinforcement Learning (DRL) with Monte Carlo Tree Search (MCTS) has shown promise in Autonomous Vehicle (AV) decision-making, MCTS suffers from high computational resource consumption and low initial exploration efficiency due to its search characteristics. To address these issues, we propose LLMCTS, a framework that integrates large language model (LLM) prior knowledge into the MCTS algorithm. Specifically, LLMCTS first utilizes the expert reasoning of LLM to guide AV decision-making and stores it as policy data. Then, MCTS uses this data together with the policy generated by tree search for DRL training. Finally, MCTS outputs the AV decision. To verify the effectiveness of the LLMCTS framework, we conduct an experimental evaluation on an unprotected left-turn intersection. The results show that LLMCTS outperforms the baseline methods in all aspects. In particular, we also demonstrate that LLMCTS can guarantee the performance of MCTS with a small number of simulations.

4. EICARS

   

   Spatial-Temporal Graph Hybrid Network Reinforcement Learning for Decision-Making of Connected and Autonomous Vehicles

   Songyu Weng, Xingyu Hu, Ye Han, and 1 more author

   In 2025 International Conference on Electronic Information, Computer and Aerospace Remote Sensing (EICARS), 2025

   Abs DOI HTML

   Multi-vehicle cooperative driving is critical to the development of connected and autonomous vehicles (CAVs), especially in mixed traffic environments where CAVs operate alongside human-driven vehicles (HDVs). However, the complexity of inter-vehicle interactions present substantial challenges for coordinated decision-making. Although deep reinforcement learning (DRL) has shown promise in complex decision tasks, existing approaches often struggle to accurately characterize vehicle interactions and to learn informative driving representations. To address these limitations, we propose a graph neural network-based multi-vehicle decision-making framework. The core of the framework is a spatial-temporal graph architecture that integrates graph attention networks (GAT) with temporal convolutional networks (TCN) to effectively model vehicle interactions and extract rich driving features for decisionmaking. The model is trained using a multi-agent deep Q-network (MADQN) algorithm. Extensive experiments in a highway offramp scenario demonstrate that our method outperforms baselines in key metrics such as average speed and task success rate, achieving a balanced performance in traffic efficiency, task completion, and driving safety.

2024

1. IEEE ITSC

   

   SPformer: A transformer based DRL decision making method for connected automated vehicles

   Ye Han, Lijun Zhang, Dejian Meng, and 2 more authors

   In IEEE International Conference on Intelligent Transportation Systems, 2024

   Abs DOI HTML

   In mixed autonomy traffic environment, every decision made by an autonomous-driving car may have a great impact on the transportation system. Because of the complex interaction between vehicles, it is challenging to make decisions that can ensure both high traffic efficiency and safety now and futher. Connected automated vehicles (CAVs) have great potential to improve the quality of decision-making in this continuous, highly dynamic and interactive environment because of their stronger sensing and communicating ability. For multi-vehicle collaborative decision-making algorithms based on deep reinforcement learning (DRL), we need to represent the interactions between vehicles to obtain interactive features. The representation in this aspect directly affects the learning efficiency and the quality of the learned policy. To this end, we propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy, the states of all vehicles in the area of interest can be adaptively noticed in order to extract interactive features among agents. We also design an intuitive physical positional encodings, the redundant location information of which optimizes the performance of the network. Simulations show that our model can make good use of all the state information of vehicles in traffic scenario, so as to obtain high-quality driving decisions that meet efficiency and safety objectives. The comparison shows that our method significantly improves existing DRL-based multi-vehicle cooperative decision-making algorithms.

2. ICACMVE

   

   Defending Against Adversarial Examples with Adaptive Quantization Table-Guided Image Compression

   Zhuang Zhang, Lijun Zhang, Dejian Meng, and 3 more authors

   In The 9th International Conference on Advances in Construction Machinery and Vehicle Engineering, 2024

   Abs DOI HTML

   Intelligent vehicles rely on deep learning-based image recognition models for accurate environmental perception, which is critical for driving safety. However, recent research has identified the significant threat from adversarial examples, which can mislead models by adding subtle perturbations to input images, thereby endangering intelligent vehicles. Existing defenses typically use image preprocessing to remove perturbations, however, most works overlook the diversity in adversarial perturbation amplitudes, leading to performance degradation in varying conditions. To address this limitation, this paper investigates the relationship between changes in image spatial frequency and varying adversarial perturbation amplitudes, revealing that increasing perturbation amplitudes lead to heightened high-frequency variations in the spectrum. Based on this finding, we propose an adaptive quantization table-guided image compression method that modifies quantization tables based on the image’s frequency spectrum, which enhances the defense performance compared to existing methods that rely on fixed quantization tables. Experimental results demonstrate that our adaptive method effectively responds to the dynamic nature of adversarial examples, significantly outperforming state-of-the-art defenses against common adversarial attacks, such as FGSM, PGD, Deepfool, CW, AutoAttack, and BPDA. The proposed method shows potential for application in intelligent vehicles, as it offers robust defense capabilities across a range of perturbation amplitudes.

3. ICACMVE

   

   Generating Adversarial Examples for Pedestrian Detectors in Dynamic Driving Scenarios

   Zhuang Zhang, Lijun Zhang, Dejian Meng, and 4 more authors

   In The 9th International Conference on Advances in Construction Machinery and Vehicle Engineering, 2024

   Abs DOI HTML

   Pedestrian detection is a key component of vision-based systems in autonomous driving, critical for ensuring traffic safety. Recent research has shown that adversarial examples can mislead these systems, highlighting the need to evaluate and improve their robustness. However, most prior work assumes that adversarial examples can be clearly captured, overlooking a common real-world scenario: adversarial examples can be blurred by vehicle motion. This motion blur, frequently encountered in dynamic driving scenarios, can obscure adversarial examples and reduce the effectiveness of attacks. To address this issue, this paper proposes an adversarial example generation method based on an image motion blur model for driving scenarios. First, we develop the model using vehicle kinematics, revealing the non-uniform distribution of motion blur parameters for pedestrians at different positions. Based on this, we propose a motion blur model-guided adversarial example generation method that adapts to both the driving scenario and the pedestrian’s position, ensuring that adversarial examples remain effective despite motion blur. Finally, we validate the proposed method through real-world vehicle experiments, demonstrating a 19.67% improvement in the average attack success rate across various vehicle speeds compared to existing methods. This method provides a practical tool for testing pedestrian detection robustness in autonomous driving.

4. IJAT

   GITSR: Graph interaction transformer-based scene representation for multi vehicle collaborative decision-making

   Xingyu Hu, Lijun Zhang, Dejian Meng, and 2 more authors

   International Journal of Automotive Technology, 2024

   DOI HTML