Interaction-aware planning and optimal control strategies for automated parking can be derived by model-based reinforcement learning (MBRL). For MBRL based parking, collision avoidance and performance guarantee benefit from the use of vehicle model integrated in the agent for planning parking trajectory. However, the vehicle model significantly influences the training process and final parking accuracy. To improve the parking performance, a model refined RL based automated parking is proposed using system identificated transfer function model. Vehicle kinematic model, dynamic bicycle model, speed and steering transfer function models approximating the longitudinal and lateral dynamic characters of the chassis control are used for comparison in this study.

To obtain the identificated model, first, the kinematic model is adopted in reinforcement learning, which is implemented with Monte Carlo tree search (MCTS) and neural network (NN), a less precise parking planner is obtained and the order sequence. Second, the real vehicle data are collected to obtain response characteristics of chassis control system using the order sequence. Then, the vehicle model is used to train a reinforcement learning model. The receding horizon search scheme for the motion planning and control is realized via tree search guided by trained NN. The NN learns the experience of tree search. MCTS search stronger moves for parking, resulting in the reinforcement of parking planning and control in the next iteration. In order to speed up the off-line learning process, multithreading parallel computing is employed, which encourages each instance of simulation with different strength of action exploration. A sampling strategy of increasing sampling times on-the-fly is proposed to promote the robustness of the proposed algorithm.

The influence of different vehicle models on the learning process and control is studied by the simulations. It shows that low order transfer function models are preferable for the proposed method. The real vehicle experiment validates the effectiveness of the proposed method.

Dr. Shaoyu Song, Tongji University, CHINA; Prof. Dr.-Ing. Hui Chen, Tongji University, CHINA; Dr. Jiren Zhang, Tongji University, CHINA; Mr. Fengwei Hu, Tongji University, CHINA