Reinforcement Learning with World Models for Autonomous Excavation Optimization in Wheel Loaders
2025 (English)Conference paper, Published paper (Refereed)
Abstract [en]
Automating the bucket-filling task in wheel loaders is challenging due to the complex, nonlinear interaction between the bucket and granular material. This work presents a model-based reinforcement learning approach to optimize the bucket-filling strategy for Zeux, Volvo's autonomous electric wheel loader concept. A Long Short-Term Memory (LSTM) surrogate model is trained on data from Volvo's high-fidelity simulator to emulate realistic dynamics, enabling efficient policy training using Proximal Policy Optimization (PPO) with imagined rollouts. This reduces computational cost and eliminates the need for direct interaction with the high-fidelity simulator. Compared to Volvo's current rule-based driver model, the learned policy achieves 89% improvement in productivity and 56% increase in energy efficiency. Our results show that world models can accelerate reinforcement learning for heavy machinery control, enabling the discovery of strategies that outperform controllers based on human expert behavior.
Place, publisher, year, edition, pages
Elsevier BV , 2025. Vol. 59, p. 72-77
Keywords [en]
Autonomous Systems, Bucket-Filling, Deep Learning, Heavy Machinery Simulation, Reinforcement Learning, Wheel loaders, World Models
National Category
Control Engineering Computer Sciences Robotics and automation Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-375953DOI: 10.1016/j.ifacol.2025.12.184Scopus ID: 2-s2.0-105026936365OAI: oai:DiVA.org:kth-375953DiVA, id: diva2:2033397
Conference
66th International Conference of Scandinavian Simulation Society, SIMS 2025, Stavanger, Norway, Sep 22 2025 - Sep 24 2025
Note
QC 20260129
2026-01-292026-01-292026-01-29Bibliographically approved