Indoor copter not correctly following guided commands

Hello,

we are working on a EDU450 based indoor copter with localization using a lidar-based system. We are sending VISION_POSITION_ESTIMATE messages using mavros to the copter. All pose estimation is happening on an onboard computer, so there is no network delay.
We are using guided mode to move the copter around, and during today’s test we experienced a strange behavior. During the flight I switched from guided to loiter using a remote controller and taking control to just stop the copter for a moment. Then I switched back to guided, and the copter started to go again. But it did not go to the target location provided, as you can see in the following screenshot for the x axis and in the log which can be downloaded here: https://drive.google.com/file/d/1_Nsbn4QopI9u4BTcc-6q2LaxwWtN8TGM/view?usp=sharing

Unfortunately, this behavior led to a crash with only very minor damage, but I have no idea why this has happened, and it looks like a flight controller problem to me.

I would be glad if someone with more experience in analyzing log files and/or using non-GNSS localization could take a look and maybe offer some advice!

Thanks in advance
Felix