Scan:669: scan past end of eeprom

OK, first ever post, be gentle… ;-p

I’m seeing the following in the ~/APM/info/rover.log file “scan:669: scan past end of eeprom”, which can’t be a good thing…!

I’m using a Pi 3 (Frambuesca, PXFmini-RaspberryPi3-frambuesa-25-10-2016.img), PXFmini, no additional hardware, Mission Planner, and the latest source from ArduPilot:

$ git clone GitHub - ArduPilot/ardupilot: ArduPlane, ArduCopter, ArduRover, ArduSub source, ie. the master.

Mission Planner is responsive to turning the board, ie. the UDP MAVlink is working.

This error does not happen with APMrover2-release [Edit: ie. v3.1.2, so correction, I’ll not try with APMrover2-3.1.1]

$ cat APM/info/rover.log :

Interrupted: Segmentation fault
scan:669: scan past end of eeprom
scan:669: scan past end of eeprom
Raspberry Pi 2/3 with BCM2709!
MS5611 found on bus 0 address 0x00
MPU: temp reset 6208 0
Interrupted: Segmentation fault
scan:669: scan past end of eeprom
scan:669: scan past end of eeprom
Raspberry Pi 2/3 with BCM2709!
MS5611 found on bus 0 address 0x00
MPU: temp reset 6176 0
Interrupted: Segmentation fault
scan:669: scan past end of eeprom
scan:669: scan past end of eeprom
Raspberry Pi 2/3 with BCM2709!
MS5611 found on bus 0 address 0x00
MPU: temp reset 5962 4194
Interrupted: Continued
Interrupted: Continued
scan:669: scan past end of eeprom
scan:669: scan past end of eeprom
Raspberry Pi 2/3 with BCM2709!
MS5611 found on bus 0 address 0x00
MPU: temp reset 5986 0

Note: I’ve been investigating the “Interrupted: Segmentation fault” separately (it happens more on APMrover2-release): it is probably causing “Failsafe trigger” in the Mission Planner messages, and hence frequent soft restarts which it seems can crash vehicles.

Mission Planner Message tab extract (read these upwards):

EKF2 IMU0 tilt alignment complete
Failsafe trigger 0x1
EKF2 IMU0 initial yaw alignment complete
Ready to drive
Beginning INS calibration. Do not move vehicle
Warming up ADC
<startup_ground> Ground start
Barometer calibration complete
Calibrating barometer
Initialising APM
EKF2 IMU0 tilt alignment complete
Failsafe trigger 0x1
EKF2 IMU0 initial yaw alignment complete
Ready to drive
Beginning INS calibration. Do not move vehicle
Warming up ADC
<startup_ground> Ground start
Barometer calibration complete
EKF2 IMU0 tilt alignment complete
Failsafe trigger 0x1
$ sudo vi b :
break AP_Param.cpp:669

Start gdb :
$ sudo gdb --args APMrover2.elf -A udp:10.0.0.2:14550 -B /dev/ttyAMA0 -l /home/erle/APM/logs -t /home/erle/APM/terrain/

GNU gdb (Raspbian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type “show copying”
and “show warranty” for details.
This GDB was configured as “arm-linux-gnueabihf”.
Type “show configuration” for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type “help”.
Type “apropos word” to search for commands related to “word”…
Reading symbols from APMrover2.elf…done.

Load the breakpoints (see above):

(gdb) source b
Breakpoint 1 at 0x213ec: file /home/erle/a-l/libraries/AP_Param/AP_Param.cpp, line 669.

Run it:

(gdb) run
Starting program: /home/erle/APMrover2.elf -A udp:10.0.0.2:14550 -B /dev/ttyAMA0 -l /home/erle/APM/logs -t /home/erle/APM/terrain/
[Thread debugging using libthread_db enabled]
Using host libthread_db library “/lib/arm-linux-gnueabihf/libthread_db.so.1”.

Breakpoint 1, AP_Param::scan (target=target@entry=0x7efff4e0, pofs=0x7efff4de, pofs@entry=0x7efff4d6)
at /home/erle/a-l/libraries/AP_Param/AP_Param.cpp:669
669 Debug(“scan past end of eeprom”);

A back trace:

(gdb) bt
#0 AP_Param::scan (target=target@entry=0x7efff4e0, pofs=0x7efff4de, pofs@entry=0x7efff4d6)
at /home/erle/a-l/libraries/AP_Param/AP_Param.cpp:669
1 0x000228c4 in AP_Param::configured_in_storage (this=this@entry=0xf480c <rover+4676>)
at /home/erle/a-l/libraries/AP_Param/AP_Param.cpp:1147
#2 0x0008f51c in configured (this=0xf480c <rover+4676>) at /home/erle/a-l/libraries/AP_Param/AP_Param.h:390
#3 set_default (v=, this=0xf480c <rover+4676>) at /home/erle/a-l/libraries/AP_Param/AP_Param.h:590
#4 RangeFinder::RangeFinder (this=0xf47c8 <rover+4608>, _serial_manager=…, orientation_default=)
at /home/erle/a-l/libraries/AP_RangeFinder/RangeFinder.cpp:537
#5 0x000197fc in Rover::Rover (this=0xf35c8 ) at Rover.cpp:50
#6 0x000129a8 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at APMrover2.cpp:36
#7 _GLOBAL__sub_I_hal () at APMrover2.cpp:424
#8 0x000d294c in __libc_csu_init ()
#9 0x76cb8228 in __libc_start_main (main=0x7efff6e4, argc=1994231808, argv=0x76cb8228 <__libc_start_main+168>,
init=0xd28fc <__libc_csu_init>, fini=0xd2960 <__libc_csu_fini>, rtld_fini=0x76fdf464 <_dl_fini>, stack_end=0x7efff6e4)
at libc-start.c:246
#10 0x00012fa4 in _start ()
(gdb)

Excellent debugging. We will look into it and post back.

Thanks, Grant.

1 Like

Could we get a stack-trace for the segmentation fault?

It might be the scan-past-end-of-eeprom is unrelated.

1 Like

I do get the scan-past-end-of-eeprom on my NavIO (I don’t have an ErleBrain). It isn’t fatal, and I’m pretty sure I see it in SITL from time-to-time, too.

I do not get segmentation faults.

Thanks @gmorph :slight_smile:

Regarding the “Interrupted: Segmentation fault” seen in rover.log:

  • Is very rare on the Latest source that I’m using (master from 2017-07-26 16:46:03. Reported in Mission Planner Messages tab as “APM:Rover V3.2.0-dev (572df8b8)”). Right now I’m giving the Latest build a good long chance to have a Segmentation Fault [Edit: nothing in 25 minutes].

  • It is very regular on the Release v3.1.2. I’ll create a separate thread for that, with stack-trace @peterbarker :wink: . I was hoping to use Release v3.1.2.

Regards, Geoff

As requested there is a stack trace in the following thread (for APMrover2 release v3.1.2, not the latest v.3.2.0-dev referenced above, where the Segmentation Fault seems rare):
https://discuss.ardupilot.org/t/apmrover2-release-v3-1-2-regular-segmentation-fault/19508
Regards, Geoff

Looking at the other post it seems that segmentation fault has been fixed by Lucas.

Thanks, Grant.