Nice! This has been one of my favourite blogs here. Glad to see it coming together!
Initial tests on the 2m wide circuit (changing speed):
An 1/8 R/C bike pursuing the car (with a 10 laps mission programmed):
On the 1/18 scale course why is it missing waypoints 5 and 6 so badly? Would it help to reduce the waypoint radius?
The main problem is that it is not yet repetitive on the 2m wide track. See that the first trials (20181125) were on a rainy day, but were successful. I tried on a sunny day with no clouds at all and couldn’t get a single lap.
The 2m wide track is on a corner, surrounded by trees, perhaps with less satellites.
Is it possible to rely more on inertial navigation and less on GPS? I thought I saw a setting for that.
I see these to weight the compass (magnetometer) versus the GPS units:
Does the navigation not use the gyro and accelerometers to calculate an inertially-derived position and velocity? For a car driving close by ferromagnetic objects and executing rapid maneuvers with high yaw rate in a geographically small course, it seems this would be preferred.
I’m envisioning the inertial navigation would guide the car on the 10 µs to 10 s range, and the GPS would correct the position much more slowly (less often), like 10 s to 100 s. Does the code support this approach?
I have a lot of respect for the OP for fighting through this for many months. Especially given that the quadcopter with the same hardware works so well, this must be extremely frustrating.
Earlier you mentioned the throttle is limited to a low value for safety during testing. How often does the commanded throttle reach this limit?
I would think this clipping behavior would cause a severe nonlinearity that could possibly be faking out the control algorithm. If it is hitting the throttle limit often, consider relaxing (i.e. raising somewhat) the limit and see if that helps the behavior.
Indeed, testing in the big circuit, with CRUISE_SPEED up to 4.5 m/s RCOU.C2 reaches SERVO2_MAX=1600, so it is clipped. But speed regulation is not important now, and the results on the big circuit with RTK are reasonable, although just in one last test. The point now is to follow the desired trajectory reliably with RTK. The car hits the barriers constantly in these tests, so it is not convenient to reduce SERVO2_MAX.
So I started tests on the 2m wide circuit, and the results are not consistent. And because of the ChiBios problems with µSD cards I have not got yet long logs.
This is an example of RCOU.C2 on the 2m wide circuit. With CRUISE_SPEED 1.1 m/s it is rarely clipped, although the log is short:
RCOU.C2 has a minimum of 1517, which corresponds to SERVO2_TRIM. I have not tried yet to insert DO_CHANGE_SPEED waypoints, and have no idea if the car can brake. I haven’t tried either DO_SET_REVERSE waypoints, meant to drive in reverse. These type of competition cars don’t have reverse, so if this implies pulse under SERVO2_TRIM the car would brake.
Perhaps Randy can clarify braking.
I think that proves throttle clipping is not a problem. On a side note, the output seems to be demanding high acceleration and jerk in a few spots (around sample 90000 for example), which by itself is perhaps a symptom of the problem.
I still think reducing the waypoint radius would help behavior on the 1/18 track. The current diameter of each circle is the entire width of the track, allowing close approach to the barriers to be a valid solution. Some of the circles even overlap, which I think would validate vague paths.
I encourage reducing the waypoint radius to 0.2 m and trying again on the small track.
Can you post your most recent log? I’m interested in helping.
Here is a long log of a 10x32 mission (10 laps) in the 2m wide circuit:
Rover_20181214_RTK_Asoger18_12.bin (100 MB)
There are two runs, whose videos are:
On the first video, initially the car had to be constantly put in place; this improved for whatever the reason, and recording was started, so not all waypoints appear. The second is moreless complete (the mobile phone used has a 10’ video limit, so recording has to be restarted, losing some seconds, and later joining videos.
The tlog’s are included in both videos.
The red flashing led on the GPS on top of the car indicates “rtk float” (steady on indicates “rtk fixed”).
As seen, the laps are irregular: bad laps, with car knocking the barriers and even being stopped, car completely out of trajectory, and good laps indicating that coordinates used are moreless correct (perhaps waypoints 8+32n can be shifted 1m south).
On the average, perhaps the number of detected satellites increased from the first run to the second:
With respect to the irregular waypoints linkage, it can be related to this:
First, I think you should pay attention to the following thread: John Easton's battle with 'Wobbly Path' - [NOT SOLVED]
Second, I notice that you have some P-oscillations in steering. Decrease ATC_STR_RAT_P by 50%. Your actual steering rate is matching your desired turning rate, so that is good (FF-gain is set very close to correct).
Your TURN_MAX_G is set to 0.3, and I think this is far too low (causing your car to try to turn at all times rather than hit the navigation lines and stay on them). You need to get your car to hit its desired heading on the line VERY quickly so that you have precise control of the navigation. Then from there you can “soften” things up. Right now your navigation is “sloppy,” and I think in part it is due to TURN_MAX_G and maybe NAVL1_PERIOD. Your navigation is so “quick” that I think NAVL1_PERIOD may need to be <5.
Thanks a lot for the link and your comments about parameter adjustments, not only valuable for car behaviour improvements, but also for understanding their meaning towards a better knowledge of all this. I will test these changes shortly.
In fact, the steering oscillations are clearly seen on some instants on the videos, and I had occasionally seen them also in the big circuit.
However, I’d like to know how to make the car behave always the same, good or bad, but the same. There are good laps and bad laps, and the second run is better on the average. There are more satellites on the average on the second run, and the tlog part of the videos (starting on both at 20") clearly show the good and the bad laps.
The 2m wide R/C car circuit is on a corner, surrounded by tall trees.
HDop is on the average better on the second run:
Observe carefully instants following t=165 on the second run:
While in a good lap, the car suddenly loses trajectory and then recovers. Down and left in the video are shown HDop and Sats, which increase and decrease briefly respectively and then recover. This is a capture once recovered:
(Captured after recovery)
The car is using three compasses (the one on the RTK GPS is not connected):
-Two external at the front bumper, away from the motor.
-The Pixhawk internal one (perhaps should be disabled).
The GPS is above a copper plane, now floating. Perhaps it should be grounded, and the GPS ground connected directly to it.
Would it be convenient or possible to filter the GPS or compass data to avoid sudden trajectory changes?
BTW, this is the version used:
ArduRover V3.5.0-rc1 (f53c86f5)
PX4: bcbd027a NuttX: 1472b16c
PX4v2 00320046 35365118 37353630
With NuttX, no µSD card logging problems.
The key in my opinion is to make sure your steering tuning is spot-on sot that way you can clearly tune the navigation and GPS settings. Once you get rid of the oscillations, then we can move forward.
https://docs.emlid.com/reach/antenna-placement/ tells me that you might want a larger ground plane to receive better satellite data. My Here+ and Emlid Reach RTK GPS units on my plane have very low HDOP readings, but they have a very clear sky and multiple constellations enabled.
Here is yours:
Regarding GPS readings, you might want to try different values for GPS_NAVFILTER. Try 8 and see if that makes any difference.
You might also consider adding an elevation mask to the satellites you receive because satellites that are low in the sky may be giving you bad position data. (GPS_MIN_ELEV). Try 20 to start.
To extract more consistent data from your GPS, it might also require turning down the position rate: GPS_RATE_MS to 200 or 250.
This in combination with changing GPS_GNSS_MODE might help you get more satellites. With the Emlid Reach M+, I know that GPS, GLONASS, and QZSS satellites could be reliably logged at 5Hz (200ms). If you added more constellations, you might have to turn down the logging rate to 1Hz (1000ms) or it would skip GPS events.
After three sessions more or less successful, it seems that the cause for the car going really bad is the RTK station mount point chosen: there are significant differences.
On the first session all went bad. Changing the ground plane, connecting it to electrical ground or not, disabling the internal compass and changing parameters didn’t make significant changes. As suggested, I finally left ATC_STR_RAT_P=0.3 (previously 0.5), which seemed to stop steering oscillations.
On the second session I got two good runs:
Including the hud undocked in these videos with the tlogs may give clues.
The third session was dedicated to the five possibilities for the RTK station mount point. They are described elsewere and are physically different, and one of them is virtual (which gave the worst results). It must be taken into account that they are meant for different needs and which can be very far away from the RTK physical station, but for these tests the distance is 14 Km.
This is the tlogs video with the hud at 1x speed:
These were the trajectories obtained with two of the mount points, which were unacceptable (the second one is virtual):
This other was a bit better:
This other was almost acceptable:
Finally, these were two runs with a mount point with good results:
So the conclusion is first of all if having several mount points, try all and choose the best one. I wonder if there is any parameter (as hdop and satellites for the GPS) to evaluate the quality of the mount point connected, since what appears in Mission Planner RTK screen showed good data for all above mount points.
Wait, your the baseline to your base station is 14km?! That’s a fairly long ways for L1-only, and with a horizon surrounded by trees, I can guess that the GPS navigation would be a little less reliable. With a local base station <1km away, I would guess that you could actually get a GPS fix (instead of bouncing between 3D fix and float all of the time)
We can see GPS.Status in log 36 bouncing erratically between 4 and 5. I’m assuming 4 means DGPS and 5 means RTK.
Looking at GPS.Status from the quadcopter log in August Simultaneous_Copter_20180822_30.bin, we can see it is rock-solid at 4.
Is it possible to defeat the RTK to see how well that works? The quadcopter seemed to work fine without it. I think that would be a worthwhile experiment.
Alternatively, I would bring the car closer to an RTK station, say within 1 km, and attempt Auto driving some waypoints. I’m envisioning something simple in a parking lot near the station. First see if GPS.Status stays constant at 5. If so, then see how well the car stays on the waypoint path. If not, then pursue electromechanical fixes to get it steady at 5. If not steady at 5, then using RTK is maybe actually hindering progress.
4 is 3D Fix, 5 is RTK Float, and 6 is RTK fix. Roughly 2.5, 1, and 0.1 m accuracy respectivley. Bouncing between a RTK float and GPS is going to have a lot of movement because non-RTK position is uncorrected. It would actually be better to totally eliminate RTK because of the on-off nature of signals that are being received.
Placing the base
“Typically the distance between the reference station and local rover shouldn’t exceed 10-15 km…”
The circuits are 14 Km away.
This is GPS Status with the same GPS installed on a drone moreless 15 Km away from the RTK station with a clear sky:
This is GPS Status with the same GPS on the car on the big circuit on the succesful mission above:
That is the RTK station I have. The circuits cannot be moved and the trees cannot be cut. With the RTK GPS the targets are:
-On the big circuit add DO_CHANGE_SPEED waypoints and improve lap time. How can I brake the car?
-On the small circuit try a Balance Bot:
GSoC 2018: Balance Bot with Arduino
if knowing how to make it work with stepper motors (not DC motors).
Now with the best mount point I get acceptable results on both circuits, and even I could drive an 1/8 R/C bike against the car doing this mission:
Clearly the 1/8 R/C bike wins.
On four of the five mount points available Mission Planner shows the RTK station coordinates with many decimal points (same as the coordinates published by them). On the virtual one MP reports a virtual station 4 Km away from the circuits; however the results are the worst. I’ll consult the RTK station people about this.
I suggested that as an experiment only. Information gathering. Data collection. A quick way to figure out whether RTK lock is the only problem with the car or not.
Imagine if you ran my proposed experiment, RTK proved solid at status 6, yet the car still occasionally wandered. You would say, “Aha! I just wasted several months fixing something that was not the root cause.”
Imagine if you ran my experiment, and RTK was still jumping around from 4 to 5. You would say, “Aha! The problem is not distance from the station or trees, but maybe an intermittent electrical connection.”
Maybe the experiment would show solid 6, and the car is cured. You would say, “Well, there is the problem but until Ardurover developers can fix the guidance algorithm to stop being so sensitive to change in GPS status, I need to give up on RTK altogether and rely on DGPS only.” After all, the quadcopter worked fine without RTK, right?
No matter what, in one test session you would get a wealth of information. In my opinion it would help identify a solution quicker than the approach you have been taking.
I hope this helps. I want to see your car running lap after lap at 20 m/s held to the path like a slot car!