We are getting more into the 4.1 version which actually has pretty good features. Today we have been trying to use the lane switching between compass yaw and GPS position, and compass yaw and optical flow (PX4flow). We set the source parameters as follow:
Before flying, we performed a ground test which was to arm in loiter mode, lift the drone and carry it around, and then switch to lane 2 that is the one with the optical flow. After switching, it triggered the EKF failsafe because EKF variance. Taking a look at the logs, it seems that there are not corrections at all after the switching:
If I try to arm selecting the second set of sources, it does not arm and throws a pre-arm message about not having position estimation. From my experience with the EKF2, I do recall that an origin and a home were required for triggering the position estimation using the optical flow sensor. I suspect this problem is solved when we have GPS in the first lane since it sets the home and the origin as we could see in QGC.
I gather I am missing something in the configuration, may be we need to set differently the EK3_SRC2_POSXY. Also, we noticed, for long time, that the quality parameter of the PX4flow sensor is always 255 which means is always healthy. This is something that happens if we flash the version that is available for arducopter in QGC; an KLT algorithm one. When we compile the firmware, something we have done for changing the default parameters of the PX4flow since it does not have EEPROM, we noticed that actually the quality parameter changes depending on how good is the matching between frames, is it somehow used this quality parameter for any check in Arducopter? We got it when we use the hex optical flow sensor. I can help to take a look at the commit it is currently used and see what may be the problem.
Is there a separate lidar on the vehicle or just the px4flow sensor? It’s mentioned on the wiki but in case you missed it, when using any of the optical flow sensor a separate lidar is required. The lidar incorporated into the px4flow and hereflow are really not usable.
By the way, I normally don’t make hardware recommendations when there are multiple options available but I think the hereflow, cxof, etc are all better than the px4flow sensor. They are all smaller, lighter, cheaper and in my experience at least, they perform the same or better.
I’m looking into optical flow modules and considered the CX-OF but that seem difficult to find now. The ThoneFlow-3901UY is the one I’m probably getting. Its less then $15 and can get from Seeedstudio or Digikey. Digikey looks to have US stock and only $1 more than the china Seedstudio option.
Yes, we have a LiDAR, and according to the logs it seems like working correctly. Regarding the optical flow sensor, the reason why we still use the PX4flow is because we have access to the source code and we have certain flexibility for changing the lens and certain parameters such the exposure and certain thresholds in the algorithms. What we noticed with the hereflow indoors was that the light affected it more than what it affected the PX4flow; below 50 lux according to my records it does not perform well. However, I would like to do more testing for confirming these findings since it was a long time ago, and the advantages you mentioned make the effort worthwhile.
I repeated the test today, the curious thing is that it seems such it is not fusing the optical flow. As it can be seen in the plots below, as soon as we switch to the second set of sources, the variances and innovations of the positions and the velocities stop being calculated:
This is something that we have seen happens when there is no correction, measurements, available. I guess the EKF variance triggers because the lack of correction. Again, the test was not a flight but me arming the drone, without propellers, and carrying it around. I switched to source two during the “flight” and that is what is shown in the plots. The log can be found in the link below:
In the Lua script, I have something similar to what you have in the ahrs-source.lua script:
I don’t see any downward rangerfinder data so I wonder if you are sure that one lidar has been setup downward facing and with RNGFNDx_ORIENT = 25?
I wouldn’t worry too much about the position and velocity innovations flat lining. The optical flow sensor data is not a position or velocity so I think this might be normal (I’ve never checked but I wouldn’t be surprised).
All right, we finally got it working. Thank you for your feedback @rmackay9 ! I noticed certain things that are interesting to evaluate for future applications, please forgive me if this post is a little bit long.
Yes, you were right about the RNGFNDx_ORIENT = 25 parameter, we did not set it up correctly in this drone. That was our mistake.
We had tested the optical flow sensor configuration on ground, swinging the drone and comparing the gyros and the flow measurements, as shown below:
After we set the range finder correctly, we still had issues in the ground testing. It triggered EKF variance anyways when I lifted it and switched position. Again, when I switched to source 2 (the one with the optical flow). It is like it stopped fusing the data, or it never did with the optical flow, since the innovation of the velocities did not change even if there were available optical flow data:
In that same log, after switching back to source 1, I was able to arm using the source 2 without the pre-arm check of position required. It looks again as it was not fusing the velocities, but allowed me to move in loiter mode without EKF variance:
We decided to fly it since we knew if we armed in loiter mode with the second source it was not going to throw EKF variance and land. The optical flow did not seem to be working at all since the position was not consistent and it was drifting all over the place, but this time it looks like the flow itself was not received though it was not a wiring issue since the gyro data was still received:
We tried with a different sensor, we reflashed it, and we also used the mavlink inspector and the QGC and it did not seem to be a bad sensor.
We decided to try again the hereflow optical flow and it worked. Now we know, as @rmackay9 that we should not expect the innovations to change when we switched to source 2:
Some conclusions, there is definitely something inconsistent with the PX4flow. As I pointed out, this may be related to the version of the Firmware of the PX4flow that is used and that it is always reporting quality 255. The performance with the hereflow was great in loiter mode though it jumps when we switched from optical flow back to GPS, which I believe it is a consequence of some error accumulation in the position estimation. A part of that the vehicle flew great. Apparently, all our problems were due to a bad parameter and to the PX4flow itself. I wonder if we can still connect the PX4flow with a newer firmware using Mavlink since it has a UART for this, but I guess we will just need to try.
To be clear, the firmware on the PX4flow sensor is the PX4Flow-KLT firmware recommended on the wiki? If it is this firmware but it’s not working then I could re-test it to be sure we haven’t broken the AP driver but TBH, I’d like to avoid this testing if another firmware is being used that could be the source of the problem.
AP’s EKF does make very limited use of the optical flow quality value… but it only checks if it is 0 (unhealthy) or >0 (healthy) so there isn’t any additional advantage to having a more granular quality value.
I’ve added the issue re the position jump to the Copter-4.1 issues list. Maybe we aren’t probably sending or handling the position reset event. This should be fine but perhaps we’ve broken it somehow.
yes, I think that is is the firmware that is upload by QGC. The problem with the quality parameter is that it is always 255 even if we put the drone in a completely dark room. Thus, it will always be reported as healthy. I will re-try anyway today to do more ground testing with the PX4flow and see if I can discover something in the Firmware using GDB. However, we used it with the same firmware and set-up in the version 4.0.7 and it worked. What I am going to do is to re-flash the 4.0.7 and try it just to make sure everything is correct, then I will reflash the 4.1.0-beta4 for testing.
Did you solve the quality problem?
I’ve tried 2 versions of firmware, the first one is provided by wiki.
The readout quality value is always 255, the same problem as you mentioned.
I was wondering if other provided information is valid? (flow_comp_x ,flow_comp_y…)
The second version of firmware is provided by QGC (v3.3.1), and the readout quality value changes from scene to scene. I thought this version of firmware is more riliable.
Since I am trying to implement drone hovering indoors ,so the readout value from px4flow is important to me.
I don’t personally know the differences between the more up-to-date firmware provided by QGC but I think it would be best to stick with the AP specific firmware that we know works.
If you can’t get the optical flow feature to work could you provide an onboard log? Maybe we can spot what the issue is.
We stopped working on that since the hex optical flow worked decently in our environment, so we did not solve it. A couple of thoughts for you if you decide to keep on trying it, strongly recommending you to do a lot of ground testing before flying:
1)The quality parameter is a measure of how good the pixels were redetected between frames, of course the higher the more pixels that were able to be matched. This is why it is very important. This is solved in newer commits that you can find in Github. However, be aware that the algorithm for matching pixels is different in newer tags and commits. Arducopter uses KLT.
2) The newer branches had an issue with the on-board sonar. They implemented a “Kalman filter” for the altitude and the Z speed. It does not work well and it stops reporting altitude because it easily diverges. Thus, if you try the new Github tags, be aware of this.
3) Since it is not directly compatible with Arducopter, I suggest you to try Mavlink using the UART connection instead of the I2C. I did not try it, but that was my plan since there are standard supported Mavlink messages in Arducopter and the PX4flow is able to send them since it has the Mavlink library.
A final thought, my motivation for using the PX4flow was the possibility of playing around with the code. The PX4flow has parameters that can be helpful to tune for your environment such as the exposure of the Toshiba sensor; indeed some people complained in the past because the default set is not the best in some cases. Since it does not have an EEPROM, the only way to play with them is to change them on the Firmware and reflash. I can see why you may want to use it. However, it is a project that is not frequently reviewed and that rarely has new commits. Keep this in mind because that decreases your chances of getting help.
