Apsync/TX1/openKAI questions

@rmackay9 I have a pixhawk2/TX1/ZED set up on the bench with the latest apsync and am trying to configure for obstacle avoidance only (no position sense). Configured as in the wiki it looks as if I have some success as the “Bad Vision Position” is gone but I have no indication of objects visible to ZED in the HUD. The HUD background is solid black.

If I only want obstacle avoidance, do I still configure EKF the same and leave GPS enabled?

In the wiki it states that if I want to use depth sense that openKAI configuration needs to be changed. Is there any place that describes how to set configuration?

Thanks, Randy Bachtell


If you’re only interested in the distance sensing from the ZED/TX1/OpenKai then simply set ardupilot’s PRX_TYPE param to 2 and reboot the flight controller (i.e the pixhawk or similar).

It’s probably best to check that the serial connection between the TX1/APSync and the flightcontroller/ArduPilot is working as well. If you’re able to connect through wifi from your ground station and download parameters etc then the link is definitely working. If that’s not working then there’s perhaps an issue with the serial connection or the SERIALx_BAUD or SERIALx_PROTOCOL isn’t correct. I think you’ve gotten past this though…

…ah, it’s possible you may need to set SERIALx_PROTOCOL to “2” (for mavlink2) but I’m unsure of this.

The OpenKai configuration should only need changing if you’re mounting the ZED camera upside-down or pointing out the back of the vehicle, etc. If it’s pointing forward and is right side-up then no openkai config changes should be required.

Ah, I know what the issue is, The “black HUD” is the key evidence. APSync is grabbing the camera and streaming it to the ground station before OpenKai can start using it.

So we need to turn off APSync’s streaming video on this vehicle. If the wifi connection is up go to this address: a web page should appear that allows disabling the streaming. I think there are two buttons, one to stop streaming now and one to stop the automatic streaming on startup. Both should be off.

At the moment we can’t both stream the video from the ZED and simultaneously use it for the depth map. This hopefully will be possible in the future, Kai (from OpenKai) says he’ll add this.

EDIT: I’ve updated the ZED wiki page just now to clarify that the APSync streaming video should be disabled and explained how to do this. Thanks for testing this and reporting back on issues so we can fix them!

I’m also slightly concerned that the instructions say the camera should be upside-down… i’m not sure that’s true anymore with our latest release of OpenKai. I think it should be rightside-up but I’ll need to check. It should basically work in either case.

Thanks Randy, apsync is connecting and I have the web app, however the web page does not show an option for video. I only have the standard options as the previous apsync version. Flashlogs,filesystem,system control,system status,calibration and flight parameters. They are all functioning properly and it looks like position sense is working. I am just on a test bench but when I set home position it locks in and appears to move if I move the board around the bench. I’m trying to set this up and verify operation before I sacrifice my 960 hex, it gets expensive to repair.

Damn, sorry. I found the proper site. forgot to add the “:8000"

1 Like

We need to bring those two configuration pages together, it’s on the list!

Thanks for all your help Randy, It appears as if this is now working as it should. I will transfer TX1 to the test aircraft and let you know how it does. Cheers, RB

Just to clarify before I fly this beast since we have no visual: bFlip-0 is upside down and bFlip-1 is normal orientation?

Thanks again, RB

I was finally able to get a short test flight with openKai and nothing bad happened😎. I’m not sure what to expect with the proximity reactions. It looks as if it’s working but it’s very vague. I was very cautious so couldn’t really tell. I maneuvered toward my shed and came within a meter or so and stopped manually but at that point it backed away. The snow has returned so no more flying for awhile. My ZED is upside down for ease of mounting so should I change bFip to 1?

Edit: DataFlash log showed good prx distance measurement. I’m running 3.6 dev. Was flying in pos hold, perhaps I should try loiter or alt hold next flight.

Final edit: Clear weather and more test flights. OpenKai/Apsync works great, I played with some AVOID parameters and avoidance with GPS works perfectly (as long as your in Loiter or Alt Hold😎). The wiki may be a little misleading on that regard or perhaps it was wishful thinking on my part.

Cheers and thanks again,
Randy B

Hi @rbachtell
I need your guidance regarding obstacle avoidance methodology using zed stereo camera + NVIDIA Jetson TX2 + Openkai. I am looking forward to implement obstacle avoidance on pixhawk cube running arducopter and would like to inquire the following:

  1. Can you please about the robustness of the technique, how much effective and reliable it is?

  2. Did you find any delay in distance measurement or range degradation issues during your trials using the technique. This is very important in case of high speed copters i.e. 10-12 m/sec as if the distance measurement is late, it is very likely that the copter strikes to the object before it is detected and braking is applied.

Thanks in advance.
Yasir Khizar

Ya sir, I have not continued working with the project and I don’t think much further development has been done. Sorry I can’t be of more help.

Randy Bachtell