How can I create a code for precision landing?

Ground Control Software MAVLink
1

I have a HolyBro x500 V2 Drone with a Pixhawk 6c and M10 GPS. I have attached an Intel RealSense d435i camera facing the ground and a Nvidia Jetson Orin Nano, and a mechanical claw.
I am using the realsense camera to perform object detection. I want to write a code using pymavlink that, once an object is detected, make the drone land on top of it. I understand I cannot use the GPS as it is too imprecise. My current idea is to create a feedback loop where I will calculate the center of the bounding box of the detected object, and use attitude movements (pitch 5 degrees for 0.2s, Roll 5 degrees for 0.2s, etc.) until the center of the bounding box is within a range of pixels that under the center of the drone. Once this has happened, I will tell the drone to land.
Is this the most accurate/easiest way to do it? Can you think of an easier way to achieve the same goal? I don’t really care about speed too much.
Edit: I have just seen this page: Landing Target Protocol | MAVLink Guide and was wondering how/if I could use it for my use case. I also want to test it in SITL MissionPlanner before doing an actual test.

2

Does anyone have an idea on how to do this?

3

There is documentation on that and some Blog posts on this forum.

4

Hi @Anay_Gokhale ! You can check this out: GitHub - snktshrma/precision-landing: UAV precision landing package for Ardupilot

Instead of the landing target, I used position target to correct positional error. I won’t say this is the best technique, but I think some modifications for your requirement will make it good to go for your task.

5

Hey do you know how the drone moves when it is given position target? As in does it use gps or does it change attitude in order to move to the relative location? My GPS is not very precise so I do not want it to be used for the precision landing part
What modifications do you recommend?

6

@amilcarlucas I wanted to test it in mission planner first so I created this code:

import time
import numpy as np
from pymavlink import mavutil

# Connect to SITL
connection = mavutil.mavlink_connection("tcp:127.0.0.1:5762")
connection.wait_heartbeat()
print("Heartbeat from system (system %u component %u)" % (connection.target_system, connection.target_component))

# Request data streams
connection.mav.request_data_stream_send(
    connection.target_system,
    connection.target_component,
    mavutil.mavlink.MAV_DATA_STREAM_ALL,  # Request all streams
    10,  # Frequency in Hz
    1  # Enable streaming
)
# Camera Field of View (Adjust if needed)
CAMERA_FOV_X = 70  # Horizontal FoV in degrees
CAMERA_FOV_Y = 50  # Vertical FoV in degrees

# Function to send LANDING_TARGET messages
def send_landing_target(angle_x, angle_y, distance):
    """ Sends a LANDING_TARGET MAVLink message to SITL """
    connection.mav.landing_target_send(
        int(time.time() * 1e6),  # Timestamp (microseconds)
        0,  # Target ID
        mavutil.mavlink.MAV_FRAME_BODY_NED,  # Frame of reference
        angle_x,  # Target angle X (radians)
        angle_y,  # Target angle Y (radians)
        distance,  # Distance to target (meters)
        0.2,  # Target width (normalized)
        0.2,  # Target height (normalized)
        0, 0, 0,  # X, Y, Z (optional, set to 0)
        [0, 0, 0, 1],  # Quaternion (default orientation)
        mavutil.mavlink.LANDING_TARGET_TYPE_VISION_OTHER  # Target type
    )
    print(f"Sent LANDING_TARGET: AngleX={angle_x:.3f}, AngleY={angle_y:.3f}, Distance={distance:.2f}")

# Simulate object detection and send landing target messages
try:
    while True:
        # Simulated landing target coordinates (Replace with actual detection logic)
        obj_center_x = 0  # Assume center of image (640x480)
        obj_center_y = 0
        frame_width = 640
         frame_height = 480
         
        # Convert pixel coordinates to angles
        angle_x = ((obj_center_x - frame_width / 2) / (frame_width / 2)) * (CAMERA_FOV_X / 2)
        angle_y = ((obj_center_y - frame_height / 2) / (frame_height / 2)) * (CAMERA_FOV_Y / 2)

        # Convert to radians if needed
        angle_x = np.radians(angle_x)
        angle_y = np.radians(angle_y)
        # Simulated depth camera measurement
        estimated_distance = 2.0  # Assume 2 meters to the target

        # Send the MAVLink landing target message
        send_landing_target(angle_x, angle_y, estimated_distance)

        time.sleep(0.5)  # Send messages at 2 Hz
    except KeyboardInterrupt:
    print("Stopped sending LANDING_TARGET messages.")

Before I began, I made the sim drone takeoff through mission planner. Then I switched the mode to GUIDED and ran the code. When the landing target messages were being sent, I switched the mode to land, however the drone landed straight down and did not move at all. What am I doing wrong?