Use a discharged battery or adjustable power supply and calibrate the Flight Controllers voltage sensor at a low voltage, using you digital multimeter as the reference. It will be closer than the cheap battery monitors.
Most standard “power bricks” that come with the flight controllers are a little non-linear, so best to have the voltage being accurate around the level where you’d expect a battery failsafe.
Set the sensor type to “Other” and follow the instructions to set the voltage shown on your DMM. https://ardupilot.org/copter/docs/common-powermodule-landingpage.html
Multimeters can run from free (Harbor Freight gives them away with a purchase) to a lot of money for a branded meter like a Fluke. I have several from cheap to expensive. A $25 one from the same Harbor Freight I picked up while in the field has been working as well as my $150 Fluke for Hobby purposes. But what’s available would depend on where you are I imagine.