The power needs to be isolated and very well filtered, and that’s an important part of a bench multimeter. It’s one reason they cost more.
For a battery-operated multimeter, a buck or boost converter will have high frequency voltage ripple that needs to be properly regulated, because otherwise the ripple in the supply voltage will likely induce noise that affect measurement. By contrast, batteries are not noisy power supplies, and it would be a waste to put extensive filtering in place when designing an instrument meant to run off batteries.
You would also want to make sure that the power supply was isolated. A battery-operated multimeter is isolated by not being connected to other circuits. A typical buck or boost converter is common ground, not isolated. So if you converted a battery-operated multimeter to use a USB power supply, even if you filtered the input voltage well enough, if you had a common ground with a circuit you were testing you could destroy that circuit, the multimeter, or both.
A multimeter will have a safety rating involving electrical isolation. If you hack a multimeter to make it not battery-powered, you are defeating that safety rating and taking full personal responsibility for the consequences of lack of isolation. Depending on what you are measuring, this could be a fatal mistake.
Therefore, just use batteries, rechargeable or otherwise, or a bench multimeter designed to provide this safety.
I hope this makes it more obvious why you don’t find a lot of multimeters that meet your specification.