Trying to find any sort of fantasy manga with a married couple as the main characters
Preferably without harem

The only one I can think of is Risou no Himo Seikatsu. While there are strong fantasy elements it is largely about political drama involving Royal and Noble families. The characters are likeable and well written but it can get a bit expositiony

Lv2 kara Cheat datta Moto Yuusha Kouho no Mattari Isekai Life Immediately comes to mind. They are very much monogamous and have a nice power couple dynamic.

I don't think it is classed as fantasy but the first series which came to my mind was [url=https://www.mangaupdates.com/series.html?id=148209]

Retired Heroes has a married couple with a kid. Story centers around the family in a fantasy setting with magic and powers, the mom and dad used to be famous heroes hence the title. Really good healthy, married relationship imo