“there is no point fighting botting and RMT because if you heavily inconvenience them, they will find a new, much less effective way to continue.”

In my opinion, fighting botting needs to start at gameplay, like OP suggests, but i would first start with small changes that most would not notice.
For example, vendor greys, greens, they should be removed from popular botting targets. (Things like spider silk being farmed for the AH is actually fine, since that gold isn’t raw, it just exchanges hands, doesn’t contribute to inflation).
Add those greens to mobs and areas that bots would find difficult, such as elite camps, where players would be incentivised to group and play together.
Add soft mechanics such as spells that you need to Line of Sight or death, which very long cast times.
Funnel wealth and value into questing, as shown in SoD so far, it works quite well to keep normal players head above water in an economy that shoots upwards due to botting.
Games like this, when they don’t evolve with modern times, are open to hideous exploitation, as we have all seen.
Mages god modeing through dungeons? Nerf what enables it.
Rogues pickpocketing through under the floor using flyhacks? Nerf what enables it.
The problem with Blizzard and botting is that they don’t change the game enough, but when they do, its awful, see experience level difference from the tbc boosting nerf.
tldr: target nerf botting strategies until there are none left.