We don’t need a horse to get around the map when we have already discovered everything. We just need fast travelling to dungeons we have already visited.
Instead of this wasting time running from waypoint to dungeon just allow fast travel to already discovered dungeons. Allow us to use our nightmare sigils or whatever during the fast travel menu interaction.
It’s an arbitrary time gate that makes no sense in a game like this. It’s something that is done in mobile games to sell items that can eliminate that time gate. It’s not fun, interesting, or immersive to have to travel to the closest way point to get to a dungeon.