The explanation given is there is pretty much nothing that can be done, it is not the server that is under stress but networking before reaching blizzard.
Which is caused by addons. Those addons don’t even need to be installed by me I can be running the game without any addons but other people’s addons collecting data may cause flooding of the network before reaching the server. Which coupled with distance to server causes delays which are multiplicative.
So only option is to move closer to where server is hosted is the answer I was given pretty much.
So all people who have this issue there is nothing that can be done apparently.