In a classic client with a public server, you always swim horizontally when entering water. In fact, quite a bit of downward turn is needed.
Also, when you reach the surface, it makes your swim horizontal. In Classic, you keep swimming upwards (thus very slow), except sometimes.
What annoys me most is that I didn’t find water / head bob yet. So that when swimming, I’m still above water when looking up (to see what’s in the distance).
Overall I don’t mind to have the modern swimming - being able to swim up with space (like flying mounts) is quite nice (it used to do a single jump).
Since I don’t always go underwater, I think it’s indeed that it follows the last Right Mouse Button orientation. If you were turning down, you’ll dive. If horizontal or up, you’ll not - or even you’ll swim up if you dive with X.