As we all know, Blizzard are exposing their own rating for M+, which is a copycat from RIO with little to no changes so far, in the game. This is a big mistake and in this post I will explain why.
First, let’s check what RIO does. On paper, it does nothing bad, it just shows which instances you completed on which level to others and lets them make a decision on whether to take you or not. The problem is in that last part - that it lets other people make a decision on whether to take you or not, based on the numbers presented. Because these other people are people, their decision is subjective.
You could ask - but what about the numbers presented? The decision, while subjective, is still more informed than it would be otherwise, no? The answer is that it does not really work that way.
A simple example: let’s say there are two people in the queue with equal rating, both specs are a good fit for your forming party, but one of the specs is meta and the other is not. Who you are going to pick? You are going to pick the meta spec. It’s not a question, by the way, it’s a statement, this is true de-facto, it’s in the numbers. So, the rating was telling you that two characters were equal, but you still had a clear preference. The chances of the two characters should have been equal, but you made them unequal. You introduced your own bias.
The community has many such biases. They are introducing an error into each decision. And what’s worst, because the beneficiaries of these errors complete more instances and get more rating and get more gear and, yes, get more experience as well, these errors snowball.
The truth about RIO is that it is NOT a measure of capabilty. It is much more a measure of how conforming you are to the collective biases of the community. This does NOT mean that a non-meta spec that completed some +10s is a better fit for +15s than a meta spec that completed some +15s. No, the latter guy is a better fit for +15s currently. He has more gear and more experience. The problem is, the latter guy has more gear and more experience and the former guy less of everything because the system was reflecting community biases at every step and this was boosting the latter guy and drowning the former guy. If the former guy was being accepted at the same rate as the latter guy, he would have been on par with him wrt all of gear, experience and rating. But he is not and that’s a problem.
Ratings are a good thing. But ONLY when they are paired with automated matchmaking. Whenever they are paired with manual matchmaking, they are overrun by the collective biases of the community. Wrt WoW and M+, ratings don’t matter at low instance levels (eg, before +5, because that’s easy, risks are low) and at high instance levels (eg, after +20, because if you got up there, you have everything and picks are going to be based on setups, who knows whom, etc). But they matter in the middle, and, unlike in PVP where ratings form a pyramid and most people are in the low portion of it, in M+ ratings form a skewed bell-shaped figure and most people are in the middle. In the middle, where ratings are the main tool used to compose groups.
So, RIO is a bad idea, because all ratings paired with manual matchmaking are a bad idea. And Blizzard’s own rating is a bad idea for the exact same reason. Blizzard’s own rating is an even worse idea than an external rating from RIO, because a built-in rating is going to be used more widely and so the biases it spreads are going to snowball harder.
By introducing their own built-in rating while still staying with manual matchmaking, Blizzard are:
- amplifying all issues with meta vs non-meta specs,
- making it much harder to balance specs and classes, because they plain won’t see half of them in material amounts other than at low keys,
- letting a lot of people who play unfavored specs down, forcing them to “deal with it” by switching to a “better” spec or class, making it all even more imbalanced.
An automated matchmaking system would avoid all three of these things. Alas, it seems we cannot have an automated matchmaking system, for whatever reason. But if we cannot have one, then at least Blizzard should not be taking things that are already bad and make them worse! If anything, instead of legitimizing the idea of using rating in a manual matchmaking system that de-facto amplifies community biases, they should have made it MORE difficult to check what rating you are: by taking away API means to show tooltips in an LFG window and by stopping tracking highest completed instance level altogether in the armory.
Built-in M+ rating is a big mistake, which is going to cause a lot of tears, a lot of apathy and disillusionment, and will ultimately make M+ a worse place to be and will make less people participate.
(Really sorry for the wall.)