WoW servers Infrastructure

Support Technical Support
1

I’m very disappointed of WoW servers and really surprised how Blizzard in 2020 still struggle with a poor performance issue. Having this major problem never addressed for a decade should raise question about your technical team who manage the servers.

This server issue been going for a long time, how the server performs poorly and management just watching.
The game can’t be played many times, massive lags, disconnection, down time, multiple restarts, maintenance…etc. Do you know this hurt the business and customers lose the trust on Blizzard ?

What surprise me the most is the Horrific SLA and lack of redundancy in your servers. I been working hosting website servers for fortunate companies around the globe (Sony, Microsoft, Nintendo, DHL, Deloitte, Phillips, Dell… over thousands of companies) and when a server goes down, we bring it back up immediately, also most of the time customers never feel the downtime or any restart since there are always 2 servers minimum in each region backing each other. In times when high traffic is expected due a season or a launch, servers are upsized and ready in advance to manage the load.

But with Blizzard servers nothing of this is present, they launch the game knowing every time there was an expansion there will be high traffic, what’s been done about it? nothing. Hasn’t your tech team heard of AWS ? or Azure? that you can upsize your instance on demand / create new servers on demand for a specific period of time to handle the load ? dedicated S3 buckets or Blobe Store to handle Assets. Apache dispatchers behind each servers to handle cache synching together on top of CDNs. All this can be automated and takes few minutes to upsize vertically or horizontally an instance

I’m really surprised that Blizzard is 20 years behind with their IT infrastructure and every time after a long day at work when i want to have some good time playing the game, i am either unable to logging , keep DC or character is unable to perform any action due to server’s CPU and memory spiking, caused by coding issues, memory leak (lack of testing on stage before going to prod) and poor infrastructure.

If you are using a 3rd party to manage your servers then time to change this, invest on a 3rd party who have a better SLA, solid experienced team or create and automate your own servers.

Review your infrastructure, minimize the cache requests on servers (example a healthy webserver Publisher will perform only 10% of cache while the other 90% is handled by the dispatcher server attached to it, then CDNs support the dispatchers)

4 Likes
2

You seem to think that WoW runs on web servers, heh.

Personally I haven’t had problems you describe as they seem to be almost exclusively limited to Full servers like Draenor. Shadowlands’ launch was in my experience very smooth.

3

Blizzard run their own data centers, they even have their own CDN and internet subnet.

They don’t lease servers off anyone, or pay anyone (meaning third parties) to manage them.

But thanks for the laugh.

4

Thanks for your concerns.
Game servers will use same strategy as web servers, there are assets in a game (like textures for example) all this should be handled on a blob store or an S3 bucket, the dam should be always separate.

If blizzard use their own data centre then that’s the problem, we are not in the 90s AWS and azure are there for this and no data centre can beat what they offer in there instance or the availability they have. Most of companies use AWS and it work cheaper as you only pay for what you are using.

In AWS or Azure you can upsize or clone your servers for a limited time when there is an event, then later get them back to their original size.

When you own your data centre its not cost effective to do so as you will have to permanently pay for it.

Blizzard should know what I’m talking about and I think its time for changing the way they approach the servers infrastructure.

CDN isn’t an issue, is last layer you should worry about and there are many CDNs offers out there or you can use yours if it cover many regions (still owning yours is not cost effective)

In terms of security top Banks in the world use AWS, I don’t see where Blizzard will have a security concern more than a bank to have their own data centre

5

Banks may use AWS for hosting some static or semi-dynamic webcontent.

They do not use AWS for housing financial data. One of my friends works in IT support for Barclays.

AWS has a history of security vulnerabilities.

6

It’s not an AWS issue if you get hacked, it’s your responsibility to secure the server on the top of the AWS encryption, they don’t manage your server for you, they only rent you the box and rest is up to you what you use it for and how to secure it.
There are many layers of security you add to your hosts, making another layer of data encryption and revoking all unauthorised access, injections and so on.

Barclays host parts of their websites with us and many other banks host everything I’m not gonna go on details.
IT support have no access to the servers and have no idea of what’s been used as they are not involved on website servers.
Only dev teams work with us.

AWS or Azure is the way to go for blizzard plus reviewing their servers implementation, caching efficiency, analysed server monitoring

7

Funny thread.

You think WoW loads textures, models etc. on the fly while playing game? Newsflash: it doesn’t. They are stored on the local disk, and at most downloaded from the different server if the client has not finished downloading before starting the game.

CDN? WoW is not some file transfer service.

I am sure you know your stuff about web servers but it is obvious you don’t have even basic knowledge about servers as it relates to MMOs. Youtube has several videos going through the basics and common problems (which are very different from the problems web servers face).

Blizzard in fact uses AWS when it suits the game or service, eg. Overwatch uses them occasionally (and OW is much newer and entirely different genre game).

WoW, as far I know, doesn’t use AWS for actual game servers. You also fail to realize that WoW was originally designed well before any magical cloud stuff appeared (newsflash: they are somebody’s server rooms, too). Legacy code is a real thing and in case of WoW you can’t just conjure a magical new MMO framework (not that such things even exist).

Several Blizzcons have had panels where they go through engineering related questions and stuff. Eg. in 2015 where WoD’s launch is discussed and what caused the problems. I suggest you take a look at them.

This thread is entirely off-topic for the Support forums since you have no actual support questions and you just seem to be grandstanding.

8

You work at a top-tier hosting provider (it seems), yet you continuesly make assumptions.

Rule #1: assumption is the mother of all mess-ups.

1 Like
9

I digged up the 2015 Blizzcon discussion and summary from MMO-champion. Sadly it was not part of the virtual ticket so it was not recorded officially so you have to do with a phone recording but it is good enough. Well worth listening fully if you are at all interested on the actual behind the scenes stuff. Obviously some of the stuff is almost certainly out of date or changed by now:

10

The goal of this thread was requested by Blizzard support after exchanging emails with Activision to create such suggestion in the forum.
Although I prefer this kind of conversation to stay in emails because we all know some people may take this discussion off topic.

There are no assumptions when it come to using cloud hosting services its financially a better option and provide better redundancy also run by professionals with lots of experiences in that specific domain.

CDN is not an FTP service, this is a caching mechanic used to cache data requested by users then served back to other users requesting the same data. Lagging in a game, getting disconnected, load time… most of this is related to caching issues, where all request will hit the server instead of the caching layer which will get the server down or become unresponsive.
In addition to the CDN which are external to the servers there is another internal custom layer of cache used in front of each leg and this play a main role in both security and cache handling, this caching servers should cache 90% and take load from the main server, then main server only left to handle 10% of requests.

There are many 3rd party CDN who excel in this domain like Akamai, Cloud Front …etc