A statistical analysis (binomial distribution, p-value ≤ 0.05) shows that for 43.6% of players, the division of starting team is not a purely random 50/50 chance.
Hey everyone! My name is Bl0n, and I'm the developer behind R6 Analyst. As more and more people are using the app, we've collectively compiled a rather interesting dataset of over 21500 Ranked matches, all played in the past 45 days.
I’ve been diving through this dataset, looking for some tasty nuggets of information. And I have found some interesting stuff that I thought I’d share!
Today, I’d like to highlight the issue of how Siege divides up who starts on Attack and Defence, and how there is currently definitely something broken with that!
An analysis of 12871 Ranked matches across 282 players from all over the world, show quite definitively that there is some lopsided bias in the division of Orange and Blue teams. A statistical analysis (binomial distribution, p-value ≤ 0.05) shows that for 43.6% of players, the division is not a purely random 50/50 chance. And a subset of players (<15%) are “stuck” with receiving either Orange (defence-first) or Blue (attack-first) on nearly all (>75%) of their matches.
Almost exactly a year ago, there was a ‘public outcry’ from a subsection of Siege-players, that their matchmaking appeared strangely lopsided. They were getting either significantly more Defence-first (Orange team) or Attack-first (Blue team) in their games. This appeared to be well outside the realm of random chance.
High-profile people in the Siege community, like Pengu and AnneMunition noticed this too, and boosted the issue’s visibility.
Ubisoft responded and in the Y4S3.2 patch, a purported ‘fix’ was implemented. It was documented as:
"FIXED – Fixed an issue with Attacker side logic bias during lobby creation. Attacker and Defender side assignment is now completely random.”
Afterwards there were still some reports that the issue didn’t seem completely resolved. AnneMunition tweeted she noticed this as well.
However, the issue seemed mostly resolved, and things went back to normal.
I do have some personal experience with this as well. I have always, since I started playing in Y3S2, have had significantly more matches start on Defence (Orange) than on Attack. This didn’t change when it was supposedly patched, which is why I was eager to do more research given the chance.
Whenever I mention 'player', I mean 'account'. I will use the terms 'Orange' for defence-first, and 'Blue' for attack-first, as that’s how it is in-game.
So to the actual research: I dove through my dataset of 21511 Ranked matches, tracked by 1750 unique players. The players are from all over the world (at least 87 different countries) and play on all three major data centers (APAC, EMEA, NCSA).
I needed to look at individual players, and how often they started on Orange or Blue. As I needed a reasonable number of matches tracked per player, this reduced the matches I could use. My app is very new (launched in early May) so that limited the data significantly.
I ended up with 282 players who had tracked a reasonable enough number of matches, and between them they had played 12871 Ranked matches. The most tracked by a single player was 241 matches, with the average at 45.6 matches per player.
Of the 282 players, there were 123 for whom the matchmaking appears to be statistically significantly different from a random 50/50 chance of receiving Orange or Blue. This means 43.6% of players are getting something other than a 50/50 random chance for team colour during matchmaking.
For 15% of the players in the dataset, this biased matchmaking is very extreme, with at least 75% of their matches starting on the same side. Some explicit examples are;
A player with 153 matches tracked, with 138 Orange and 15 Blue.
- That's a 90% chance of starting Defence first!
A player with 121 matches tracked, with 107 Orange and 14 Blue.
- 88% chance of starting Defence first.
A player with 241 matches tracked, with 50 Orange and 191 Blue.
- 79% chance of starting Attack first.
A note about the dataset: There are various possible biases in the dataset, especially with regards to MMR and region. However, I don’t believe these (should) have any influence on what team (Orange or Blue) you are assigned during matchmaking. Therefore, I do believe that this sample is representative of the entire Siege-playing population, for PC at least.
Last year, the rumours flying around were that it had to do with your Ping in-game. Alternate explanations had to do with how many you were stacking with, when entering matchmaking.
I always play with a ping of 9ms, and are almost always in a 5-stack. I contacted some of the other players that were getting extremely significantly biased matchmaking (>75% going one way or the other), to ask their usual ping and matchmaking.
Only three players responded.
They all get Orange in almost all matches.
They all have a ping of 9ms.
However, one says he almost always plays solo-queue, whilst the other two mainly queue in 4- or 5-stacks. They get Orange in ±80% of their Ranked matches. One of them has played 168 matches last month, with 134 of them on the Orange team.
I was also able to determine that at least three of the players that are assigned Blue team a statistically significant amount of times, are located in countries that are commonly associated with having a higher-than-average ping.
Unfortunately, this is far from conclusive, but does give food for thought.
I think it’s fair to say that there is definitely still something funky going on with how matchmaking works, and how the Orange and Blue teams are divided. What exactly the cause is, is not (yet) clear.
Obviously, as a whole, Siege is slightly defender-sided. My dataset suggests that getting Orange first does give you a very slight advantage on most maps. But more than that, as the majority of games don’t go into Overtime, this can mean you could play significantly more of a single Side throughout the seasons, which can get boring, in my opinion.
I think it’s up to Ubisoft to look into and verify my findings, and then to acknowledge it’s a known issue and work towards finding a more robust fix than that was applied in the Y4S3.2 patch.
We'll be releasing a few more analyses very soon! One about Overtime, and another about Spawnpeeking. Keep an eye on our Twitter account for more on that!