Feels like the devs don't understand what's wrong with the MMR system
I read through the official news topic here: https://forum.deadbydaylight.com/en/discussion/315687
All it does is talk about how some people favor fast queue times while other people favor fair matchmaking,The way they describe how matchmaking tries to find people close to your MMR and then expand over time to find people further away from your MMR is normal for any kind of matchmaking system, and it sounds based on that post that they'll just experiment with these numbers.
But it doesn't adress the issue that the way people's MMR is decided to begin with is the core of the issue. Based on interviews I've read, MMR for killers seems to be purely based on whether or not you get kills or let survivors escape, and for survivors it seems to be based on whether or not you escape or die.
Do the devs understand that there's more to the game than living or dying? If you're able to play like a god (on either side) and make the match miserable for whoever you're facing, and then be able to lower your MMR anyways by choosing to die at the end or let the survivors escape... Then you can keep facing newbies forever.
And that's the problem... Not everybody plays to win, personally I enjoy dying to the killer at the end, and after reading the interviews about how MMR is calculated it made so much sense why I keep matching with newbies despite having over 1500 hours.
To fix the MMR system, they need to look at more than just whether or not survivors live or die.
the MMR system needs to look at more variables than that. For example, how long a chase lasts before the survivor goes down, percentage of missed skill checks, percentage of missed swings, how fast a killer is able to locate survivors, how much time a survivor spends idling without progressing the objective. These are just examples, the most important thing is probably how long chases go on for before a down, since that's the most skillful interaction on both the survivor's and killer's side.
I apologize if this is very long, it's just concerning that nothing about the MMR calculation itself is mentioned when they talk about changes, and that they only mention how these numbers are planned to be used while finding matches.
Comments
Where does it state that you win when you escape? The game doesn't say "you win!", Hence this is an opinion that you're treating like a fact.
The reality is that this is a matter of opinion and I've met more people seeing it from the team point of view, yet both sides exist.
Because, counting it the way you do... How do you count a killer win/loss? Killer is on the opposite side, meaning that if his opponent loses then he wins, and if his opponent wins he loses. If you are counting survivors individually then you would have to count the killer as having won or loss against each survivors individually.
There are other games similar to DBD that also go for the team way of counting wins and losses.
Regardless of your opinion on this, it is completely unrelated to the bad MMR system.
As i have explained too many times now:
- MMR systems are meant to give you a score based on your SKILL AT PLAYING THE GAME
- The matchmaking then attempts to match you with players close to your score
- Whether or not you live or die has almost nothing to do with a player's skill at PLAYING THE GAME, hence it defeats the purpose an MMR system is supposed to serve.
I dont know what's hard to understand about this.
Of course, we can get back to the point.
So, the thing that you're still missing about the MMR system - or, I should clarify, about MMR systems, from my research they all seem to work like this - is that you take a broader view. Zoom out from a specific match, look at all the player's matches, and ask the following question: Are they winning more than they're losing?
If the answer is yes, it stands to reason that they're more skilled. If they're not, it's usually a balance issue.
It works like this for other games, why wouldn't it work here? DBD isn't more complicated than, say, Valorant or League of Legends. Why would it need a more complex system? What doesn't work about the logic I just outlined?
Just to be clear:
Under the current in-game tutorial: "Survivors: How To Win - The primary objective of a Survivor in a trial is to escape."
There's absolutely nothing about needing to escape as a team mentioned in order to win. If it were, it would say something like, "The primary objective of Survivors is to have 2 or more survivors escape in the trial." It doesn't. There's no debating this. Is working as a team absolutely needed in this game? Sure, no denying it, but the teamwork is only limited in the scope of each individual survivors needing to work together so that they themselves can escape, nothing more, nothing less.
Teaming is a game strategy, not the game's end goal. It's like trading resources in Catan. You don't normally trade with the person in the lead with the most victory points, but you often have to team up with other players so that you yourself can eventually get ahead.
If many of you look under Steam / Epic's offical DBD website under Key Features, it even mentions this:
"Survive Together... Or Not - Survivors can either cooperate with the others or be selfish. Your chance of survival will vary depending on whether you work together as a team or if you go at it alone. Will you be able to outwit the Killer and escape their Killing Ground?"
They've been very clear about this - it's obvious that people who says otherwise are arguing in bad faith, or just closing off their eyes and ears like children because they don't want to accept it, given it runs counter to their narrative.
That said, would it have been a better game if this game was based off of a team-oriented objective? In my personal opinion, yes. But it's not what I want - or what anyone else wants. It's about what it is, and isn't. And as the tutorial indicates under "How to Win", that's not what this game is. It's better to just accept the facts, and have honest discussions about it than to continue twisting the truth around to fit a false narrative.
Chases must influence elo/mmr points.
it baffles me that you agree to "get back to the point", yet you still use your opinion of a win condition as an argument to the MMR system. I've been trying to tell you that winrate has nothing to do with SKILL in this game, yet you keep going back to winrate.
and no, this is not how MMR systems work in every game. Games generally adapt their ranking system to judge what's "skillful" in that particular game.
In CSGO, you can win a match and still have your rank lowered because you did poorly. You can also lose a match with a great score and either rank up or not rank down as much (i forget which, either way the loss doesn't hurt your rank as much).
In CSGO, however, it's your amount of kills VS deaths that count for the most, but in those kind of games this makes sense as having a high K/D ratio shows you're more skillful than your opponents.
I feel like maybe the developers have looked at these kind of games and decided to use the same way to count a person's MMR, but DBD is a completely different kind of game. It's not a team VS team where both sides are equal in power. It's a 1v4 where the killer is much stronger than each individual survivor, which is meant to create a balance in a 1v4. Hence when it comes to escapes VS deaths, the side with 4 people needs to be counted as a team, because 4 people on that side is going against 1 person on the other side together.
On top of this, a person's individual skill needs to be counted, and a person's individual skill can be determined through direct interactions with the person on the other team. I'm NOT saying kills, deaths or escapes shouldn't count AT ALL... But it SHOULDN'T BE THE ONLY THING that's counted.
----------------------
Generally, how wins and losses (no matter your opinion on how they are counted) should work in an MMR system, is that they act as a multiplier to your overall match rating.
Let me make a proper example of how a proper MMR system should work. This has to work differently for killer and survivors, but for the sake of simplicity, this example is for survivors. For this system, a player's personal rating during a match is counted as a number between 0 - 100. If your number ends up below 50, your overall MMR goes down, if it's above 100 your MMR goes up. Your number starts at 50 when the match starts, and actions you do during the match either raises or lowers it. For example, chases could raise or lower your number depending on how long they last (but let's not get too much into the system itself, that's not the point of this).
Now, let's get to the part where KILLS, ESCAPES or DEATH come into play.
How you would generally develop this, is to apply a multiplier to the player's final score depending on whether the match was won or lost. This multiplier needs to take 6 different scenarios into mind for survivors:
(an advanced system would make special multiplier cases too for things like Myers Tombstone addon that can instantly kill a survivor right away, but ignore that for now)
All of these scenarios would have a different multiplier towards your personal score, and takes into account both team wins and personal deaths. Any examples of actual numbers are imaginary, as these numbers highly depend on the design of the base score system. Keep that in mind...
For example, if Steve does very poorly in the match, goes down almost immediately during chases and ends up dying, he might end up with a low base score of 20. His team, however, end up doing all 5 gens and escape! These 3 players will have the "match won, and you escaped" multiplier, boosting their personal match rating by for example 50%. So even if LAURIE did okay but not great, and had a rating of 40, her multiplier boosts it to 60, so her MMR goes up thanks to her escaping.. STEVE, however... He did so bad that his rating was at 20, and he also died. He then uses the "match won, but you died" multiplier, which in this case I'd put at a 0% change since it was a 1v1 with the killer and he lost, yet the team won. His rating is still 20, and his MMR goes down... Now rewind, imagine Steve survived! He still did poorly, got downed immediately during the chases, but his team mates came back and saved him at the end with Borrowed Time and took hits to get him out. Steve's rating is maybe ~26 because he "won the chase" at the end there. His rating of 26 is now multiplied by 50% because he survived, but this ends up as 38 which STILL ranks down his MMR due to how poorly he did. Similarly, if LAURIE had died at the end with a rating of 40, her escape wouldn't have boosted it past 50, and her MMR would have gone down.
TL;DR:
i apologize this message is long, but i ended up explaining how a proper MMR system is supposed to work, where it doesn't only care whether you lived or died, yet these things can make a big difference on whether or not you rank up or down. I am not going to have any further conversations on this unless it's clear that whoever responds to me has read the entirety of the 2nd part of this post (everything below the line of dashes). Sorry if this sounds rude, but it gets tiring when people completely ignore good arguments and points about things and then having to repeat myself, so i took the time to properly make an example of a proper MMR system in this post.
They cannot do it as they clearly do not know what is a good DbD player. Just look at their stream when devs are playing... they do not even know the basics of high skill play... on any sides.
Old rank system was working better giving funnier trials and a better mix between try hard and competitive matches. Moreover, you could not end up in a ELO hell.
MMR must be removed, it doesn't work and will never work if they don't hire someone that actually knows how to develop a good system.
100% agree. Having no MMR system would be better than having a bad one.
This influencer is talking about it here and yes, numbers are decreasing since the SBMM release... it's crazy that BHVR didn't not remove it. We have lost half the players on peak times and 30% on average... It is so huge and it is still decreasing. EVERYONE i know hates the SBMM, EVERYONE.
Again, not my opinion. Someone above your message also outlined this, the game's win condition is not ambiguous and not up for debate.
So, I did a little googling and CSGO does seem to be an exception in this regard, though nobody seems to know how that game's MMR system works? So we clearly can't be looking to it for instruction, since there are no instructions to be found. My previous googling on the topic covered six other games (DOTA 2, Rocket League, League of Legends, Smite, Valorant, Rainbow Six Siege) and found that all six of them have an MMR system very similar to DBD, as well as TrueSkill being a system that's based entirely on wins and losses. I don't know exactly what games use TrueSkill, but Microsoft came up with it so I'm assuming at least Halo does? Either way, this does seem to be the norm for MMR systems, so your suggestion for a "proper" MMR system seems to be breaking the mould.
The only exception in my previous googling was Smite, but that was because it also used player level in the matching portion, the number seemed to be calculated the same way.
Regarding your suggestions for a moment... they still suffer both from attempting to change the game's win condition in a manner I'm not convinced it supports, and being far too hung up on expecting players to prove their skill with specific actions when that simply isn't always feasible and gets messy very quickly. This is how the emblem system works, and we saw how badly that turned out; better to just use the proxy for skill when looking at aggregate games, don't you think?
But to focus on your example again, we're outlining Steve's play in one game. One game is statistically irrelevant for any MMR system, we need to know how he plays overall; it could be that his escape this time was a fluke and he normally dies, or that his poor play in this match was an outlier and he's usually more of a team player. These are the questions that matter for a matchmaking system, not how he played in one single match.
for the love of chthulhu stop being so hung up on "win conditions". it has nothing to do with the MMR system.
my suggestion mentions amount of survivors escaping or dying as a condition to apply a multiplier to their personal performance during a match. Don't think of it as a "win condition" then, just a way to measure how well each side did during the match, and even then it's more important how each player did individually as my example showed. I'm not sure if you read my example properly since you're misunderstanding even the most basic of things, and unfortunately I can only explain things, but I cannot understand them for you.
I feel like you're either trolling me, why would you think this suggestion only outlines ONE game? OBVIOUSLY the outcome of each game will either raise or lower your OVERALL MMR, not just be like "oh hey last game you did well, now you get sweaty killers for ONE match!". Like bro, you still need to calculate performance during each match, and then use that calculation to adjust MMR. It doesn't mean that match is the only one that matters.
You're going to have to run that by me again - we know the MMR is calculated based on whether you escape or not for survivor, which is the win condition of the survivor role. Similarly, it's calculated based on how many kills you get as killer, which is the win condition of that role. It's pretty clear that the win condition has quite a lot to do with the MMR system, since it's almost the only thing the current system is based on? Is this still you insisting that's "my opinion" of the win condition even after it's been explained that's not the case?
As for mentioning that your suggestion only outlines one game, I could've been clearer there- my point is more that even in that one game, it very succinctly highlights why looking at anything other than win/loss ratio over a large aggregate batch of matches gets unnecessarily complicated. The Steve in your example, if he plays consistently the way described, is going to lose more matches than he wins. Measuring how effectively he ran the killer, or any other stat you'd care to track, just isn't necessary; the end result on average points to the same conclusion if you just look at his winrate.
Like I said earlier, I looked up a couple different games and their MMR seems to work like this too. This concept seems to be pretty well-trodden ground for game devs, it's not come out of nowhere and it's not as obviously stupid at some people seem to think.
I get where you're coming from but just want to point out two things. first being the rng aspects of this game and why they, imo, are what differs from any other pvp game with MMR.
the second regarding your basketball analogy.. or any sport for that matter involving teams.. where the goal of the game IS ultimately to win the game by outscoring the opposing TEAM claiming the contribution of, let's say, the [current season scoring title holder averaging 60 points a game] best player on the floor is "not skilled" is assuming that this one player is solely responsible for the team and very unfair. So instead of using hockey, or basketball... any team game.. an analogy I can get behind despite agreeing with both sides would be an MMA fight.
the arguing side is saying this essentially...if a fighter, on average after 10 5 round fights, was going into the last round up 4 rounds to none and proceeds to dominate the final round but at the 10 second mark they were knocked out/caught in a triangle choke and tapped out, are they "not skilled."
argument here goes both ways bc one could say yes, they arent "skilled" bc they leaned into the wrong counter, got knocked ou and lost, up in the score card or not. winner=skilled. where as the other side of the argument can say they are technically the more skilled fighter winning every round but end up in an unlucky/######### situation where, had they continued a bit longer, would have won. or in dbd speak.. the problem is when a survivor is looping a killer without getting downed, doing gens and making hook saves all game just to (and this is where the rng factor I mentioned earlier comes into play) for example, get found by the killer at the precise time the last Gen is finished and noed procs, downed, hooked and left to hang by the other survivors.
that, as one example of rng affecting the outcome, is why neither side of the argument (everything done leading up to and including escaping vs escape rate alone) can fairly be assessed in MMR.
if that makes sense. js.
It's...tricky.
As someone who started under ranks and moved to what I'd guess is intermediate-ish skill since then, I'd much rather be a newbie now than then. Maybe 2/3 of my games were against veterans smurfing, or simply people with dozens of times my hours.
Now...maybe 3 games in 5 are pretty high quality.
I'd like to see survivor MMR be a bit more nuanced (I'm okay with the way killer works) but it's a very difficult task.
Idk how but I just went through a long ass comment, edited a typo and ended up deleting it, explaining what I will here tldr
where I see your point... two things i wanna point out are 1) using any team game as an analogy is disingenuous bc the argument relies on putting all the responsibility of a team's overall goal (outscoring the opposing team) on one's shoulders.
an easier analogy, imo, would be an MMA fight.
what the other side of the argument is saying is this...
if a fighter, based on an average of 10 5 round fights, is up 4 rounds to 0 going into the 5th (which he also dominates on the scorecard) is knocked out with 10 seconds left, does that really mean he's the unskilled fighter of the two?
where your point holds water in that every decision up until the point a win or loss is determined matters and essentially is voided out bc, despite performing better for longer, the fighter still technically lost, I just want to address the second point I mentioned earlier and why it causes this dilemma.
rng.
when a survivor loops the killer without being downed, does gens and unhooks other survivors the whole match just to get caught by the killer at the precise time the last Gen finished and noed procs, camped and left to rot by all the other survivors....where I do see and Agree with both point of views, neither can be implemented due to the rng factor essentially flipping the outcome of a match on its head at the drop of a dime.
where one side is right in they did show "skill" all match in everything they did, truth is they died.
and where the other side says, you lost and well, the goal is essentially to win (escape) the fact that they were killed isn't necessarily a representation of a lack of skill, just the outcome of a random, inconvenient circumstance.
Correction. They don't know how a 1v4 game should be balanced in general. Let alone the MMR......
It's important to understand that the utility of MMR is even more important and helpful in games with RNGs. As I emphasized in the previous posts, MMRs account for *everything* that happens within the game, and that includes elements of any RNGs. So *over the course of many games*, MMRs will still end up matching players with similar skill levels - if it doesn't, then it indicates a more of a problem of a balance issue (e.g. - RNGs only favoring certain players.) That doesn't question the validity of MMRs - just the balance problems.
As I stated previously, MMR works for *any* pvp game. It doesn't matter whether it's 1v1, 10v10, or 1v10. In a team-based victory games (like basketball), each players within the team is absolutely responsible for the whole team. It's never enough to just to know the individuals in your opponents' teams, but they should be able to understand the strengths and weaknesses of the players in your own team, and cover for them (and be covered) in order to attain consistent victories. Sure, assigning specific 'positions' or 'roles' in team-based victory games should mitigate some of that, but if that isn't enough, then "not-my-job" type of attitude often leads to instability within the team, and then losses. Ultimately, individual players are responsible for winning or losing as a team, regardless of whether people think that's unfair.
This is why the biggest names in MMRs like ELO and TrueSkill base their algorithm mainly on win/loss only, and not on individual performances - so that individuals don't prioritize their own personal goals over the team objective.
Here's what the creators of TrueSkill has to say on the matter:
"It is interesting to note that Elo’s update equation depends only on the win/loss outcome. Similarly, TrueSkill’s update equations take into account only the finishing order of the players/teams involved. None of these systems incorporates the actual final score, say, the number of kills in a shooting game or the finishing time in a racing game. This is a deliberate choice.
First, taking into account only the finishing order makes these rating systems universal because in almost any game a finishing order can be computed from the detailed game outcomes.
Second, it is crucial that the purpose of the game and the behavior of the rating system be aligned: people striving for high ratings should be forced to play in accordance with the spirit of the game. Taking the margin into account by which a game was won can be very misleading. As an example, consider a player who is trying to catch up with the opponent before the time is up, thereby taking into account the risk of falling further behind due to counter chances of the opponent. Should the rating system be designed to discourage this fighting spirit?
For the updates of the individual players’ scores in team games it is crucial to consider the finishing order of the teams only. If individual performance metrics are used in the rating system players will aim at maximizing those metrics instead of teaming up with their team members and try to win the game. For example, if you choose to reward the number of flags scored in a capture the flag game, do not be surprised if you find everyone rushing for the flag at once, but also some dead bodies who died with the flag in their hands betrayed by an overly ambitious team mate."
https://www.microsoft.com/en-us/research/wp-content/uploads/2006/10/Game-Developer-Feature-Article-Graepel-Herbrich.pdf
I'll also post an opinion piece by Sirlin (a blogger who is most known for balancing games) in regards to TrueSkill when Overwatch's matchmaking was a concern to him:
"TrueSkill intentionally and explicitly does NOT use any individual performance metrics. Their argument is that no matter what game you're talking about and no matter what metrics you measure to determine how well a given player did, it's necessarily imperfect compared to using only win/loss. The point of trying to guess if a player did well or not is how much they contributed win/loss, but the win/loss stat is the most accurate measure, they say. You'd introducing error by adding ANY other metric.
In addition to introducing error, you're warping incentives. For example, if you measure "damage done" as one metric, then it means players will attempt to maximize "winning AND damage done" rather than just "winning," which is not great. You can also very easily accidentally do a lot worse: you might accidentally give incentive not to play support heroes in a game where you really need support heroes on your team. (It seems this is already true in Overwatch.)
In many cases, it's almost hopeless to even devise a metric. If a character's role has to do with healing, you can't actually use how much they healed as a measure of much. If you did, it would penalize a healer on a team that played so well they didn't need as much healing. Or even worse is a character like Mei. Her ice walls can do a lot, her slow and freeze effects can do a lot too. But to actually quantify that into a metric correlated to win-rate? That's a huge error effect waiting to happen. My friend suggested the best metric how effective you were with her is to monitor the opponent's chat to detect how much they are cursing about Mei.
Yet another issue is that it's easy to accidentally create competition within a team for no real reason. For example, if number of kills is a metric that affects your rating, then your teammate killing an enemy that you could have killed essentially "stole" ranking points from you. That's clearly a bad dynamic.
I think Microsoft TrueSkill's reasoning makes sense here. It's a good case against ever using any individual performance metric when adjusting ranking points after a win or a loss."
https://www.sirlin.net/posts/tag/Overwatch
But above, I'm speaking of team-based victory games, which DBD isn't. It's important to note that in DBD, in which working as a team is only relevant until the point where each player can assure their own individual victories, players' need to understand where that 'point' of turning is in the game becomes part of their important skill set to have.
____
Like I said, MMRs will work for any pvp game, and that should include MMA in your example. Granted, I don't know anything about MMAs, but in principle, everything is the same. One of the MOST important aspect of understanding MMRs is that it's not just about one match - it's about series of matches, and where the players are flowing within it. Each individual matches don't matter, nor do they say anything about players' skill level. After all, as you said, there is always RNGs, or just some lucky punch. But when you monitor a sea of matches, and see the general direction in which the player is going, that actually gives a better picture of what is happening.
So let's take a look at your MMA fighter over a series of say, hundred different matches. As you say, they may have been just a skilled fighter in a "unlucky situation." But if "luck" is actually the only reason they lost, then out of hundred matches, how often do you think they would actually be losing? Probably very few. After all, their opponents (who, in your example, are less skilled) will *also* have to occasionally face that same "unlucky situation" and lose, in addition to losing due to their own lack of skill. If you translate this to MMRs, you'll see that your MMA fighter would generally be increasing in number, despite some dips here and there.
But let's take a look at another situation in which your MMA fighter *claims* that he's really skilled and only losing due to "unlucky situations", but their MMR seems to show that he's losing vast majority of his games against his opponents - say, losing 90% of the time. Well, what does that mean? There are few possibilities, and none of it has to do with "luck." 1. What they believe is "skill" - actually isn't. Let's say your MMA fighter trained hundreds of hours to perfect this one particular combo in which he jumps up like a pole vaulter, doing a triple axel in the air like an ice skater, and landing with a fury of punches from the air - he claims that he is skilled because A. it takes lot of training and effort to accomplish such feat, and B. because nobody else in the MMA can do it.
The problem is, it doesn't translate to them ever winning. So in the context specifically in regards to MMA, that isn't actually "a skill" - it might be helpful to them in some other sports or games, but not in MMA. Why? - because it doesn't help them win in any way. 2. The second possibility is that what they believe to be a skill *is* a skill (and would generally correlate with their ability to win) - but there are other factors that are continually diminishing that skill set - basically always outweighing it. For example, may be they worked too much on strength training, but neglected cardio, and therefore don't have the energy to last long enough in the fight. Or they may just be proned to make a bunch of small mistakes due to their anxiety problem, and once those mistakes are all accumulated together, make even their "skillful moves" meaningless in the end. Ultimately, in all of these cases, the problem lies with the player - not an issue with the MMR.
Hopefully this is clear enough for many of you to understand that "random, inconvenient circumstances" don't matter (if it truly is random) because MMR is always about multitudes of games, and the *general* directions that the players are moving towards.