FYI: Stats - 'Red Ranks' weren't REALLY Red Rank Matches
Or at least, they're not what some of us believed they represented. What I personally had hoped for was a graph that demonstrated the skill ceiling on both sides - where the most skilled players are going up against other most skilled players, so that the point of balance can be more easily identified. However, despite being labeled as "Kill Rate VS. Pick Rate (Red Ranks)", the samples of matches collected for the data for November 2019 Statistics apparently don't actually demonstrate Red Rank VS Red Rank matches.
If you look through my recent posts, you may observe that most of my posts lately have been about requesting information from @Peanits and @MandyTalk about how they had defined 'Red Rank Matches' for their data collection. I've actually never received an answer from @Peanits , despite tagging him around five different times in different Statistics-related threads (which, for some of them, he has also posted in.) @MandyTalk did respond after several attempts at tagging her, at which point she stated:
"With regard to the stats, I believe it;s the start of the match so all players would be red ranked at that point."
Thinking that settled my inquiry, I was getting ready to inform the Japanese forum about what I had learned from MandyTalk. But then another Japanese forum member tagged me to inform me that the Japanese Community Manager had also answered the same question, and this is what he said:
"今回の統計でのレッドランクとは、"殺人鬼がマッチ開始時にランク1～4だった"マッチングです。マッチした生存者のランクやマッチ後の昇降格は考慮していません。"
For those of you who don't speak Japanese, this is what it basically translates to:
"For this particular statistical data, Red Rank is defined as 'matches that consisted of Rank 1-4 Killers at the beginning of the match.' The ranking of survivors or the rank results after the match were not given consideration."
If the Japanese community manager is to be believed, that means that these 'Red Rank' matches aren't necessarily the result of multitudes of 'Red Rank Killer vs 4 Red Rank Survivors' matches as I had hoped they represented. It could just as well be 'Red Rank Killer vs Red Rank, Purple Rank, Green Rank, and a Brown Rank' types of matches. Heck, there doesn't necessarily even have to be a Red Rank among the survivors. Ultimately, if there are no available Red Ranked survivors when the Matchmaking Algorithm are grouping players together, Red Rank Killers will be facing lower (Purple to Grey) ranked survivors. I won't even bother mentioning the differently ranked SWF problem we have since I'm sure everyone already understands.
Now, of course, even if it were truly 'Red Rank vs 4 Red Rank' data, it doesn't necessarily mean that the graph would change. After all, we don't actually have the data for that. For example, even if the criteria is simply 'Red Rank Killers vs Anyone', if 95% of the matches just happens to be against 4 other Red Ranked Survivors, then the graph would pretty much stay the same. However, if majority of the samples actually do NOT consist of four Red Rank Survivors per match, then the graph that were shown to us, in all likelihood, would deviate much from the graph for the real Red Rank vs Red Ranks matches. It may also explain why so many Red Rank Killers on the forum seem to express having different experiences when facing 4 Red Ranked Survivors. It may explain why the kill rate appears to overlap so much with the 'All Ranks' graph.
So in order to bring back confidence in the graph shown to us, several things need to happen.
- 1. Provide the percentage of 'All 5 Red Rank players' matches within the total number of matches that were sampled overall.
- 2. Provide the Survival Rates of 4-Red Rank Survivors VS any-ranked killer they faced (preferably by killer type).
- 3. For any future statistical information, please give us more detail on HOW you came up with the numbers. Instead of having to repeat that DCed matches were not included in each individual posts of different forum members, provide that information in the actual Stat post. Instead of forcing people to guess at the meaning of 'Red Rank' matches, provide that information in the actual Stat post. Just be honest and transparent about the data, so that you don't end up appearing as if you are pushing a narrative that you want people to believe.
I'll leave a link to the Japanese thread that contains the relevant information:
https://forum.deadbydaylight.com/ja/discussion/106222/開発に公開してほしい統計を書き込むトピック/p2
Comments
If that's true the statistics tell us even less about everything, because of the recent change to swf matchmaking.
The issue isn't whether the developers decide to publicize the statistical data or not.
The issue is they are willing to use the same set of data to make balancing decisions. There have been several instances in the past in which some developers on the forum would refer back to a data that were presented to us to justify their current decisions or opinions. The issue is occasionally they decide to compile data in ways that don't show accurate information for what they are seeking to find, or interpret some results wrongly to justify their own biased experiences.
Look at the fact that for the longest time, they continued to use disconnected matches as part of the data samples - continually skewing the results for kill/survival rates. The only reason they now understand more about getting accurate statistical result is because this community has repeatedly pointed it out for them. Even then, they had continually pushed back, stating ideas like, "more data samples are always better, regardless of whether they should be considered outliers or not," or "it really doesn't make that much of a difference anyway." Fortunately, after many years, they've finally listened. They've made progress.
The fact of the matter is, they are better off sharing with the Community what they are doing, how they are doing them, and why they have done them in that way. That way, the Community can offer their own expertise and criticism for the developers.
Of course, whether they choose to learn from the criticism they are offered to do better in the future, or attempt to legitimize their past decisions and continue to make the same mistakes, that's all on them. After all, human pride is one of the worst traits that people can have.
Basically, the stats released tell us nothing about the game or it’s balance or how good the players are or really, anything at all.
They are basically just numbers that exist.
Also, there were quite a few other issues with that data they had to add.
The kill rate isn't good for determining balance as it is including a multitude of factors like NOED, survivors trolling after exit gates are open, hook swarming after exit gates are open because they've already won and don't care, suicides on hook, were they just camped to death and died on first hook, among many others including the rank points you introduced.
The reason I include all these is because even though they resulted in kills, those kills are not indicative of "balance". Yes they mention how they don't solely use the kill stats for balancing, but since they're going for a 2 kill balance that seems "slightly" misleading. Also, presenting them to the public like this with all those flaws is going to cause these "misconceptions" even further.
Not to mention as we saw in previous graphs, the stats of red rank games is very different from rank 1 games so that should be separated for more accuracy.
All in all with so many issues affecting these "kill rate" stats, we really should be shown "pip rates" for these games as well because even though it has its own flaws, it would be much more accurate for showing killer strength than kills are as pip rates would knock out a little of the outlying issues I mentioned above. IE filtering out a 4k from NOED that's a black pip, especially since the pipping system is their actually ranking system not the kills for determining how well they played.
We still have all the killers that haven't had their pipping issues fixed like Plague and killers with 1 shots, but still a better system for determining balance.
@knell Thank you for providing this information and clearing up a few questions I have had myself.
If this true, that it may have just been Red Rank Killers the data is based on, would mean the data itself is worse than meaningless. It is data providing a false reality and misrepresenting actual facts.
I am happy BHVR released data, but I am not happy with how they did it. There are so many questions about specifics, about how this data was collected and what it actually represents.
Thanks again and keep up the good work!
Even if it was only red rank survivors, that still doesn't tell us much because most red rank survivors aren't even good players.
Swf matchmaking probably did slightly inflate kill rates, but otherwise I don't see a problem here.
This really is something if what that Japanese CM said is true. Why would they even give us stats knowing full well that it's misleading as fudgesickles.
Bumping because this is really important.
This is a massive issue.
Well, hot damn, looks like @Detective_Jonathan just found his newest partner in crime solving for the DbB community.
Yeah, that's my other big question. It is super easy to rank up as Survivor and there are a lot of boosted players in red ranks
You can't balance anything based on this
Since this thread was brought back up, I'd like to take the opportunity to quickly correct something I have stated in my original post: "Ultimately, if there are no available Red Ranked survivors when the Matchmaking Algorithm are grouping players together, Red Rank Killers will be facing lower (Purple to Grey) ranked survivors."
Given that the new Matchmaking Algorithm was presented to us on October 2nd (as part of the September Developer Update) and therefore went into effect around that time, and our recent Statistical Data were collected from around August 20th to November 20th, this means that the implementation of the 6-rank limit should have affected the data at least part of that time period. So if the new matchmaking was working as intended during those times, it would have been impossible for those (solo-only) matches to have contained survivors who were in the Brown or Grey Ranks as I had originally claimed. That stated, even with the new matchmaking algorithm, it would have been possible for Rank 3 or 4 Killers to end up facing an entire solo's crew of Rank 9 or 10 survivors, so my overall point I believe still stands. (In addition, if SWF matches were also included as part of the data, it would be also possible for the Red Rank killers to occasionally face some Brown or Grey Ranked survivors.)