2019 in Numbers - ITS data analysis

So what is the reconciliation between these numbers and what CB put up at CanCon? They seem to be two completely different sets of data?

 

I heard second hand that the CB numbers are for Satellites and Interplanetario, they may also be over a different time frame.

 

But surely that wouldnt give them numbers this high? I have to wonder if they threw in numbers from the campaign and leagues and one offs. It seems to starkly contrast experience and opinion. I can accept variation in the data sets to some degree because one is going to be more complete than the other, but frankly this is ludicrous. Several results are out right contradictory.

 

If I had to guess, the CB data shows stats for games, not tournaments. So if you brought army A to a 3-round tournament, that would show up as 3 games in CB's data, but only 1 tournament in this set.

 

On that note, @Ieldin Soecr, would you be willing to produce a chart that's just straight game-win-rate (total wins / total games played) per faction? That should match the CB slide more closely, and would be a useful way to compare the datasets.

 

Further comment on win rates - I don't actually know how to interpret this ratio in a meaningful way. It's basically "strength" over "popularity", which means that a weak but unpopular faction and a strong but popular faction would both approach 1.0 in the limit?

 

Also the hidden data of "when an unpopular faction does well, it means skilled players gravitate to it / when a popular one does mediocre or poorly, it means new or poor players gravitate to it" and the influence of forum/FB "wisdom" on that, as well.

 

Winrate is not a useful metric in Infinity with Ties, Minors and Majors granting different points. A Faction that scores a lot of Ties and Majors is going to Outperform a Faction that has more Wins, but most of them Minors or less Ties and more losses.
These are the proper stats to look at and there aren't many surprises there:

Factions sorted into a tierlist by TP Performance (with old JSA excluded for obvious reasons):

1. 0.54 Dashat
2. 0.52 Shasvastii
3. 0.51 Tohaa
0.51 Spiral Corps
0.51 OSS
0.51 Aleph Vanilla
0.51 Varuna
8. 0.48 Combined Army Vanilla
0.48 Caledonia
0.48 SAA
11. 0.46 Ikari
12. 0.45 Foreign Company
0.45 TAK
0.45 FRRM
15. 0.44 Steel Phalanx
0.44 Ramah Taskforce
0.44 Hassassin Bahram
0.44 IA
19. 0.43 Nomads Vanilla
0.43 Ariadna Vanilla
0.43 Neoterra
22. 0.42 Starco
0.42 Bakunin
24. 0.41 MO
25. 0.4 JSA
0.4 Onyx
0.4 Tunguska
0.4 Haqq Vanilla
29. 0.39 ISS
0.39 USARF
31. 0.38 Corregidor
32. 0.37 Morats
33. 0.36 PanO Vanilla
34. 0.35 QK
0.35 Druze

That's the spread. Take it with a grain of salt (check the pickrate lists why). Still some of these armies are flat out performing better than others.
Mind you that the difference in performance in between 0.35 and 0.54 is not roughly 50% as the number suggests. Scoring low on average also means you're very very unlikely to win a tournament, you need those majors to have a realistic chance at getting a trophy.
@Ieldin Soecr could probably pull tournament win relative to games played percentages. They'll show that the top 7 have won significantly higher amounts of tournaments relative to games played compared to the bottom 7.
QK and Druze might not have won a tournament at all given their relative lower games played and last years numbers.

Tournament wins are a much lower sample size but an interesting metric to determine the Skill involved in playing a Faction up to its potential.
A faction that has a high winrate with low avg TP and avg tournament victories suggests a low Skill ceiling or a flat out bad Sectorial lacking tools (Corregidor 51,64% WR but low avg TP, MO 49,37% WR with bottom 3 avg TP).
Low winrate with high avg TP and tournament victories suggests a Faction is difficult to play but a high Skill ceiling if you can use it correctly (Dashat with 44.42% WR but highest avg TP, Spiral with 45.45% WR but top 3 avg TP).
I would not recommend to trust those Winrates at all tbh, something seems off apart from Major/Minor/Tie being very important.

A few things to point out is stuff like Vanilla PanO or ISS. Performs terrible, yes, but looking at what people run suggests that's partly to blame on new players running with their Faction box and enough stuff to make it 10 Orders... that just doesn't turn out very well by default.

Overall this more or less matches last year's number, shows the old problems and what N4 needs to address.

 

In the video, @Bostria said their results were for the last few months, so it might not even cover the whole ITS season so far. Which is going to lead to major discrepancies when compared to the whole of 2019.

 

@Ieldin Soecr this is amazing data, I love being able to dig into this!

I'm curious, do you have a faction breakdown by ITS ranking for last season? I'm curious if there are any differences when you look at people that place within the top 10-25% of events and people that place below them? This would be a really interesting break-out, as it could isolate the statistics of newer players who are just breaking into the meta vs players that have experience.

 

@Ieldin Soecr
How come in the excel sheet you linked most recently the win rate % arrange the factions in a different order to those given in your original graphic for win:loss ratio in your original post. I don't understand how win rate % can give different rankings to win:loss ratio?

 

the % winrate is a valid way to look at single games, but not for tournaments. Winning 1-0 counts the same there as wining 10-0, so for tournament has to be used combined with other data. A faction with a high winrate but low TP % would mean that while it can win, it will usually do it by low, for example

in an ideal-balanced game state, the won and losts games would be near equal and only based on the gamer hability. Top (and botom) players would choose based on aesthetics, gamestyle or lore, as any other player, and we could find top players in any faction (because the reasons for choosing would be unrelated to game balance), so going near the 50% would mean a good balance.

A similar metric would be comparing tournament winning % for each faction (not games, but tournaments), and then compared with the % of use (%winning - %use) would show if a faction is overperforming/underperforming at winning tournaments (in the ideal-balanced game state from before, that subtraction would be very near to 0 because more people going with X faction, would make X faction to get more chances to wining, because there would be a higher % of good players)

 

Skill floor and skill ceiling are a thing. Win rates will vary depending on whether the game is balanced like CB seem to do it (aiming for roughly equal rates regardless of stratum) or if it is balanced as Blizzard does with Starcraft 2 (aiming for fairly fine-tuned balance at highest level and letting the ELO algorithm handle lower skill level balancing).

A faction with a high floor (easy to use) and a low ceiling (little gained from mastering) may stomp the overall statistics, but may be experienced as being uninteresting or bad as a player or meta gathers gameplay experience. The same goes for the skill floor/ceiling of dealing with a faction as its opponent. For example, I have a feeling that Dahshat is a bit like that, and that win rates with them will go down as players learn to deal with McMurder (I think a lot of people play Impetuous wrong with him) and linked Rui Shi.

What I'm saying is, there are underlying factors that will naturally fudge with statistics.

 

Random variation between the two samples would account for it. Neither this nor CB's numbers really give reason to conclude any factions are over/underpowered, and the two datasets yielding totally different results supports the conclusion that there's nothing in the data indicating imbalance.

 

The last data set (the faction breakdown posted yesterday) that looks at only tournaments paints a picture that is a less favorable than the other data sets.

When you look at tournaments only, the spread is between 57.18% (for Dashat) and 35.42% (for Druze). This is much closer to the spread I had expected to see, and does highlight some factions that fall outside the norm. You can see that Druze, QK, Morat and Vanilla PanO all fall under a 40% win-rate, and Shas, Aleph, Tohaa, Varuna and Dashat are all >54%. That's a huge delta if you think about it.

The other thing that I'm still really interested in seeing is how player experience affects these results. I'd love to see the top quartile

 

I'm saying that I genuinely don't know how to derive information from the ratio of win rate to pick rate. Is higher or lower better? What does 1.0 mean, vs. 0.5 or 1.5? I don't understand what it's trying to measure.

Edit: I was also looking at the wrong chart; I see that in this one they're all centered around 0.5, and no entries are over 1.0.

 

for that, more than the top quartile, you need to look at all the data, but split in parts, not only at the top

 

Trying not to sound condescending. If you want, ask for wich chart do you need an explanation and I will try to explain as good as I can (if I know). A bit of general explanation (any math teacher here is free to correct me, I am only a programmer): The X% is an aproximation that from every 100 cases in, X accomplishes the expected, for example, a 52% wins means that faction, for each 100 games, has won 52 wich is avobe average (the average should be near 50%, but a bit below because the ties). Sometimes that % is used as a value from 0 to 1, when that happens, it is just the same as before but without multiplying by 100. A 0,45 is equivalent to 45%. If it is good or bad, that depends on each chart, because each one has their own meaning.

 

@Mahtamori What do you think people are doing wrong with McMurrough's impetuous trait?