1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

2019 in Numbers - ITS data analysis

Discussion in 'Access Guide to the Human Sphere' started by Ieldin Soecr, Jan 23, 2020.

Tags:
  1. Hecaton

    Hecaton EI Anger Translator

    Joined:
    Nov 23, 2017
    Messages:
    7,205
    Likes Received:
    6,535
  2. inane.imp

    inane.imp Well-Known Member

    Joined:
    Jan 28, 2018
    Messages:
    6,040
    Likes Received:
    7,177
    +1 to the cheers of acclamation: this is all very interesting and seriously cool.

    But... the question that I keep coming back to is "how much of a faction's performance is determined by the average skill level of its players?"

    I think PanO shows this: while the poor performance of vanilla can in part be explained by the importance of Fireteam to a faction reliant on FTF Firepower, the difference seems much larger so I think you're seeing good PanO players avoid vanilla.

    Would it be possible to normalise the performance based on a player's ELO rating? This would account for the fact that a player with low ELO are expected to lose more than they win and this isn't necessarily a reflection of their faction choice.

    Alternatively, show the average ELO of the players who played a tournament with that faction. This would allow broad assessments like "part of the reason Tohaa performs so well is because it's a strong faction but also because it has a disproportionately experienced player base".

    -----

    In surprising revelations, I'm surprised by the relatively low pick rate of the Kriza.

    Which leads me to my second question: I'm interested in how picking a particular unit/profile correlates with winning. Without the data that says "this list was taken in this match" you can't get that perfectly, but you can proxy it fairly well with by taking the overall tournie pair as a whole.
     
    bladerunner_35, Lesh' and DukeofEarl like this.
  3. Hecaton

    Hecaton EI Anger Translator

    Joined:
    Nov 23, 2017
    Messages:
    7,205
    Likes Received:
    6,535
    That's practically very difficult because if a player primarily plays a more powerful faction, their ELO will be higher.

    In any case, not surprised by the performance of Dashat in this list. It's clearly working as intended.
     
    meikyoushisui likes this.
  4. meikyoushisui

    meikyoushisui Competitor for Most Ignored User

    Joined:
    Nov 28, 2017
    Messages:
    1,803
    Likes Received:
    2,804
    Great analysis.

    I don't think anyone should really find any of the top end that surprising at this point. People have been talking about how incredibly powerful Dashat is since it was released, I'm not surprised to see that in the evidence.

    In the below 0.5 area, you have ASA, VIRD, O12, CA, Shas, Aleph, OSS, Tohaa, Spiral and Dashat, with Dashat, Aleph and VIRD below .45.

    It's also good to see numerically shown that JSA is struggling recently, at a solid 1.0 in YJ (which I'm sure is just an error in reporting, is there still a placeholder for JSA under YJ?) and .59 on the main list, so towards the bottom third of the game.

    The biggest surprise here is ISS's performance, that's a really full-on case of how the mighty have fallen. ISS was the strongest sectorial in the Pre-Druze game, but it's clear that none of the releases since have really favored what ISS does. As someone who sold as much of his ISS as he could before moving halfway around the world, I would still be happy to see it get an update, though at this point I think the more likely outcome is it gets mrrf'd for a new sectorial in the new battle pack.
     
    Belgrim, Hecaton and Abrilete like this.
  5. Mahtamori

    Mahtamori Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    12,018
    Likes Received:
    15,302
    Even just adding one more axis (player interest "does the faction spark joy?") and you'll get a more complicated puzzle. Varuna and Dahshat both have fairly similar win rates and if players migrated to a faction based on its performance, we would see at least 200 Dahshat players, but Dahshat is under 100. It could be that Dahshat as a faction simply doesn't attract people (is it too far fetched to say that it is an "ugly" and/or boring faction that turns people off it?). But Dahshat is an NA2, naturally they are smaller? No, look at JSA - larger than any vanilla faction and most sectorials.
    JSA on the other hand has a very low win rate and here we could introduce yet another axis of evaluation - difficulty. JSA is notoriously a difficult faction and can we really say that a faction that attracts a lot of players and is difficult to play really is underperforming? I don't think so, it might simply be that it's a bit of a newbie trap and that might be bringing the scores down.

    So.
    These aggregates shows the result of at least three different and major influences on player performance and distribution.

    I do note that while vanilla Pan-O is low in participation, at the same time that's true for most vanilla factions. Even Ariadna which is generally held as one of the strongest vanilla factions has a similar level of participation. Maybe, just maybe, fireteams spark joy in people and reduces the difficulty stress and that's why people are migrating towards sectorials?
     
    wes-o-matic, inane.imp and Sedral like this.
  6. daszul

    daszul Well-Known Member

    Joined:
    Oct 11, 2018
    Messages:
    732
    Likes Received:
    876
    Interesting Data.

    Would be cool to see the win/loose ratio per profile as well!

    And to include all available profiles as well,
    to show which profiles are (seen as) "bad".

    E.g. Chaksa Longarms were never used in Tohaa,
    but twice (as Haris) in Spiral.
    But did that choice work out?
     
    Abrilete likes this.
  7. Scoria

    Scoria Member

    Joined:
    Oct 18, 2018
    Messages:
    21
    Likes Received:
    18
    Looking at the Faction Profile Pickrate sheet, it looks like one guy turned in an old JSA list (either it's one they've had knocking around for a while or they saved an old version of Offline Army) and won with it.
     
  8. Armihaul

    Armihaul Well-Known Member

    Joined:
    Nov 24, 2017
    Messages:
    1,470
    Likes Received:
    1,112
    more than fixing by ELO, what could be taken out is a % of the top and bottom players for each faction (maybe 5% or 10%) using that ELO,and redo the metrics with that. This way, can be taken out the extremes.

    Another way would be to divide the player base in tiers (top 33%, bottom 33% and mid 34%) and redo the metrics for each tier. That way we could see the perfomance based on the hability and maybe see which factions are easier to use
     
    bladerunner_35 likes this.
  9. HellLois

    HellLois What the Hell...Lois?
    CB Staff

    Joined:
    Apr 25, 2017
    Messages:
    1,537
    Likes Received:
    4,137
    @Ieldin Soecr Amazing work! Its great to see things like this. is very similar to the data we get.
     
    #29 HellLois, Jan 24, 2020
    Last edited: Jan 24, 2020
  10. Ieldin Soecr

    Ieldin Soecr Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    129
    Likes Received:
    329
    Thanks for all the positive feedback:grinning: makes all the effort worth it.


    I have look it up and I have 22 tournaments in France in 2019. One of them wasn't reported correctly (https://otm.corvusbelli.com/private/event/info/00e7315c8, first round M0rk4 has 11 OP) and 2 have not been reported yet (https://otm.corvusbelli.com/private/event/info/13634a080 and https://otm.corvusbelli.com/private/event/info/7a80f828b). The last one seemed to have not been picked up (https://its.infinitythegame.com/event/bc6c52471-guild-con-3).

    Jep, their lower player count (Half of CHA) mitigates their weaker overall performance in this metric.

    [​IMG]

    Number of minor and mayor wins to losses per faction. Draws are not included.

    [Profile Pickrate]

    I added a additional column to the profile list. it contains the W/L ration of the profile compared to the ration of the faction overall. Values >1 show that a profile has a higher W/L ratio than the faction, <1 the ratio is lower.
    As mentioned above this statistic is not very reliable for two reasons:
    1. We do not know what list a player used each round, so it only can be an aproximation.
    2. A lot of profiles haf very low pick rates, so that the error margin is huge.

    Also, with all the faction data, keep in mind that only around 50% of the player base submitted lists to OTM, so it is not representative to the whole playerbase.


    ELO as dimension to track player skill has multiple problems. First it is resetted after each ITS season, what makes the tracking over one year problematic. Secondly is the ELO hard to integrate as it changes after each match within the CB system, so what ELO should be taken, the one before the match ot the one at a specific point in time. I will look if I can come up with something. Maybe I will aggregate the player performance per person and reflect that back onto the factions played.
     
  11. Mahtamori

    Mahtamori Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    12,018
    Likes Received:
    15,302
    JSectA in the Yu Jing group finished dead last. Likely they didn't win a single game and scored very few OP.
     
    meikyoushisui likes this.
  12. Scoria

    Scoria Member

    Joined:
    Oct 18, 2018
    Messages:
    21
    Likes Received:
    18
    Are you sure you're looking at the right graph? The only one they're dead last in is the one that measures number of participants, because there's only one guy playing. They're top in the Fraction of Possible TP Earned and Average Ranking graphs, and don't appear in the Wins/Usage one.
     
  13. Mahtamori

    Mahtamori Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    12,018
    Likes Received:
    15,302
    This one is much more fun to look at! I perused it by deselecting all <1% entries and while point 1 is problematic (it is fully possible given that a single tournament amounts to as much as 1% of a faction's unit selections to artificially pump some factions results by making utter "meme" list A and a serious list B that is used for all missions). Generally speaking what this statistic seem good at is ferret out when thinking outside the box was often rewarded or when it was often punished;

    It's not surprising that Daoying is increasing win rate for Invincible Army, but the non-LT is scoring higher than the LT and Cube Jäger being the second most win rate intensive unit does raise eyebrows! One thing seems certain, though, we should actively avoid the Daoying snipers.
    For Dahshat I think everyone was expecting Hunzakut to score high here, but with 4% participation and over 2 W/L the Zuyong Combi FO comes in from nowhere.
    O-12 with 3 lists and presumably 9 wins, Omega non-LT, non-FO, Multi Rifle does make you pause to think and Wild Bill as well as Razor Minlayer/AHD showing good promise.
    For Imperial Service it seems like Celestial Guard Control Device is mandatory, but it is the Celestial Guard Spitfire that seems to have rewarded people the most for thinking outside the box! That said, Celestial Guard KSCD is picked in 93% of all matches, but still manages to pull a 1.02 W/L while the faction itself scores lower than 1, so it's almost like every single one of those remaining 7% tournaments without a KSCD, ISS scored some heavy losses.
    Dakini deserves a special mention. In all factions where it is present it is very commonly taken and score highly as long as it is the HMG or more preferably the Multi Sniper. Except for in OSS where it doesn't seem to increase win-loss to near the same degree, even though they can link.
    OSS instead seems like they should be focusing on Apsara KHD, various Dasyus, and a Garuda with Combi. This isn't to say "don't take Dakini", but rather that "unlike in ISS or Aleph, for OSS Dakini won't be what turns your loss into a win"
    EVO Hackers is also something I'm noticing a lot across many factions. It seems like a unit that you don't bring accidentally or by habit and when you bring it you bring it with a plan and that over 60% of the time that plan works every time.

    Important: Any unit or profile that's picked in all lists will share all losses as well as all wins - regardless if they were consistently MVP or not. Expecting Kamau Sniper to score extremely well for Varuna? 85% of all Varuna tournament lists had one, so a very large amount of the losses as well as wins would be ticked for the Kamau sniper, but it does seem like tournaments where Kamau weren't there only for the heavy weapons didn't go so well...
     
  14. Mahtamori

    Mahtamori Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    12,018
    Likes Received:
    15,302
    It's an interesting pollution of the numbers, sure, but I can't find this "average ranking". It's Top % I am looking at and that one they score dead last in because lower is better. The TP by rounds doesn't mesh with the Top % (it meshes with other high performing factions - Dahshat, Aleph, OSS, Varuna, Combined and Shasvasti all scored higher on PossibleTP and had a lower Top% value, which is consistent with each other). What this suggests to me is that the JSectA pollution is different tournaments. It shouldn't be possible to score 70% of TPs possible in a tournament and still finish last.
     
  15. Ieldin Soecr

    Ieldin Soecr Well-Known Member

    Joined:
    Nov 23, 2017
    Messages:
    129
    Likes Received:
    329
    That match was a Leage game (Only one Round) where the player got the Bye. So he has 2 TP but no win (Bye is counted seperatly)
     
    Mahtamori likes this.
  16. HeadChime

    HeadChime Well-Known Member
    Warcor

    Joined:
    Nov 23, 2017
    Messages:
    111
    Likes Received:
    351
    I took the data provided, and just sorted it into columns here.
    I also calculated means and standard deviations, and marked them on the data.
     
  17. Zewrath

    Zewrath Elitist Jerk

    Joined:
    Nov 28, 2017
    Messages:
    2,000
    Likes Received:
    3,484
    Meh.. I wouldn't read too much in to it. There's two main factors as I see it:

    1. ISS re-work was new with HSN3, which was ages ago. Now everyone else got major releases and toys to play with, then people simply gravitate towards that.

    2. Lack of representation and skewed data. Every time I go to any major event, I rarely, if ever, see anyone play ISS. Any when I actually see anyone play ISS, they are (from my experience, based on traveling across Europe), terrible players. Like Z-tier players, so I don't think it's the fault of the faction in this case. My best guess is that competitive players seeks out optimized armies/profiles so OSS/VIRD/TAK have a much greater draw power on higher tier players, so the "bad" players are left to play ISS or what ever faction they end up choosing.
    Who knows, this is just a summary of what I've observed and I could be entirely wrong.
     
    Savnock, Lesh', Sedral and 2 others like this.
  18. Diphoration

    Diphoration Well-Known Member
    Warcor

    Joined:
    Aug 9, 2018
    Messages:
    1,349
    Likes Received:
    2,533
    Woo! Thanks @Ieldin Soecr for the valuable information.

    Some of the numbers are unsurprising and very close to our local consensus, however, some numbers are extremely surprising to me.

    Some of the unit pick rates and win-rate on some factions are very odd in my opinion.

    I think one of the thing skewing the data is the availability of some units in the starter pack, the newer factions usually have their strong faction-defining profiles in the starter boxes, while some other factions have them available in extra purchases (and sometimes needs to be purchased multiple times).

    I do not know how hard/long it is for you to poll more information, but I would be very interested to see the win% of the top quartile for each faction and the unit pick rate at high competitive level.

    Thanks once again for your work! <3
     
    meikyoushisui likes this.
  19. Diphoration

    Diphoration Well-Known Member
    Warcor

    Joined:
    Aug 9, 2018
    Messages:
    1,349
    Likes Received:
    2,533
    A new player that picks up Operation Coldfront + Beyond will have most of what makes OSS good.

    A new player that picks up Wildfire + Beyond Wildfire will have a good amount of what makes Sha good.

    A new player that picks up ISS starter or any of the other YJ starter is still missing everything that makes ISS strong. They need to go and get multiple purchases to get the core of the faction (2 boxes of Kuang Shi) and then some more purchases to get the good guns (Su Jian, Rui Shi), and then some more purchases to get the good specialists (Ninja, Xi).

    If you were to give a limited $$$ budget or purchase limites to a competitive player and ask them to make a list to go an event, they would never pick ISS, so it’s not hard to imagine that this would skew it in favour of the new sectorial which have much more focussed purchases.

    I feel like the average ISS player will probably be bringing a suboptimal list more often than a OSS player simply because the average ISS player did not yet buy all of their faction defining profiles.

    I think CB has done a good job at making new boxes contain the juicy stuff that makes faction tick. A repackaging for the old faction starters would be really neat.
     
    #39 Diphoration, Jan 24, 2020
    Last edited: Jan 24, 2020
  20. Hecaton

    Hecaton EI Anger Translator

    Joined:
    Nov 23, 2017
    Messages:
    7,205
    Likes Received:
    6,535
    Aight. Really looking forward to the rebalancing come N4!
     
    Freki likes this.
  • About Us

    We are a company founded in 2001 in Cangas (Spain), and devoted to design and manufacture games and figures. Our main product, Infinity the Game, was born with the ambition to satisfy the most demanding audience, offering the best quality.

     

    Why are we here?

     

    Because we are, first and foremost, players.

  • Quick Navigation

    Open the Quick Navigation