+ Create Blog
View RSS Feed

admin staff

11-16-2017 at 03:59 AM
30 Comments
Rate this Entry

Negative ESI Explanation

Hi All

We noticed that few variations have negative ESI . This is due to the "In Process" submissions . The ESI calculations are based on ranks and we were excluding the "In Process " submissions while calculating ESI which resulted in negative values.

Now if a user submits a score and it is " In Process" then the ESI will be calculated and added to his overall ESI.
Once the score is accepted and his/her previous score is removed then the respective old ESI value will be deducted from the Final ESI.

So in essence we will award additional ESI when a new score is submitted.

Let us know if you have any questions or Concerns

Regards
Admin Staff
Thanks bensweeneyonbass thanked this post

Comments

Page 1 of 2 1 2 LastLast
  1. MyOwnWorstEnemy's Avatar
    Ok - If I'm understanding this correctly, if a member submits a better time/score on a track they already have an accepted record on, they will temporarily be awarded additional ESI on the 'in process' submission. Once the 'in process submission is approved, the older record is removed and the ESI is adjusted downward.

    An example:

    1) Player ESI is 10 (one approved submission)
    2) New Submission (in process) is worth 11 ESI
    3) Player ESI is 21 (10 + 11) during the duration that the new submission remains 'in process'
    4) If New Submission is Accepted, Player ESI is 11 (21-10) or
    5) If New Submission is Rejected, Player ESI is 10 (21-11)

    Is this correct?
  2. admin staff's Avatar
    Yes This is correct. This new rule will apply to submissions being submitted and accepted starting 1
    1-16-2017 at 04:59 AM

    Regards
    Admin Staff

    Quote Originally Posted by MyOwnWorstEnemy
    Ok - If I'm understanding this correctly, if a member submits a better time/score on a track they already have an accepted record on, they will temporarily be awarded additional ESI on the 'in process' submission. Once the 'in process submission is approved, the older record is removed and the ESI is adjusted downward.

    An example:

    1) Player ESI is 10 (one approved submission)
    2) New Submission (in process) is worth 11 ESI
    3) Player ESI is 21 (10 + 11) during the duration that the new submission remains 'in process'
    4) If New Submission is Accepted, Player ESI is 11 (21-10) or
    5) If New Submission is Rejected, Player ESI is 10 (21-11)

    Is this correct?
  3. JasonV91's Avatar
    For further clarification - does this mean that whenever a person submits a score, they immediately get awarded ESI (before the score is adjudicated), whether they already have a previous score on that track or not?
  4. admin staff's Avatar
    That is correct. The ESI calculation is dependent on Ranks . Since we are ranking the submissions before they are adjudicated, we have to recalculate the ESI too.

    In previous cases we were not calculating ESI for the "inprocess" submissions which led to negative ESI's for the already adjudicated submissions.

    Regards
    Admin Staff

    Quote Originally Posted by JasonV91
    For further clarification - does this mean that whenever a person submits a score, they immediately get awarded ESI (before the score is adjudicated), whether they already have a previous score on that track or not?
  5. JasonV91's Avatar
    Thank you for the clarification.

    That being said, I'll go on record and say I disagree with awarding ESI and/or including an "in process" submission to a gamer's record count (especially WRs). Basically, this approach starts with the assumption that the submission will be accepted, and rewards the submitter (along with penalizing his competition) before TGSAP has run its course. In theory, I could submit to 1000 tracks and falsely inflate my ESI and WR count, while also "stealing" those WRs from the true holders until my submissions are all rejected. While this is obviously an extreme example, look at the more realistic case of the dozens and dozens of old submissions sitting in the queue...most adjudicators will only vote No if the choice is obvious (due to the 5% penalty risk), so any questionable (but not outright bad) submission stays in the queue forever, never to be voted on - gamers who have submitted those will never have their ESI reduced back to its correct value, and any WRs that had been "taken" from another gamer will never be reinstated.

    I'd be curious to hear what others think about this.
  6. admin staff's Avatar
    Hi Guys

    We are really open to discussion on this issue. The technical problem here is that ESI has to be calculated taking rank as an input.

    The ="inprocess" scores are ranked before they are accepted . So we have to recalculate the ESI for that record at that point else the ESI calculation for other users goes negative .


    Regards
    Admin Staff
  7. Snowflake's Avatar
    my previously submitted scores arent adding to my esi since they were submitted before this. does this mean if they're rejected i'll lose points even though i was never awarded them in the first place? also if its accepted will i never get them since it assumes i already have them?
  8. Snowflake's Avatar
    also, while we're taling about esi, i noticed something for both atari 2600 and atari 400/800, dont know if its true for all platforms didnt check but its true for those.

    when you click on a particular game, it doesnt just show the ESI for that game, it now also shows the esi for the platform as a whole -- awesome, love the edition. Of course, you can also see platform level ESI by click on the platform itself. problem is, the two different ways to platform level ESI give different numbers.
  9. MyOwnWorstEnemy's Avatar
    Thank you for the response and clarification.

    It seems there are two options unless an alternative solution is found.

    Option #1 - Submissions pending final adjudication outcome are shown on the leaderboard and inflate ESI calculation, and 'true' ESI is calculated once the adjudication is final.

    Option #2 - Submissions pending final adjudication are not included on the leaderboard and are only added to the leaderboard and contribute to ESI calculation once adjudication is final.

    Based on the two options, I'd rather go back to the way it was before (option #2). I view ESI as the premier statistical measurement to compare player skills across various platforms and games, demonstrated by player achievement. A fluctuating and inflated ESI will lead to confusion, and not worth the benefit of viewing 'in process' submissions on the current leaderboard.

    Thanks for being open to community feedback.
  10. Max's Avatar
    Quote Originally Posted by MyOwnWorstEnemy
    I view ESI as the premier statistical measurement to compare player skills across various platforms and games, demonstrated by player achievement.

    A fluctuating and inflated ESI will lead to confusion, and not worth the benefit of viewing 'in process' submissions on the current leaderboard.
    Shaun please.

    "The ESI score concept is a powerful, logical, holistic statistical measurement system that can analyze total achievement across all video games, platforms and categories."

    I am not understanding the desire to over complicate leaderboards. My preference would be simplicity for everyone:

    Post and compute valid scores once validated. Remove and compute invalid scores once invalidated.
  11. MyOwnWorstEnemy's Avatar
    Quote Originally Posted by Max
    Shaun please.

    "The ESI score concept is a powerful, logical, holistic statistical measurement system that can analyze total achievement across all video games, platforms and categories."

    I am not understanding the desire to over complicate leaderboards. My preference would be simplicity for everyone:

    Post and compute valid scores once validated. Remove and compute invalid scores once invalidated.
    Yes - manipulating ESI before full adjudication will result in premature calculation. This condition impairs the ESI score concept of reaching its full potential of being that powerful, logical, and holistic statistical measurement system. Premature calculation causes ESI inflation and therefore is a unreliable statistical measurement unless you factor in a margin of expected deflation. Premature calculation is confusing and awkward to explain and justify.

    Option #2 please.
    Updated 11-17-2017 at 07:24 AM by MyOwnWorstEnemy (typo)
  12. sdwyer138's Avatar
    Quote Originally Posted by MyOwnWorstEnemy
    Option #2 - Submissions pending final adjudication are not included on the leaderboard and are only added to the leaderboard and contribute to ESI calculation once adjudication is final.
    Big fan of this right here.
    Thanks MyOwnWorstEnemy thanked this post
  13. bensweeneyonbass's Avatar
    Rank all pending submissions as Rank # 9999 so they all show up at the bottom, or if that causes more ESI problems how about using a text string instead of a number? Or if that causes more ESI problems can there be a null value instead of a number or letter?
    Likes Blackflag82 liked this post
  14. JasonV91's Avatar
    Quote Originally Posted by bensweeneyonbass
    Rank all pending submissions as Rank # 9999 so they all show up at the bottom, or if that causes more ESI problems how about using a text string instead of a number? Or if that causes more ESI problems can there be a null value instead of a number or letter?
    Even the 9999 method will affect ESI, as it will award more to all the scores above it.

    While it's somewhat interesting to see what scores are "in process" for a track, it's not worth it if it fouls up the WR count and ESI calculation. I would prefer things to go back to the way they were - remove "in process" scores from the leaderboard, and problem solved...if the score successfully passes adjudication, it will get its place on the leaderboard soon enough.
    Likes MyOwnWorstEnemy liked this post
    Updated 11-17-2017 at 10:17 AM by JasonV91
  15. D.B. Cooper's Avatar
    Negative ESI has been a problem long before In Process submissions were included on the scoreboard.
  16. JasonV91's Avatar
    Quote Originally Posted by D.B. Cooper
    Negative ESI has been a problem long before In Process submissions were included on the scoreboard.
    Agreed, but I can only remember seeing it on tracks where negative numbers are the scores (i.e. golf games)...examples like that would be a separate issue.
  17. D.B. Cooper's Avatar
    Quote Originally Posted by JasonV91
    Agreed, but I can only remember seeing it on tracks where negative numbers are the scores (i.e. golf games)...examples like that would be a separate issue.
    Somewhere in another thread I posted a couple of examples of negative ESI for games that did not have negative scores. I did a quick scan of some scores and couldn't find any so maybe that issue has been fixed.
    Likes JasonV91 liked this post
  18. D.B. Cooper's Avatar
    I found it http://www.twingalaxies.com/showthre...995#post879995

    Looks like it was fixed so thanks to TG for that.
  19. CWK's Avatar
    Have to agree that ESI should not be calculated untill after the score is accepted. If the vote functionality needs to be removed than so be it
  20. bensweeneyonbass's Avatar
    Welp, if I had to choose between ESI not being calculated on pending submissions or having the pending submissions displayed on the leaderboard, I'd choose the latter.

    ESI isn't as important to me as being able to look up a scoreboard and see if there are any current submissions on it.
    Likes Blackflag82 liked this post
Page 1 of 2 1 2 LastLast
Join us