Madison's 8th Annual Tecmo Super Bowl Feedback

[Knobbe]

Any sort of ranking system that someone can put together that is simply competent I'll get behind and help support.

I believe this would encourage increased participation in competition.

[Knobbe]

BTW, sorry i didn't title this "feedback" as opposed to "online critique". The connotation of that wording really doesn't reflect this thread.

[qb_browns]

As average pointed out an ELO system or any ranking system will be crippled by the lack of sample size.There really aren't any mathmatical tricks around that. The only way to increase sample size would be if you had the the full game stats for every player which I know you don't. Then you could rank on a play by play level with MUCH larger samples.

Sure you can probably generate ELO rankings but you won't be able to be very "confident" about their accuracy. A simple strength of schedule adjusted winning % or point differential adjusted for schedule like SRS on pro-football-reference.com will be nearly as accurate and much simpler.

I'm more concerned about the model first before we tackle the sample size issue. If we can determine the model that allows us to start at point A, we can then concern ourselves with point B and so on. If I sign up with the National Scrabble Association or the FIDE (the chess people), I am immediately given a beginner's rating. From that point forward, using their model, I can advance my elo rating at tournaments. If we can crank out a model--it doesn't have to be perfect, it doesn't have to be fair, it just has to be a model we can all being tinkering with--we can begin to address issues such as sample size, stats, player ratings at time of match, etc.

At the end of the day, I'm only thinking that this can be used as a tool to help live tournament seeding, allow more players to have objective feedback about their play, and allow the online community a way to begin to grab newer players through allowing them to grow up a ladder instead of either being hammered by a top player souring the Tecmo experience for them.

[bruddog]

If you have everyone's opponent the only slight improvement might be to adjust the raw win % for average opponent win% (Strength of Schedule). Don't think that will change things much since the seeding is probably decent. I

Ranking by the raw win % is going to be about as accurate as any ranking method because again...sample size.

If you want additional "feedback" for people you could break down how they've faired vs different bunchings of oppoenent strength.

A simple example

0-3 vs > .600

5-5 vs >0.400 && <0.600

4-2 vs < 0.400

The reason TO BE CONCERNED ABOUT SAMPLE SIZE is that it doesn't make sense to spend the effort coming up with some elaborate ranking system if something simple

gets you 95% of the way there. My point is that ELO will won't give any additional information over a different/easier ranking system. At the end of the day ELO just spits out a final number like anything else but I'm pretty sure its much more complex to calculate and won't really be more accurate.

The overall tournament records of online guys I know a lot about PRETTY MUCH follow their aggregate win performance in online leagues.

[qb_browns]

The reason TO BE CONCERNED ABOUT SAMPLE SIZE is that it doesn't make sense to spend the effort coming up with some elaborate ranking system if something simple gets you 95% of the way there. My point is that ELO will won't give any additional information over a different/easier ranking system. At the end of the day ELO just spits out a final number like anything else but I'm pretty sure its much more complex to calculate and won't really be more accurate.

I'm not seeking out the second coming of Arpad Elo, I'm just wondering what the model should look like at first. Do we want to focus on winning percentage? Do we want to focus on some other way to base our rankings? I'm not overly concerned about the sample size issue right now because our sample size is so small.

I guess I'm not framing the issue correctly. I would love being able to more finely seed fields across the tournament spectrum and allow others to benefit from going to these tournaments. I'm not looking to come up with some crazy model or even a high quality specific ELO-style rating. If the model thrown out already works as a basis and includes an acknowledgement of the lack of sample size, then we can start tinkering with that model. If another model for small sample size comparisons is out there that we can use and is a better fit, then that could be model. It doesn't have to be an ELO style model, we can rip off Bill James or Voros McCracken or Jonah Keri or Sebastian Pruit or anybody else from the world of statistical analysis. We're not looking to present at the Sloan Conference, we're just looking for any tweaks we can come up with for tournament play to properly seed the field on merit as opposed to reputation or "who you know."