power_rankings_faq.pdfWhat are the Power Rankings?
The Power Rankings are a new metric designed to compare athlete performance across the National Ninja League. The NNL team, with assistance from NinjaWorks, has spent over a year developing this metric. This new metric uses a mathematical formula to rank all NNL athletes based on their relative performance, regardless of the number of competitions an athlete has done.
Each athlete will have a personal power “rating” that starts at 1500 and fluctuates up or down based on how well they do at various NNL competitions. Athletes will gain or lose points based on their own performance. The number of power ranking points gained or lost will vary depending on the size of the competition (number of competitors in that division) and the power rankings of other athletes in attendance at that comp. A bigger competition with stronger athletes will yield the winner more points. Therefore, under this method, a top athlete could theoretically compete at a single competition and show up at the top of the new Power Rankings page.
The Power Rankings will be updated after every NNL comp. Your own numerical rating will only change when you compete, although your relative standing will change when others gain or lose points around you. Athletes will be able to track their Power Ranking history, seeing how their rating changed after every comp. You can view both a high-level overview of the ranking system and the full calculations later in this FAQ.
Why have both the Power Rankings and the National Standings (traditional points)?
The two ranking systems serve different purposes: The National Standings recognize the athletes that achieved the most in the regular season (rewarding strong performance at multiple comps), while the Power Rankings recognize the athletes that performed the best relative to the competition.
Which will determine the run order at the World Championship?
As in past years, run order will first be determined by National Standings. Starting with the Season V World Championship, the “tiebreaker” will be the athlete’s Power Ranking. (In the past, the “tiebreaker” was how early in the season the athlete qualified, with athletes who qualified earlier in the season running later than athletes with the same amount of points who qualified later in the season. Now, athletes with a higher power rating will run later in Stage 1.)
Of course, as always, the run order for Stages 2 and 3 will be re-sorted based on the athlete’s performance in the previous stage, with the top clear time from the previous stage running last.
How do the Power Rankings work?
(This is a high-level overview without any math. You can view the calculations used here.)
Each athlete starts with 1500 points and gains or loses points depending on how they fare at competitions, not only based on their standing in the event but also based on the competitiveness of the field. In other words, you’ll earn more points for beating ninjas with higher Power Rankings. This is loosely based on the Elo rating system, which was developed for use in chess but also has applications in two-player games like Go and Scrabble and serves as the basic framework for online matchmaking in many video games.
The Power Rankings are not meant to reflect any advantage for athletes who were able to participate in a large number of competitions. our rating will be adjusted by an “adjustment factor” based on how many competitions you compete in.
How did you come up with the Adjustment Factor?
Season 4 data was used to evaluate the Power Rankings model based on our formula, and we tried several adjustment factors until we found one that best reflected our goals for the system. We wanted to see athletes that only competed in one or two competitions but won against a higher difficulty of field be ranked just as highly as athletes who competed in five or more competitions and performed roughly as well.
Will my Power Rankings be negatively affected if I do more competitions?
In theory, no. Obviously, you would stand to lose a small amount of points if you perform below expectations, but the Power Rankings were designed such that the number of competitions you participate in should have next to no effect on your rating.
What competition data is included in the Power Rankings?
The Power Rankings are based on performance in both NNL Season 4 and Season 5. If a competitor changed age groups between seasons, only comps from their current division are included in the calculation of their rating. The Power Ranking only includes athletes who have competed in at least one Season 5 competition (that is, athletes that competed only in Season 4 are not included).
Do the Power Rankings compare men against women, and vice versa?
No. Power rankings are only calculated within the athlete’s division (age AND gender, ex. Adult Male). In theory, we could compare males and females together as they both run the same course, but as athletes are ultimately only compared against their own gender in the individual competition standings, we wanted the Power Rankings to reflect this as well.
Will ties be broken among competitors that failed the first obstacle?
No. The Power Rankings consider anyone that failed the first obstacle to have 0 obstacles completed in 0 seconds, so matchups between two athletes that failed the first obstacle would be considered a draw. However, the competitor with the higher power ranking will lose some points and the competitor with the lower ranking will gain some points (since the lower-ranked competitor was able to perform just as well that day as the higher-ranked competitor).
The Power Rankings are a new metric designed to compare athlete performance across the National Ninja League. The NNL team, with assistance from NinjaWorks, has spent over a year developing this metric. This new metric uses a mathematical formula to rank all NNL athletes based on their relative performance, regardless of the number of competitions an athlete has done.
Each athlete will have a personal power “rating” that starts at 1500 and fluctuates up or down based on how well they do at various NNL competitions. Athletes will gain or lose points based on their own performance. The number of power ranking points gained or lost will vary depending on the size of the competition (number of competitors in that division) and the power rankings of other athletes in attendance at that comp. A bigger competition with stronger athletes will yield the winner more points. Therefore, under this method, a top athlete could theoretically compete at a single competition and show up at the top of the new Power Rankings page.
The Power Rankings will be updated after every NNL comp. Your own numerical rating will only change when you compete, although your relative standing will change when others gain or lose points around you. Athletes will be able to track their Power Ranking history, seeing how their rating changed after every comp. You can view both a high-level overview of the ranking system and the full calculations later in this FAQ.
Why have both the Power Rankings and the National Standings (traditional points)?
The two ranking systems serve different purposes: The National Standings recognize the athletes that achieved the most in the regular season (rewarding strong performance at multiple comps), while the Power Rankings recognize the athletes that performed the best relative to the competition.
Which will determine the run order at the World Championship?
As in past years, run order will first be determined by National Standings. Starting with the Season V World Championship, the “tiebreaker” will be the athlete’s Power Ranking. (In the past, the “tiebreaker” was how early in the season the athlete qualified, with athletes who qualified earlier in the season running later than athletes with the same amount of points who qualified later in the season. Now, athletes with a higher power rating will run later in Stage 1.)
Of course, as always, the run order for Stages 2 and 3 will be re-sorted based on the athlete’s performance in the previous stage, with the top clear time from the previous stage running last.
How do the Power Rankings work?
(This is a high-level overview without any math. You can view the calculations used here.)
Each athlete starts with 1500 points and gains or loses points depending on how they fare at competitions, not only based on their standing in the event but also based on the competitiveness of the field. In other words, you’ll earn more points for beating ninjas with higher Power Rankings. This is loosely based on the Elo rating system, which was developed for use in chess but also has applications in two-player games like Go and Scrabble and serves as the basic framework for online matchmaking in many video games.
The Power Rankings are not meant to reflect any advantage for athletes who were able to participate in a large number of competitions. our rating will be adjusted by an “adjustment factor” based on how many competitions you compete in.
How did you come up with the Adjustment Factor?
Season 4 data was used to evaluate the Power Rankings model based on our formula, and we tried several adjustment factors until we found one that best reflected our goals for the system. We wanted to see athletes that only competed in one or two competitions but won against a higher difficulty of field be ranked just as highly as athletes who competed in five or more competitions and performed roughly as well.
Will my Power Rankings be negatively affected if I do more competitions?
In theory, no. Obviously, you would stand to lose a small amount of points if you perform below expectations, but the Power Rankings were designed such that the number of competitions you participate in should have next to no effect on your rating.
What competition data is included in the Power Rankings?
The Power Rankings are based on performance in both NNL Season 4 and Season 5. If a competitor changed age groups between seasons, only comps from their current division are included in the calculation of their rating. The Power Ranking only includes athletes who have competed in at least one Season 5 competition (that is, athletes that competed only in Season 4 are not included).
Do the Power Rankings compare men against women, and vice versa?
No. Power rankings are only calculated within the athlete’s division (age AND gender, ex. Adult Male). In theory, we could compare males and females together as they both run the same course, but as athletes are ultimately only compared against their own gender in the individual competition standings, we wanted the Power Rankings to reflect this as well.
Will ties be broken among competitors that failed the first obstacle?
No. The Power Rankings consider anyone that failed the first obstacle to have 0 obstacles completed in 0 seconds, so matchups between two athletes that failed the first obstacle would be considered a draw. However, the competitor with the higher power ranking will lose some points and the competitor with the lower ranking will gain some points (since the lower-ranked competitor was able to perform just as well that day as the higher-ranked competitor).
