Asashosakari 16,843 Posted January 10, 2010 As you know, I even collected data about the highest KK ranks during rikishi careers. But by and large, SPSS has shown me that for rikishi in their third year onward the best predictor for "career highest rank" (to use the terms from the Doitsubase) is "highest rank" and not "highest KK rank". I have little doubt that it's true in the aggregate, I just have a nagging feeling about those 7-0 and 6-1 jumps (especially the former) and the distortions they temporarily introduce in a particular rikishi's assessment. The prototypical case is probably Minaminoshima, who fluked a 7-0 and proceeded to follow up with six MK that dropped him back down beyond his yusho rank. My impression is that rikishi who go 7-0 and aren't clearly below their current talent level at the time tend to have a rather high likelihood of such big corrections, before they're back on their "actual career path" again. Still, there's probably a healthy dose of hindsight involved, which is why I'm posting my reservations about Kaisei (his 3-1 start last basho almost caused me to waver...) and Takanoiwa, so they can be suitably ridiculed in a few months. (Shaking head...) I'm still pondering how I feel about Kurosawa... Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted January 10, 2010 (edited) If you're up to it, I'd love to see a past snapshot (quite a ways back, say, from Hatsu 2000) so we can see what the biggest hits and misses have turned out to be. The algorithm may still struggle with the makushita level, but perhaps that just means there are more factors yet to be identified, and the combined brainpower of the forum might spot a few things. Will Hatsu 2003 be OK as well? (I still have that file on my computer, the Hatsu 2000 was overwritten). All those predictions are based on data until 2003, not on the current data. First of all, the hits, i.e. guys correctly predicted for future sekitorihood ("high" is their best rank until 2003; "c-high" is their later or current career high): Rank Shikona high c-high pred ========================================== Ms1w Dewanofuji ms1 J9 M15 Ms3e Kakizoe ms3 K J5 Ms3w Hidenokuni ms3 J13 J9 Ms6w Kokkai ms1 K M7 Ms9w Tanaka ms1 J4 J5 Ms13w Tochifudo ms3 J12 J12 Ms14w Yoshiazuma ms5 J14 M9 Ms20e Kotokikutsugi ms20 S O Ms20w Raido ms5 J2 M2 Ms21w Toyonokuni ms2 J13 J9 Ms27e Tamaasuka ms14 M9 M2 Ms33w Ryuo ms33 M8 M4 Ms34e Oga ms9 J6 J12 Ms37e Hochiyama ms28 M14 M9 Ms40w Hakuba ms40 M16 M2 Ms44e Chiyohakuho ms27 M6 O Ms46e Ama ms15 O M4 Ms50e Kirinowaka ms50 J4 M2 Ms53e Musashiryu ms53 M2 M7 Sd3w Suzukawa ms48 M9 M2 Sd6w Toyonoshima sd5 S O Sd16e Hakuho sd16 Y M2 Sd25w Hokutokuni sd25 J12 M2 Sd28e Koryu sd28 M11 M2 Sd33w Tokitenku sd33 K M13 Sd43w Yakigaya ms55 M16 M11 Sd49w Hagiwara sd49 S O Sd53e Daiyuchi sd53 J10 M9 Sd62e Hoshihikari sd46 J1 J3 Sd64w Hakurozan sd64 M2 O Sd65e Mokonami sd41 M7 J5 Sd76w Kakuryu sd40 S M2 Then we have the misses - rikishi that made it to sekitori without my computer realizing it: Rank Shikona high c-high pred ========================================== Ms8w Asofuji ms4 M13 D Ms19w Yotsuguruma ms19 J8 Ms9 Ms21e Shigezakura ms20 J12 Ms9 Ms24e Kyokunankai ms20 J2 Ms17 Ms28e Daimanazuru ms2 M16 D Ms32e Dewanosato ms14 J14 D Ms33e Maikaze ms8 J10 D Ms36e Raiko ms20 J11 Ms9 Ms39e Kotokuni ms16 J4 Ms9 Ms41e Kotokasuga ms22 M16 Ms17 Ms49w Roho ms49 K Ms9 Ms55w Bushuyama ms23 M3 Ms3 Sd6e Kotoshiiba ms50 J14 Ms29 Sd19e Katayama sd19 M13 Ms19 Jk30w Kotooshu jk30 O Ms34 Jk31w Hoshikaze jk31 J12 Jd19 And finally - false alarms (rikishi for whom I predicted sekitorihood, but who haven't made it): Rank Shikona high c-high pred ========================================== Ms2w Asamiyoshi ms2 ms2 O Ms5w Shishio ms1 ms1 J12 Ms9e Fujinohana ms1 ms1 M15 Ms17w Nadatsukasa ms6 ms4 J9 Ms19e Kimenryu ms4 ms4 J11 Ms22e Daishoma ms7 ms4 M13 Ms22w Kakuo ms8 ms8 J12 Ms23e Tashiro ms7 ms7 M15 Ms24w Hokutoarashi ms6 ms6 J7 Ms25e Asahibenten ms5 ms2 J9 Ms26w Otoryu ms3 ms3 J9 Ms31w Kirinofuji ms21 ms12 J1 Ms36w Murata ms36 ms3 M15 Ms38e Kotohino ms9 ms9 J12 Ms40e Matsumidori ms30 ms14 J9 Ms46w Kirinoumi ms12 ms10 J9 Ms48e Kainowaka ms14 ms14 J9 Ms52w Hisanohana ms45 ms45 J1 Ms57e Musashifuji ms7 ms7 J12 Sd5w Tatsuyutaka sd5 ms12 J11 Sd8e Aoba ms12 ms9 J1 Sd10w Ominami ms31 ms31 J9 Sd12e Takamihana ms4 ms4 J1 Sd18e Niioka sd18 ms41 M2 Sd21e Rachimi ms4 ms4 M9 Sd23w Hoshizakura ms59 ms12 M9 Sd29e Fusanohana ms13 ms13 J12 Sd32w Hananosato sd9 ms8 J9 Sd39w Daishochi sd23 ms15 M9 Sd44e Nakabuchi ms12 ms12 J12 Sd49e Kamakura sd24 ms56 J3 Sd61e Yoshimura sd61 sd40 O Sd61w Chojimaru sd46 ms3 J3 Sd87e Munakata ms13 ms10 J12 Sd93e Satoyama sd93 ms58 J1 Sd97w Nakanokuni sd97 ms3 O Sd99w Ota sd99 sd66 M15 Jd3w Daionami sd74 ms33 J5 Jd25e Wakamusashi jd25 sd53 O Jd30w Nionoumi jd30 ms16 O Jd37w Shimada jd37 ms20 O Jd40w Hokkairyu jd40 sd13 O Jd52w Sasaki jd31 ms18 O Edited January 10, 2010 by Randomitsuki Share this post Link to post Share on other sites
Asashosakari 16,843 Posted January 10, 2010 Then we have the misses - rikishi that made it to sekitori without my computer realizing it: Rank Shikona high c-high pred ========================================== Ms39e Kotokuni ms16 J4 Ms9 Ms41e Kotokasuga ms22 M16 Ms17 Sd6e Kotoshiiba ms50 J14 Ms29 Jk30w Kotooshu jk30 O Ms34 I think we've identified one factor. (Shaking head...) Seriously though, many thanks for posting that, something to delve into the next few days... Share this post Link to post Share on other sites
kaiguma 0 Posted January 10, 2010 (edited) And perhaps as a warning (to myself as well) to not take these snapshots too seriously...Rendaiyama, 22 years, 24 basho, Ms46->Ms17 Tochihiryu, 22 years, 24 basho, Ms43->J3 (Just to make sure - for that career length, the predictor being used is the high rank, not the current rank, right?) Thanks for pulling out a specific example for me to try and latch on to my understanding gap. SO assuming I understand what Asasho is assuming, Rendaiyama and Tochihiryu are in different subgroups of their cohort right? And thus bigly bigly different predictors. But even if this is so, and creates much-needed deviation, what is the true range of a predictor window? For example, in your first set you showed us 0 - 11 averaging 5.5 and 5.5 happens to be what? M15? And if 0 is Yokozuna then what is 11? So if a subgroup carries a potential range of Yokozuna to somewhere in high Makushita (?Ms3!) we are just going to call all of them M15? After all of your efforts to this point it seems rather pointless. There must be a better way of creating more deviation and spreading out the predictor within the subgroup rather than using the midpoint and sticking it to every single member of the cohort. Asashosakari's oft-repeated comment on highest KK seems very apt. Of course you'd have to write new algorithms to test the idea and figure out how to apply it, but essentially a subclass system would be identified within each subgroup. This would result in another and much larger set of "no data results" so you'd have to create a secondary predictor column, let's call it "refined." If enough data exists, you could pinpoint the predictor rather using the midpoint. Just in case I haven't made sense, let's take Kaisei and say within his cohort he is subgrouped with Tochinowaka (Ri). For the sake of this argument, both have career highs of Ms6 and both are predicted for M3. I am assuming that M3 represents a range of Y - M10. (0-4.25?) So additional mining might find that subclasses exist within the subgroup: those with career high KK within 3 points of the high, 5 points of the high, 8 points, and more than 8. [A B C D] Kaisei is thus a 'D' and Tochinowaka having achieved KK from Ms 13 (7 ranks = 2 points) is an 'A' class within the subgroup. Kaisei's prediction is refined to a range of M6 - M10 and so he is predicted at M8. Tochinowaka is refined to Y-S and so Ozeki is the refined predictor for subclass A. Also, can we have an asterisk somewhere for every member of a subgroup with say, less than 10 members? Of course not for this set, but if it's not too much trouble for the next one you do? And 10 was arbitrary. More stato-minded folks could debate on where they would want to see an asterisk. Essentially since we know when there is not enough data I'd like to know when we had "just barely enough data" as well. Anyway, my critique notwithstanding, I am very impressed by the results of your first public run for the data. Arigato gozaimashita. ps - and of course you mentioned that your data shows the best results in Sandanme and below. Maybe that's what I am getting at. Some (intelligent) variation on my example above might provide what's missing to make the data more valuable above the sandame level. Also, why not publish a predictor history of key rikishi throughout their career. For example, all of Goeido's prediction values from Jonokuchi to Sandanme would help a lot in assessing our moving target. Edit: aha! so while writing this post several of the issues were addressed by the main parties of this conversation. go figure (Shaking head...) Edited January 10, 2010 by kaiguma Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted January 10, 2010 SO assuming I understand what Asasho is assuming, Rendaiyama and Tochihiryu are in different subgroups of their cohort right? And thus bigly bigly different predictors. Exactly. But even if this is so, and creates much-needed deviation, what is the true range of a predictor window? This depends on the cohort, the ranges will become narrower, the more data points I have. For example, in your first set you showed us 0 - 11 averaging 5.5 and 5.5 happens to be what? M15? And if 0 is Yokozuna then what is 11? So if a subgroup carries a potential range of Yokozuna to somewhere in high Makushita (?!) we are just going to call all of them M15? After all of your efforts to this point it seems rather pointless. You should not forget that this example was for rikishi in their third basho. Even the best rikishi of that cohort (Kitanoumi) was Jd95 at that time. Seen from this angle, having a window "somewhere between Yokozuna and high Makushita" isn't all too shabby for a guy in his third basho, and in some murky Jonidan areas - particularly as the cohort wasn't that big. There must be a better way of creating more deviation and spreading out the predictor within the subgroup rather than using the midpoint and sticking it to every single member of the cohort. That's what I suggested in post #24. I will think about spreading predictions within categories. Also, can we have an asterisk somewhere for every member of a subgroup with say, less than 10 members? Of course not for this set, but if it's not to much trouble for the next one you do? And 10 was arbitrary. More stato-minded folks could debate on where they would want to see an asterisk. Eseentially since we know when there is not enough data I'd like to know when we had "just barely enough data" as well. Sounds reasonable, and it will be implemented at some point. I don't know when this will be the case. The program that seeks for sub-groups within cohorts takes some 15 hours to run on a decent computer... Anyway, my critique notwithstanding, I am very impressed by the results of your first public run for the data. Thanks for the feedback - it is much appreciated. (Shaking head...) Share this post Link to post Share on other sites
Asashosakari 16,843 Posted January 10, 2010 (edited) I wonder if the tradeoff between reduced fit and improved sample size would make it worth to combine cohorts a bit more. That Rendaiyama/Tochihiryu example came out rather extreme...given the limited level of detail in the age parameter (i.e. both 22 years, not 22y 0m and 22y 8m), perhaps stretching the basho groupings a bit (e.g. from 22 to 26 career basho in their case) would be beneficial once you're at double-digit counts? Edit: Of course, that would result in the same number of cohorts but more population in each, so 15 hours probably won't get it done then... Edited January 10, 2010 by Asashosakari Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted January 10, 2010 I wonder if the tradeoff between reduced fit and improved sample size would make it worth to combine cohorts a bit more. That Rendaiyama/Tochihiryu example came out rather extreme...given the limited level of detail in the age parameter (i.e. both 22 years, not 22y 0m and 22y 8m), perhaps stretching the basho groupings a bit (e.g. from 22 to 26 career basho in their case) would be beneficial once you're at double-digit counts?Edit: Of course, that would result in the same number of cohorts but more population in each, so 15 hours probably won't get it done then... The idea is a good one (apart from the computational issue), but I think that many shortcomings might be already solved by implementing "floating" predictions rather than "categorical" predictions. I have just manually re-computed the Tochihiryu vs. Rendaiyama case, and instead of Juryo 3 vs. Makushita 17 it now has become Juryo 8 vs. Makushita 4. It still seems like a large gap given their career-highs of Ms43 vs. Ms46, but in fact the Ms43 were achieved on a larger banzuke than the Ms46. The converted ranks are 20 vs. 22, respectively. It seems as if the implementation of floating predictions does not require much, so I'll definitely give it a try in the next days. Unfortunately, the week at work will be worrisome, and I've heard also that there is a basho going on. Share this post Link to post Share on other sites
Zenjimoto 39 Posted January 11, 2010 Don't worry about work, just call in sick! (Holiday feeling...) Share this post Link to post Share on other sites
Asashosakari 16,843 Posted January 11, 2010 (edited) The idea is a good one (apart from the computational issue), but I think that many shortcomings might be already solved by implementing "floating" predictions rather than "categorical" predictions. That will only solve the issue of large divergences within a cohort though, not the issue of wildly shifting projections between similar cohorts, right? What I mean is, (barring a new high rank) once the next basho rolls around Tochihiryu and Rendaiyama should still be projected relatively close to J8 and Ms4 as they're merely "one basho older", but it doesn't sound like that's necessarily going to be the case. That's where having a rolling window of aggregated cohorts would probably help, e.g. this time they'd be projected according to those historical rikishi with 22-26 basho, next time according to those with 23-27, etc. Edited January 11, 2010 by Asashosakari Share this post Link to post Share on other sites
Washuyama 587 Posted January 11, 2010 Awwww.. Moriurara is already in decline... I had high hopes now that he knows what KK is like.. Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted January 11, 2010 The idea is a good one (apart from the computational issue), but I think that many shortcomings might be already solved by implementing "floating" predictions rather than "categorical" predictions. That will only solve the issue of large divergences within a cohort though, not the issue of wildly shifting projections between similar cohorts, right? What I mean is, (barring a new high rank) once the next basho rolls around Tochihiryu and Rendaiyama should still be projected relatively close to J8 and Ms4 as they're merely "one basho older", but it doesn't sound like that's necessarily going to be the case. That's where having a rolling window of aggregated cohorts would probably help, e.g. this time they'd be projected according to those historical rikishi with 22-26 basho, next time according to those with 23-27, etc. I played a little bit around with your idea. In most cases, there is no need to take several cohorts into account, as 5/6 of rikishi who were in your last cohort will follow you into your next cohort. However, there is one crucial stage, and that is when a rikishi gets older. In the first basho after a birthday, he'll suddenly find himself in a cohort that only has 1/6 overlap with the last one. In these cases your suggestion will have the best effects. One would get an even better inter-basho overlap by not only spreading out over basho number, but by spreading out over individual trajectories. For example, if a 16-year old rikishi is in his 7th basho, and next basho he will be 17 in his 8th basho, then it is better to spread over the cohorts (16-6, 16-7, 17,8) than over the cohorts (16-6, 16-7, 16-8). Programming for this will be a little tricky, but far from impossible. The computational overhead is also no problem, as the 15-hour runtime refers to getting the parameters, not to actually making predictions based on the parameters. Alas, there is a rub to all this. If you spread over multiple cohorts, it will ultimately end up in more conservative predictions. At middle or late career stages this effect is negligible, but very early on this could become an issue. For instance, if I decided to spread cohorts for someone in his 3rd basho (i.e. include guys in their 2nd, their 3rd, and their 4th basho) and take the total average of these three cohorts, this average would be higher (and the predicted rank would be lower) than the average for 3rd basho only - this is a side effect of faster promotions in the early career stages. Share this post Link to post Share on other sites
rhino 0 Posted January 11, 2010 Thanks for the really interesting analysis! I have one suggestion, if I could be so bold. It seems that because the preferred indicator changes from basho to basho, you end up with a lot of variability in the expected trajectory. So bearing that in mind, I would be interested to see a moving average of what the predicted trajectories have been over the past year. I feel I haven't written that too clearly so here's a made-up example. Suppose over the course of a year Goeido's predicted career high changes as follows: 96 (Hatsu), 99 (Haru), 100 (Natsu), 99 (Nagoya), 97 (Aki), 99 (Kyushu) Then the number I would be interested in seeing is (96 + 99 + 100 + 99 + 97 + 99)/6 which might give a more stable and perhaps more accurate prediction for higher-ranking rikishi. Obviously with rikishi who have had short careers this might actually make matters worse because the earlier predictions will be based on less informative data. Anyway thanks a lot, I'm impressed by your hit-rate below Makushita. Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted January 11, 2010 I have one suggestion, if I could be so bold. It seems that because the preferred indicator changes from basho to basho, you end up with a lot of variability in the expected trajectory. So bearing that in mind, I would be interested to see a moving average of what the predicted trajectories have been over the past year. Thanks for the input! ;-) However, implementing this idea would require a complete rewrite of the code, as I do not save past predictions in a format that can be easily used. Moreover, while your suggestion certainly will stabilize the predictions, there can be such thing as too much stabilization. If a rikishi strongly improves, your method would be quite slow to react to this change. Still, I might give it a try, but given the massive re-coding it's not my preferred strategy right now. Share this post Link to post Share on other sites
Asashosakari 16,843 Posted January 11, 2010 Alas, there is a rub to all this. If you spread over multiple cohorts, it will ultimately end up in more conservative predictions. At middle or late career stages this effect is negligible, but very early on this could become an issue. For instance, if I decided to spread cohorts for someone in his 3rd basho (i.e. include guys in their 2nd, their 3rd, and their 4th basho) and take the total average of these three cohorts, this average would be higher (and the predicted rank would be lower) than the average for 3rd basho only - this is a side effect of faster promotions in the early career stages. No, I was also thinking of a sort of escalating approach, where the very early career stages are estimated according to single basho, and then later perhaps three basho, and even later five etc. I agree that averaging things too much in the beginning is likely to obscure important data points. And besides, for the early stages there should also be more data anyway since almost every rikishi will see e.g. his third basho, so there's (I think...) less of a need to create a larger sample size by grouping. Of course, I don't have to implement it, so anything I suggest here is little concerned with the practicality of it all. ;-) Share this post Link to post Share on other sites
Asashosakari 16,843 Posted January 17, 2010 Shikona Rank Age Exp Best Rank Prediction ========================================================== Takanoiwa Ms13w 19 7 Makushita 13 Ozeki I'll boldly predict he'll be exhibit #239 for my belief that career-high rank is not quite useful when it deviates significantly from the career-high KK rank. (Kaisei will be exhibit #238.) Still, there's probably a healthy dose of hindsight involved, which is why I'm posting my reservations about Kaisei (his 3-1 start last basho almost caused me to waver...) and Takanoiwa, so they can be suitably ridiculed in a few months. (Sign of approval...) I'm still pondering how I feel about Kurosawa... Good thing nobody listens to me anyway. (I still find Kaisei's sumo extremely unimpressive though...) Share this post Link to post Share on other sites
Randomitsuki 2,525 Posted March 13, 2010 I just wanted to admit that I haven't worked a bit on the career predictions data. (Sign of approval...) As the data are still plagued with many flaws, I will not publish them until further notice. Share this post Link to post Share on other sites
