Sumo Reference Updates

[Gurowake]

I will gladly manually input, or otherwise create input files based on, results of lower division matches.  I have no problem with tedious work that definitely accomplishes something.  (Tedious work that might not accomplish anything - that's another story.)

[Akinomaki]

On 4.11.2017 at 16:05, dada78641 said:

Yeah, it seems they just return a big chunk of HTML by AJAX. https://gist.github.com/msikma/fd353b5f309291d1f7b93c0a53acc9f0

The English index_ajax "pages" (like sumo.or.jp/EnHonbashoBanzuke/index_ajax/1/1/ ) are at least readable, in the way that you can spot the data in the mess, but I couldn't access all pages, some returned no data. From the format seen there, one could assemble the Japanese data results - but it's easier to just open the proper pages, save and process them - with just a week left for getting the solution ready.

The daily result pages all still have an accessible direct URL - with http://sumo.or.jp/EnHonbashoMain/torikumi/1/1/  for division 1 - day 1 - thus all might get opened and saved at once from a bookmark folder with "open all tabs" in a new window and then "save all tabs" in scrapbook.

The problem is, if I open day 2 directly now http://sumo.or.jp/EnHonbashoMain/torikumi/1/2/- it changes back to day 1 - on the day itself it likely is different, but maybe all direct links won't work as planned with the new pages.

Anyway, clicking through all divisions and saving them individually is just a matter of 1 or 2 minutes.

[Pandaazuma]

Please excuse me, as the very few skills I have do not include this field...

So...could we not set up our own torikumi page or file each day manually - Akinomaki or whoever can copy and upload the next day's torikumi while Gurowake and I can update the results in real time (at least in my case)?  And then Doitsuyama can set the grabber to access THAT page instead of the NSK page.  At least this would provide timely results and also prevent havoc in the dozens of games. Or is this not practical?

With only a few days to go before the basho, we need a plan soon or it's going to be a nightmare.

Edited November 6, 2017 by Pandaazuma

[Golynohana]

It turns out to be not a problem with the NSK site but with sumogames.de server not being able to reach http://www.sumo.or.jp/ site.
It gives 403 error (Forbidden).
Because I can reach the sumo kyokai site from my local pc, I will grab the results using my grabber locally and then generate scripts and execute them on sumogames.de server.
That way I can run my games for the upcoming basho.
Regarding sumo database and other games - Doitsuyama is on vacations now, hopefully he can find a solution when he comes back.

 

[Pandaazuma]

5 hours ago, Golynohana said:

It turns out to be not a problem with the NSK site but with sumogames.de server not being able to reach http://www.sumo.or.jp/ site.
It gives 403 error (Forbidden).
Because I can reach the sumo kyokai site from my local pc, I will grab the results using my grabber locally and then generate scripts and execute them on sumogames.de server.
That way I can run my games for the upcoming basho.
Regarding sumo database and other games - Doitsuyama is on vacations now, hopefully he can find a solution when he comes back.

 

Thanks, Golynohana.

I wonder why it is forbidden. Hopefully a technical reason.

[Akinomaki]

Hopefully Doitsuyama will be able to fix the problems soon.

In the meantime I'll put the extracted text files on my site, the format will be like the banzuke:

English http://achimp.de/basho/2017.11banzukE.txt Japanese http://achimp.de/basho/2017.11banzukJ.txt

- I use comma separation - the item for the reading (for sekitori: shikona first name) is only used in the Japanese version

 

[onepaperkid]

Thank you, Doitsuyama!!!!

[kuroimori]

A huge Thank You! to Doitsuyama for saving the Kyushu Honbasho Sumo Gaming Masters for all the gamers!!!  

[Kintamayama]

All is well-thanks to the Wizard.

[Doitsuyama]

I had to move the sumogames websites to a different (and better) server as the old server somehow wasn't able to view the sumo.or.jp website. The new DNS entry might not be there for everybody at once, but usually is available in a matter of a few hours, up to a day. If you see any hickups (which I actually expect with such a big change) please write here or in the games bugs section.

Entry for the sumo games should be possible now.

[Benihana]

6 minutes ago, Doitsuyama said:

I had to move the sumogames websites to a different (and better) server

That's nice to hear, because sometimes you could literally hear the old server aching under it's load.

Thank you very much, you are the best!

[Asashosakari]

14 minutes ago, Doitsuyama said:

I had to move the sumogames websites to a different (and better) server as the old server somehow wasn't able to view the sumo.or.jp website. The new DNS entry might not be there for everybody at once, but usually is available in a matter of a few hours, up to a day. If you see any hickups (which I actually expect with such a big change) please write here or in the games bugs section.

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

[Benihana]

15 minutes ago, Asashosakari said:

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

I asked myself the same. We shouldn't be worried because now the issue is known and it should be quite easy to circumnavigate future blocks.

[Doitsuyama]

31 minutes ago, Asashosakari said:

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

I'm not sure at all, if you look at the connections (with LiveHttpHeader for example) there is an access cookie which I believe is new. The old server was a cloud server which had several network related limitations in the past, so it could very well been hit by one of these limitations. It could be an intentional IP block though which would be a problem as I can't just switch servers many more times. Another solution in case of sustained IP blocks would be the option to enter daily results (like video links), so anybody viewing the bouts could enter the results, making the database potentially faster than the NSK site then...

[Swami]

Is there an issue with photos of rikishi not being available at present?

 

They appear on the banzuke - if picture banzuke option is selected - but not in actual rikishi profiles.

 

Swami

[Doitsuyama]

8 minutes ago, Swami said:

Is there an issue with photos of rikishi not being available at present?

 

They appear on the banzuke - if picture banzuke option is selected - but not in actual rikishi profiles.

 

Swami

Fixed

[Pandaazuma]

Thanks a lot, Doitsuyama.  We all really appreciate what you do.

[Jejima]

Many, many thanks to Doitsuyama for once again saving the sumo gaming day!  May I request that you check out here too, please?

[Kaiomitsuki]

Hello Doitsuyama (Alexander)

This link (Find a Rikishi) http://sumodb.sumogames.de/Rikishi.aspx

When you choose "Intai in ?"

You have two "2009 Haru" choice.... One at the good place, the other at the bottom of the list ;)

Edited November 11, 2017 by Kaiomitsuki

[Doitsuyama]

4 hours ago, Kaiomitsuki said:

Hello Doitsuyama (Alexander)

This link (Find a Rikishi) http://sumodb.sumogames.de/Rikishi.aspx

When you choose "Intai in ?"

You have two "2009 Haru" choice.... One at the good place, the other at the bottom of the list ;)

Yes, that's a bug - the second choice actually is selecting currently active rikishi (no intai yet). I had two choices, remove this entry or changing the text to "Active". I decided to do the latter even if the choice of the current basho in the box "Active in Basho?" is very similar.

 

[Akinomaki]

The database has been taken over by some microsoft server page now - I had relied on problems having been solved and haven't continued preparations for getting the results from the NSK - thus I don't have them now for kensho, pics and spirited rikishi. Those all will get very late.

[Pandaazuma]

19 minutes ago, Akinomaki said:

The database has been taken over by some microsoft server page now - I had relied on problems having been solved and haven't continued preparations for getting the results from the NSK - thus I don't have them now for kensho, pics and spirited rikishi. Those all will get very late.

It's back up and running with full results.

[Doitsuyama]

Yeah, yeah, the NSK website has changed their ajax output bigtime. I took the website offline while reworking the grabber. Actually, their changes make for much neater code as the parsing is much easier now, I basically had to remove all the unnecessary stuff and use their data structure. I'm not sure how the kyujo number for returning rikishi is done here but we'll see soon enough I guess.

[Pandaazuma]

Thanks as always. I just had a glance...looks like the yusho arasoi for Makushita and below is not working.

[Senkoho]

Could someone explain this to me? Looks like the DB is not really  familiar with the 2017 Kyushu banzuke...

(works fine for other basho)

Edited November 14, 2017 by Senkoho