Doitsuyama

Sumo Reference Updates

Recommended Posts

If I can't get the results from the database, I will have to save all NSK pages manually (because I don't know yet how to create the script I mentioned in an earlier post to do that automatically) and extract the scrapbook data - then I can share them if required.

  • Thanks 1

Share this post


Link to post
Share on other sites

I will gladly manually input, or otherwise create input files based on, results of lower division matches.  I have no problem with tedious work that definitely accomplishes something.  (Tedious work that might not accomplish anything - that's another story.)

  • Thanks 1
  • Haha 1

Share this post


Link to post
Share on other sites
On 4.11.2017 at 16:05, dada78641 said:

Yeah, it seems they just return a big chunk of HTML by AJAX. https://gist.github.com/msikma/fd353b5f309291d1f7b93c0a53acc9f0

The English index_ajax "pages" (like sumo.or.jp/EnHonbashoBanzuke/index_ajax/1/1/ ) are at least readable, in the way that you can spot the data in the mess, but I couldn't access all pages, some returned no data. From the format seen there, one could assemble the Japanese data results - but it's easier to just open the proper pages, save and process them - with just a week left for getting the solution ready.

The daily result pages all still have an accessible direct URL - with http://sumo.or.jp/EnHonbashoMain/torikumi/1/1/  for division 1 - day 1 - thus all might get opened and saved at once from a bookmark folder with "open all tabs" in a new window and then "save all tabs" in scrapbook.

The problem is, if I open day 2 directly now http://sumo.or.jp/EnHonbashoMain/torikumi/1/2/- it changes back to day 1 - on the day itself it likely is different, but maybe all direct links won't work as planned with the new pages.

Anyway, clicking through all divisions and saving them individually is just a matter of 1 or 2 minutes.

  • Thanks 1

Share this post


Link to post
Share on other sites

Please excuse me, as the very few skills I have do not include this field...

So...could we not set up our own torikumi page or file each day manually - Akinomaki or whoever can copy and upload the next day's torikumi while Gurowake and I can update the results in real time (at least in my case)?  And then Doitsuyama can set the grabber to access THAT page instead of the NSK page.  At least this would provide timely results and also prevent havoc in the dozens of games. Or is this not practical?

With only a few days to go before the basho, we need a plan soon or it's going to be a nightmare.

Edited by Pandaazuma
  • Like 2

Share this post


Link to post
Share on other sites

It turns out to be not a problem with the NSK site but with sumogames.de server not being able to reach http://www.sumo.or.jp/ site.
It gives 403 error (Forbidden).
Because I can reach the sumo kyokai site from my local pc, I will grab the results using my grabber locally and then generate scripts and execute them on sumogames.de server.
That way I can run my games for the upcoming basho.
Regarding sumo database and other games - Doitsuyama is on vacations now, hopefully he can find a solution when he comes back.

 

  • Like 1
  • Thanks 4

Share this post


Link to post
Share on other sites
5 hours ago, Golynohana said:

It turns out to be not a problem with the NSK site but with sumogames.de server not being able to reach http://www.sumo.or.jp/ site.
It gives 403 error (Forbidden).
Because I can reach the sumo kyokai site from my local pc, I will grab the results using my grabber locally and then generate scripts and execute them on sumogames.de server.
That way I can run my games for the upcoming basho.
Regarding sumo database and other games - Doitsuyama is on vacations now, hopefully he can find a solution when he comes back.

 

Thanks, Golynohana.

I wonder why it is forbidden. Hopefully a technical reason.

Share this post


Link to post
Share on other sites

Hopefully Doitsuyama will be able to fix the problems soon.

In the meantime I'll put the extracted text files on my site, the format will be like the banzuke:

English http://achimp.de/basho/2017.11banzukE.txt Japanese http://achimp.de/basho/2017.11banzukJ.txt

- I use comma separation - the item for the reading (for sekitori: shikona first name) is only used in the Japanese version

 

  • Like 1
  • Thanks 3

Share this post


Link to post
Share on other sites

A huge Thank You! to Doitsuyama for saving the Kyushu Honbasho Sumo Gaming Masters for all the gamers!!! (Iamnotworthy...)(Applauding...)(Zabutonflying...) 

Share this post


Link to post
Share on other sites

I had to move the sumogames websites to a different (and better) server as the old server somehow wasn't able to view the sumo.or.jp website. The new DNS entry might not be there for everybody at once, but usually is available in a matter of a few hours, up to a day. If you see any hickups (which I actually expect with such a big change) please write here or in the games bugs section.

Entry for the sumo games should be possible now.

  • Like 3
  • Thanks 5

Share this post


Link to post
Share on other sites
6 minutes ago, Doitsuyama said:

I had to move the sumogames websites to a different (and better) server

That's nice to hear, because sometimes you could literally hear the old server aching under it's load.

Thank you very much, you are the best!

Share this post


Link to post
Share on other sites
14 minutes ago, Doitsuyama said:

I had to move the sumogames websites to a different (and better) server as the old server somehow wasn't able to view the sumo.or.jp website. The new DNS entry might not be there for everybody at once, but usually is available in a matter of a few hours, up to a day. If you see any hickups (which I actually expect with such a big change) please write here or in the games bugs section.

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

Share this post


Link to post
Share on other sites
15 minutes ago, Asashosakari said:

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

I asked myself the same. We shouldn't be worried because now the issue is known and it should be quite easy to circumnavigate future blocks.

Share this post


Link to post
Share on other sites
31 minutes ago, Asashosakari said:

Gotta ask: Do you think it was an (intentional?) IP block on the old server, as Golynohana insinuated? If so, should we be worried that they'll do it again?

I'm not sure at all, if you look at the connections (with LiveHttpHeader for example) there is an access cookie which I believe is new. The old server was a cloud server which had several network related limitations in the past, so it could very well been hit by one of these limitations. It could be an intentional IP block though which would be a problem as I can't just switch servers many more times. Another solution in case of sustained IP blocks would be the option to enter daily results (like video links), so anybody viewing the bouts could enter the results, making the database potentially faster than the NSK site then...

  • Thanks 1

Share this post


Link to post
Share on other sites

Is there an issue with photos of rikishi not being available at present?

 

They appear on the banzuke - if picture banzuke option is selected - but not in actual rikishi profiles.

 

Swami

Share this post


Link to post
Share on other sites
8 minutes ago, Swami said:

Is there an issue with photos of rikishi not being available at present?

 

They appear on the banzuke - if picture banzuke option is selected - but not in actual rikishi profiles.

 

Swami

Fixed

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

Many, many thanks to Doitsuyama for once again saving the sumo gaming day!  May I request that you check out here too, please?

Share this post


Link to post
Share on other sites
4 hours ago, Kaiomitsuki said:

Hello Doitsuyama (Alexander)

This link (Find a Rikishi) http://sumodb.sumogames.de/Rikishi.aspx

When you choose "Intai in ?"

You have two "2009 Haru" choice.... One at the good place, the other at the bottom of the list ;)

Yes, that's a bug - the second choice actually is selecting currently active rikishi (no intai yet). I had two choices, remove this entry or changing the text to "Active". I decided to do the latter even if the choice of the current basho in the box "Active in Basho?" is very similar.

 

Share this post


Link to post
Share on other sites

The database has been taken over by some microsoft server page now - I had relied on problems having been solved and haven't continued preparations for getting the results from the NSK - thus I don't have them now for kensho, pics and spirited rikishi. Those all will get very late.

Share this post


Link to post
Share on other sites
19 minutes ago, Akinomaki said:

The database has been taken over by some microsoft server page now - I had relied on problems having been solved and haven't continued preparations for getting the results from the NSK - thus I don't have them now for kensho, pics and spirited rikishi. Those all will get very late.

It's back up and running with full results.

Share this post


Link to post
Share on other sites

Yeah, yeah, the NSK website has changed their ajax output bigtime. I took the website offline while reworking the grabber. Actually, their changes make for much neater code as the parsing is much easier now, I basically had to remove all the unnecessary stuff and use their data structure. I'm not sure how the kyujo number for returning rikishi is done here but we'll see soon enough I guess.

  • Like 3
  • Thanks 5

Share this post


Link to post
Share on other sites

Thanks as always. I just had a glance...looks like the yusho arasoi for Makushita and below is not working.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now