Akinomaki

Tips and tools to get the sumo coverage from the net

Recommended Posts

On 4.10.2015 at 08:09, Naganoyama said:

This is the kind of posting which would be great to have in some kind of information repository. Perhaps you could start a thread with this in it and people could add to its over time?

I'll try to create a thread with enough sumo content to make in into Sumo Information.

I won't go into detail about the costly NSK coverage of all sumo there is at each basho, nor about the also costly alternatives to the late NIJI to get Japanese TV: only the free resources and how to best get them onto your computer - I don't know much about getting them to a smart phone or tablet.

Basically you'll find a very complete visual coverage of the basho in the Honbasho Talk threads each basho. Kintamayama does the makuuchi, luispereira7cv posts his links to the complete NSK juryo coverage and dohyo-iri, Asojima directs us to the Mongolian commented NHK access and others fill the Sumo Reference database with links to the bouts of the payed ranks and a selection of lower division bouts.

I do only the news side, pics and some videos.

Most videos are to be found on YouTube. The embedded videos on the forum and the database have to be opened (again) on the YouTube site itself to get them. To download them many tools exist, my best choice is the "YouTube Video and Audio Downloader" add-on for Firefox. It is one of few able to to get the videos in the best quality available. The others usually can't get the 480p and 1080p superior qualities, because YouTube keeps the video and audio separate for them.

You have to set the Video and Audio Downloader to select the best Audio automatically when you select to download as "Video Only" (I think that's the default anyway) and the program joins the 2 after download. You can therefore select the best video quality available and get the best video (careful to select an mp4 version and not one of the other).

Most videos from other sites, especially Ustream and NicoNico are best to be downloaded with the VideoDownloadHelper Firefox add-on.

Apart from the NSK videos of the basho, Ustream has other free channels, often amateur sumo national and international tournaments, but e.g. Takadagawa-beya has it's own channel with videos from events at the heya about once year.

Most live coverage on Ustream can be recorded with the program Livestreamer.

To play and with it download most of the videos on NicoNico, which covers live many events like sumo lessons, special jungyo or also had Kakuryu's first official dohyo-iri and shrine visit, you have to register for free and login. Unfortunately I don't know how to get the recorded live coverage videos, for which, to watch them later, you have to make a time-shift reservation BEFORE the event starts. Maybe a tool for screen recording is the only way for them. Downloadhelper wants an extra module installed for that (also needed for downloading most of the higher quality YouTube videos), I haven't found a free one yet that works.

And back to YouTube, to find more videos not yet on the forum - and maybe post your finds then - you can of course simply search for "sumo", but you'll find more if you search for words in Japanese.

YouTube has a rather intelligent(?) search, searching just for sumo 相撲 gets other videos than if searching for ozumo 大相撲 - though you might think the latter are all included in the first search.

Sumo 相撲 gets you maybe 80% crap, because it includes paper sumo, arm wrestling, bugs sumo, robot sumo, backside sumo, leg sumo and some pervert sumo as well. But there you get amateur sumo and sometimes Japanese sumo programs that won't show in the ozumo search.

Ozumo 大相撲 maybe has only 50% crap - unfortunately to find the real good contents without having to check all promising ones, you need to know a bit Japanese.

Other searches you might try - remember to sort by date to only get the newest videos (and better restrict to video to eliminate playlist floods) and that the words are used for other areas than sumo as well, some more, some less:

Yokozuna 横綱 - Rikishi 力士 - Jungyo 巡業 - Oyakata 親方

Rikishi names are more promising, unless they are common names like Endo.

You can get the names from the rikishi pages in Sumo Reference when you switch to Japanese.

To have maybe more results shown and to play some videos that are blocked in your country, the Youtube Unblocker add-on for Firefox is sometimes helpful, those for Japan only mostly won't play with that either, it's rather meant for US and European contents.

If you have a video that's blocked then, regardless from which site, you might have to use a proxy.

More on that in my post in the thread for watching region blocked videos.

That also contains my old procedure for watching hidden NHK news videos, but that is unnecessary now.

So finally, the info that is the reason for this thread.

Thanks to Rhyen I found out that the hidden videos can be displayed and saved the same way as those visible for all.

I from now on will post the video links of NHK news videos to paste into VLC or a similar media player, to play the video and record it or to use convert to directly save the raw input.

The news pages itself are gone in 1 or 2 days for local and a week for the usual news.

The videos last less than a month for sports news, this example link of the arrest of Kumagatani-oyakata from general news still works:

rtmp://flv.nhk.or.jp/ondemand/flv/news/k10010214781_201509031513_201509031514.mp4

VLC can record nearly everything, at least the windows version (the android version can't do anything). I think the NHK videos have to be accessed with the rtmp protocol, so you need a tool that supports it and can also save it.

If you are not used to VLC, you won't get to the raw input saving easily, simply recording won't record the first few seconds (and the recording button is also hidden at first).

Ctrl-N opens the "Open media" dialog with "network", into where you have to paste the rtmp link.

Alt-O then opens the convert dialog: there select "save raw input" instead of "convert".

You either have to paste the complete path of the destination file into the field for that (the file mustn't exist - override usually fails, the folder must), or have to select the folder and enter the name of the new destination file with .mp4 or .flv.

VLC can record also (news) videos from other sites, the problem is to find the download link.

So much for the introduction. I can go into detail if there is interest and hope of course others will add their info about their ways to get to contents.

I might also add more info about getting to pictures, getting to the news may rather be a thread for Japan Japanese discussions.

Edited by Akinomaki
  • Like 20

Share this post


Link to post
Share on other sites

The basho is over, but sumo goes on.
Now the next round in details about getting the coverage from the net, in the hope to help others to enjoy sumo at the fullest also in between basho, like I do.
All my solutions are based on using Firefox with the right add-on extensions.

Again Rhyen provided the information to get me starting with it.
Here we have a 1080p video blocked in many countries, I will later post the news about the new song from KUNI for Harumafuji, a memory of his late father.


When YouTube-Unblocker Prox Tube is installed, Firefox has no problems to play this video also embedded here.

I don't know if Prox Tube is also as smart as YouTube-Unblocker was:
(This add-on only fools YouTube about the destination at the beginning, the file won't be loaded as a whole through a proxy, which would result in very slow replay.)

The info tells that Prox Tube changes the proxy for the video and back when leaving the page, but as soon as the download starts one can leave the page.
The tool also helps when searching for sumo videos. If the video won't play in your country, it also won't be shown to you in e.g. the search in Japanese for "yokozuna" like mentioned in my first post above. In a play-list it appears though. To check also for blocked videos, YouTube-Unblocker Prox Tube provides an "Prox Tube Search" button. You have to click on it for every new search though.

To download the video you have to open it directly in YouTube though with my method.
Other add-ons like Downloadhelper demand a converter that is not free, YouTube-video and -audio downloader is the choice to get the best video, especially if it is a blocked one.

There are 3 very good reasons for downloading a video:
- to save it before it disappears, to be able to watch it when you have time - and anytime again
- to enjoy the best quality without interruption when you don't have a both super fast AND reliable Internet connection
- to be able to use in full the deluxe replay capacities of VLC, that I'll explain later

YouTube-video and -audio downloader has quite some options for configuration.
The most important: It needs FFmpeg to join the audio and video parts of the 480p and 1080p videos. It has to be installed separately, the instructions in the options of the add-on tell you from where and there is a "proceed" button in the end to start the set-up.

As soon as you have the FFmpeg location option in order and have the options checked for automatic best audio extraction when selecting video only, YouTube-video and -audio downloader automatically joins the parts after the download of both parts is completed.
For large video files this causes Firefox to freeze at that point, that's a disadvantage of this add-on solution.
I advise to also use the Download Status Bar add-on, to be able to see how your downloads are proceeding and to have easy control over it.
The hope of this thread is that others add their alternative or a better solution.

Some other options that should be changed to make things easier:
- Setting the download folder to the one of your choice: most downloaders use their own folder. That makes it tedious to work with the files, especially when you have several ways to download videos: all may end up at different locations otherwise.

- Only display mp4 in the injected panel, which gets displayed when clicking on the download button, set to be displayed beneath the YouTube player:
-- removing the other formats from there avoids to accidentally download the video as webm
-- and also reduces the number of options displayed - you don't have to go to page 2 of the options to select the option for download of the best video-only available (which has the tool download the best available audio as well and join them, like mentioned above)

Downloadhelper is still the best tool for most other sites.
Increasingly, it becomes difficult to fool sites that demand the use of Flash player with add-ons like Video Without Flash, but with some, like USTREAM, Downloadhelper can still download the video as .flv
To scan through it faster in VLC, the video should be in .mp4 format though.

I use XMedia Recode for speedy batch conversion to mp4. With the option copy instead of convert, conversion is fast, even with changing audio tracks. It also removes most superfluous code that may hinder smooth replay in VLC.
Splitting it into parts or removing molesting commercials at the beginning is also fast. XMedia Recode just lacks the capability to join the resulting parts again.
It also helps to remove annoying frames around a video, but that can't be done in copy mode and makes the process slow.

Now with the video in proper form, scanning through it at high speed with VLC is a must for me, to decide in a few seconds if a video is worth watching. And of course I need VLC for fast forward/backward and jumping to some location (in flv format this takes much longer). I also like that you can play the video clean at 1.5 up to triple speed with the audio still recognizable.

I have the necessary Hotkey configuration for VLC on + and - for playing the video faster and slower, and the left and right arrow buttons in combination with SHIFT for very short jump and with Ctrl for normal jump (with alt for short etc., but I only need the 2 others).
The crucial point is to set length of these jumps to the best value: for that displaying all of the options is necessary and to edit the Hotkey settings in "Interface". I use 3sec for very short and 30sec for normal jump. You should play around with values to find those that suit you.

With the video in proper format, holding down shift with the right arrow while playing it gives a super fast forward while still being able to watch all scenes. With Ctrl-right both held down, rushing to any region of the video is done in an instant.

Edit: YouTube-Unblocker is now blocked by Firefox as potentially malicious

Edited by Akinomaki
  • Like 4

Share this post


Link to post
Share on other sites

What to search for is a good tip. I use Browsec proxy on Chrome so I can watch F1 on the BBC iPlayer, it thinks I'm in the UK. I will try it for this too. Well Browsec only has Euro connections. I have a proxy service with PIA, Private Internet Access so I will try shifting it to Japan as the source.

Edited by 808morgan

Share this post


Link to post
Share on other sites
What to search for is a good tip. I use Browsec proxy on Chrome so I can watch F1 on the BBC iPlayer, it thinks I'm in the UK. I will try it for this too. Well Browsec only has Euro connections. I have a proxy service with PIA, Private Internet Access so I will try shifting it to Japan as the source.

you can comment outside the boxes to avoid confusion

like this

Edited by hamcornheinz

Share this post


Link to post
Share on other sites

I'm not sure if I should make this part rather a different topic in the Japan-Japanese sub-forum, but I still have the hope to get with this another member to help me with pic posting.

So this one is about how to get the pics (and the news in Japanese).
I look at the new articles of the Japanese press to get the pics - with their context - and if there is one, I check the picture lists of a site - esp. if it is a site where I might have overlooked a pic - or where there are pics that are not in articles.
Sometimes this involves to look through the general pics lists, because a sumo-only pics page exists only for very few papers.
I won't include the pics on twitter, blogs etc. here, though I use them sometimes in a post - other than during the basho.

I check 2 times a day, in the morning here, at around 16:00h JST, to get all from the morning papers, and at around 0:00h JST, to get all new articles of the day. Checking earlier - during the basho - often leads to pics added or exchanged later on an existing page, with sometimes even the same direct picture link for a different pic then.
Sometimes on Nikkei the first one may never be seen again though - thus not post-able, but saving it early adds it to the personal collection.

Check for me means: an automated task loads, saves and rearranges all new articles and pics listed below, removes the copy restrictions (and ads and menus and other nuisance) and replaces all links with those for larger pics like I'll explain here.

What I use for my first overview is a sumo search with a robot news search: http://news.ceek.jp/...=ÁêËÐ ²£¹Ë ÎÏ»Î
- it also gives the Jiji articles, which are else only on a general sports page. Of course much isn't ozumo (unless you limit the search to that), but together with my other daily links I have a nearly complete access to all news.
I use "sumo or yokozuna or rikishi" as search - that increases the amount of crap (keiba and the like) but sometimes gives some interesting additional articles.

What the robot usually won't find are the "shimen" articles of the morning papers.
I therefore also look a the special sumo pages for them.

Sponichi now has a page for the year with new article on top http://www.sponichi.co.jp/sports/tokusyu/sumou/2018/index.html

Sponichi had 3 index pages before a basho and 3 during, one has to change the daily link 6 times per basho period.
The good thing is: they remain. In all other papers the old news go down the page(s) and disappear after a while. Unfortunately Sponichi doesn't create the next page immediately after a basho or after a period ends. Afterwards they are all collected on the right pages though. The latest one looks like this, till the 14th and the new one is not yet there, but the news appear in the robot search:
www.sponichi.co.jp/sports/tokusyu/sumou/2016/haru_before001/index.html
For each basho it is like haru_before001 -3 and then haru001 - 3
Index pages index: http://www.sponichi....umou/index.html

All pages have all pics of the period as thumbs below, linked to the page with the larger pic (and caption).
But the larger ones are hidden in the background and can't be copied directly. The thumbs can, and by editing the link one gets the large one:
in the pic name, like G20160214012042350_thum.jpg the "thum" only has to be replaced with "view".

Asahi has a sumo page: http://www.asahi.com/sports/sumo/
but without the shimen articles: they can be found with the Asahi sumo search
Many articles are only in part freely available, but all pics are in the free versions as well.
Free registration gives you 1 articles to read in full per day. With a 1 month limit to get older articles, that's just barely enough for sumo.

For Asahi to get the larger pics just the "L" has to be deleted from the name of the small pic:
in www.asahi.com/articles/images/AS20160216003810_commL.jpg
AS20160216003810_commL.jpg becomes AS20160216003810_comm.jpg

Mainichi only shows the starting part of their articles now, no free pages per month any more.

Mainichi has 2 sumo pages: one national as part of sports, including amateur sumo: http://mainichi.jp/sumo/

and one ozumo category, where also local articles are listed - 90% of the articles are the same: http://mainichi.jp/ch150910051i/大相撲

Galleries, especially during the basho, appear on the general gallery page: http://mainichi.jp/graphs/
One has to look for 大相撲 at the beginning, or e.g. 横綱土俵入り, to open the right gallery page, but the name for a sumo gallery might be different.
For Mainichi also the small pics can't be copied directly.
In the code or the page information e.g. for the Sumo main page http://mainichi.jp/sumo/
the links of the small pics are something like cdn.mainichi.jp/vol1/2016/02/17/20160217spn00m050020000p/4.jpg
The names are 4.jpg or 41.jpg, 42.jpg etc.
The 4 replaced with 9 gets the bigger picture, if there is one, from the members-only articles. If not, replacing the 4 with 8, 7 or 6 gives the biggest size available.

Usually articles with pics have a camera mark in the article lists below.

Nikkan http://www.nikkansports.com/battle/sumo/news/
now has a sumo-only picture page again http://www.nikkansports.com/battle/sumo/photonews/
They really hide their extra big pics, they won't even show when you click on enlarge in the article.
No picture information is shown in the page information media tab (in Firefox) - and also the code of the pic page is useless. You find them in the code of the articles, e.g.
cache2.nipc.jp/battle/sumo/news/img/sp-koshogiku160217mt-w200_0.jpg for small and
battle/sumo/news/img/sp-koshogiku160217mt-w300_0.jpg for a medium size pic.
You just have to change it to -w500 to get the large pic shown on site.
But that doesn't give the real big one: change to -w1300 to get it.
Not all pics are available in that size, in that case w500 is the biggest.

Hochi http://www.hochi.co.jp/sports/sumo/
Only the thumb shown for the top article can be copied directly:
in www.hochi.co.jp/photo/20160219/20160219-OHT1I50016-N.jpg
the -N.jpg has to be replaced with -L.jpg to get the big pic.
Hochi used to be very short-lived with both articles and pics, but since they introduced members-only articles recently they tend to last a bit longer.

Yomiuri http://www.yomiuri.co.jp/sports/sumo/ is usually on the robot search.
Yomiuri and Hochi are closely related - the Hochi site was Yomiuri Hochi in the past,
thus in pics also here the -N.jpg in the article has to be replaced with -L.jpg to get the big pic - without loading the linked pic page.


Chuspo shimen articles http://www.chunichi.co.jp/chuspo/article/sumo/news/
the pics are small, but they sometimes won't be found anywhere else - and they can be copied directly.

Sanspo http://www.sanspo.co...ts/sumo-n1.html
has also a page with links to the article pics galleries http://www.sanspo.com/sports/sumo/photolists/sumo-p1.html
- during the basho sometimes 20 or more pics in one, but only in the article itself (usually) all pics of a gallery are shown as thumbs.

Daily lists recent pics on the main page http://www.daily.co.jp/sumo/
and those can be copied. In the link like daily.c.yimg.jp/sumo/2016/02/16/Images/a_08810741.jpg
the leading a_ can to be removed to get the bigger pic: daily.c.yimg.jp/sumo/2016/02/16/Images/08810741.jpg

or better the a or b replaced with f to get recently an even larger one: daily.c.yimg.jp/sumo/2016/02/16/Images/f_08810741.jpg
They have often more pics in sumo articles which only appear among the general sports news:
http://www.daily.co.jp/general/ on that page sumo news are also listed at the bottom.

Tospo http://www.tokyo-spo...y/sports/sumou/
has unique pics from time to time, but they won't display well on the forum and need an extra link to the pic page that opens when you click on the pic in the article.

Nikkei has very few news, nearly the same as others, but with a bigger photo of the day:
http://www.nikkei.co...=DF_SEC4_CA____
Still, mostly even bigger versions usually appear on other sites, e.g. the English Japan-times page:
http://www.japantimes.co.jp/sports/sumo/
or among the Kyodo English general pics page: http://kyodonews.net/news/photos
or the 2 Sankei Shimbun general sports extra large picture pages:
http://www.sankei.com/photo/story/newslist/sty_sports-n1.html and
http://www.sankei.com/photo/daily/newslist/dly_sports-n1.html
Sankei has no sumo-only list anymore, only this: http://www.sankei.co...ialarts-n1.html
and most articles won't appear even there, but they do on the robot search.


The daily Jiji general pics page during the basho has usually some pics not on the robot search: http://www.jiji.com/jc/p
They also have loads of sumo galleries (e.g. for Ichinojo, Terunofuji, Hakuho, Kotoshogiku etc.) that are updated during the basho, with bigger versions of the above pics and sometimes more pics. All galleries are here under 大相撲:
http://www.jiji.com/jc/d3

A Yahoo sumo news search now gives a very good overview about rarer sites as well, some periodicals I never knew existed http://news.yahoo.co.jp/search/?p=大相撲+力士
 

Edit: General update, especially for Nikkan

Edited by Akinomaki
  • Like 6

Share this post


Link to post
Share on other sites

Video Unblocker information is updated because YouTube-Unblocker is now blocked by Firefox as potentially malicious.

I'm using Prox Tube as add-on now, another "YouTube Unblock" add-on seems to require registration - or manual proxy setting (for which you don't need an add-on anyway)

Share this post


Link to post
Share on other sites

How does one activate Prox Tube Search?

Do you have a screen shot of the Search box?

Share this post


Link to post
Share on other sites

FIrefox blocks one Unblocker after the other, but new appear at an even faster rate.

YouTube Unblocker Plus is an alternative to ProxTube,

ProxTube has a square icon in the address line list to activate it (without any name shown on mouse-over), gray with a face in this style :-(, activated red and smiling.

Activated a "ProxTube search" button appears at the top of search results.

Edited by Akinomaki

Share this post


Link to post
Share on other sites

My pics overview now covers 99% of what I find, from now on it's just updating and fine tuning.
This means I can here give an overview of how to use the overview.
It is a press overview and only covers press photos, with direct or indirect links to the articles as well.

Shown are the thumbnail pics, directly linked to the largest available pics on site. Only the last group (starting with chunichi) shows the largest pics - they are small anyway - because either the thumbs disappear the next day or there are none.

With the mouse pointer over a thumbnail, the available photo information is shown in a tiny pop-up window, mostly in Japanese. On most pages the Japanese is Unicode encoded and you can only have one encoding on one page: for the Shift-JIS encoded pages you thus couldn't even read the explanation if you can read Japanese - I give a "Shift-Jis" message instead.

I also show the file-size of the full-size pics at the start of the pop-up information, to have an easy way to check which is the better (the bigger) version of the same photo.

If there is an "o" on the right of a thumb, clicking on it opens a page with more information.
This is the article page or just the photo page, which has a link to the article.

The pics overview contains these sites:
The first 2 are English articles, manual means I have to add the (gallery) page manually to my daily download list to include the pics from there.

www.japantimes.co.jp
kyodonews.net (manual)
cdn.mainichi.jp (manual for galleries)
daily.c.yimg.jp
www.nikkansports.com
www.asahi.com

www.nikkei.com
www.sankei.com/photo/ (manual, Sankei XXL galleries)
www.sankei.com/images/news/
www.sanspo.com
www.jiji.com/news/handmade/ (Jiji galleries of individual rikishi)
www.jijiphoto.jp (Jiji database with watermarked pics)
www.jiji.com/news/kiji_photos/ (Jiji article pics)
www.hochi.co.jp
www.yomiuri.co.jp
www.sakigake.jp (Takekaze)
dd.hokkaido-np.co.jp (Kyokutaisei)
chuplus.jp (shift-jis)
www.sponichi.co.jp
www.chunichi.co.jp (shift-jis)
www.tokyo-np.co.jp (shift-jis)
www.iwate-np.co.jp (shift-jis)(Nishikigi)
www.shinmai.co.jp (Mitakeumi)
webun.jp/images/ (manual, no caption on mouse-over)
ZakZak (shift-jis, manual - a day or more late)

Tospo (with the paper's thumb, caption and links)

the overview is sorted by bout for the basho and otherwise by banzuke

Articles from sites without a meaningful sumo page or articles from other sections (esp. Daily) I add from the robot search - what isn't ozumo I delete manually from the overview.

Jiji database pics (sometimes) have an o-link to the daily gallery page, where one can get an explanation and the smaller version (not included in the overview). The even smaller Jiji article pics are shown, giving a link to the article.
Jiji database pics still have the explanation on mouse-over if no gallery page exists.
Jiji database pics may appear new for earlier days or earlier basho.

Edited by Akinomaki
  • Like 2

Share this post


Link to post
Share on other sites
On 19.2.2016 at 14:44, Akinomaki said:

Sponichi has 3 index pages before a basho and 3 during, one has to change the daily link 6 times per basho period.
The good thing is: they remain.

All pages have all pics of the period as thumbs below, linked to the page with the larger pic (and caption).
But the larger ones are hidden in the background and can't be copied directly. The thumbs can, and by editing the link one gets the large one:
in the pic name, like G20160214012042350_thum.jpg the "thum" only has to be replaced with "view".

They didn't remain forever: Sponichi has converted all pages from Shift-Jis to Unicode, removed all old sumo index pages and now makes the contents useless for my standard program controllable download tools - I have to use Free Download Manager by programmed keyboard control to get all pages and pics the way I want - that blocks the PC while the check works.

The good news: pics are are bigger and better and no longer hidden in the background: they can be copied easily - but they have to: embedding the image from URL is blocked by the site.

With more and more download problems for my favorite and fastest tool, old Reget, I'm now expanding my sumo news checking system to work with Uget as well: that slower version could be used by others who want to get the same Japanese news coverage that I have - but I can't provide more than a short explanation and the Visual Basic source code without any guarantee. Who wants to try it and preferably work with me on the program, please send me a PM!

Edit: The old sponichi shift-jis pages are still at: http://cgi.sponichi.co.jp/sports/sumou/index.html

Edited by Akinomaki

Share this post


Link to post
Share on other sites

Time for a little update on tips-and-tools comment=293215

Formats on news and video pages keep changing, fortunately the tools get updated as well.

I'm using now basically 4 tools to get videos:

  • as Firefox Add-on: YouTube Video and Audio Downloader - for individual videos: a quick overview of available qualities to get the best and, if needed, getting only the audio.
  • as Firefox Add-on: Video DownloadHelper - for all other sites. The tool can spot more download links now and for an easy download of not necessarily the best available quality it is very practical. Selections that used to get refused without commercial extension to record videos, now apparently work with ffmpeg in the background.
  • as windows command line tool: youtube-dl - to get the best quality, get some that get spotted but not properly downloaded by DownloadHelper and to get a whole bunch of videos in one action. I'll introduce some practical features here.
  • Some links that give youtube-dl /ffmpeg problems, may still be playable with VLC, to be recorded there.

You have to download youtube-dl manually and place it into a folder of your choice - it needs no integration into the system. You should open a command line window to paste a set of commands into and see the output log. I always have one open on startup and in my taskbar, and I have a text file with all the necessary options and to edit the commands. You need to download the tools for ffmpeg (e.g. also ffprobe) and rtmpdump and put them into the same folder as youtube-dl to get all available videos - if one is missing, you get an error message that prompts you to install that missing tool.

For many sites, you just have to give the address of the page that has the video embedded:

On 23.2.2017 at 19:56, Akinomaki said:

with video: youtube-dl gets it from the page: http://www.asahi.com/articles/ASK2Q3695K2QPXLB004.html

- for me is to paste this into the CL-window: C:\A0doc\youtube-dl http://www.asahi.com/articles/ASK2Q3695K2QPXLB004.html

But if I want to get the new video of the nobori for Nagoya at yahoo, I get the Unsupported URL error: C:\A0doc\youtube-dl https://headlines.yahoo.co.jp/videonews/nnn?a=20170619-00000047-nnn-soci

DownloadHelper lists the video but can't finish the task - copy URL gives e.g. this link

https://gw-yvpub.c.yimg.jp/v1/hls/jTjViJ2L3GkqHYg-/video.m3u8?bw=5000&session_token=wiB6mGBdHpGPMQRrahaAh0PoKm1yaABHBcHrqLI-&https=1

Youtube-dl often only needs a proper playlist without the options and automatically gets the best quality, like this:

C:\A0doc\youtube-dl https://gw-yvpub.c.yimg.jp/v1/hls/jTjViJ2L3GkqHYg-/video.m3u8

If the shortened link won't work, the long one or any link that contains something like an option has to be in quotes.

To check which qualities are available, the -F option is useful:

C:\A0doc\youtube-dl -F https://gw-yvpub.c.yimg.jp/v1/hls/jTjViJ2L3GkqHYg-/video.m3u8

- gives something ending with this:

1500         mp4        854x480    1500k
3000         mp4        1280x720   3000k
5000         mp4        1920x1080  5000k  (best)

Sometimes the best available isn't actually what youtube-dl can get best or what we want - in this case I limit the formats to only mp4 and get the best of that with the option -f mp4: C:\A0doc\youtube-dl -f mp4 https://gw-yvpub.c.yimg.jp/v1/hls/jTjViJ2L3GkqHYg-/video.m3u8

Youtube-dl gets updated often, each week you should give it a "youtube-dl -U" command to make it update to the newest version.

Another very helpful use is to get all videos of a YouTube playlist or all (new) of a channel: during the basho I use this 2 times a day:

C:\A0doc\youtube-dl https://www.youtube.com/channel/UCKd-A2mpjj52t5BPQeYF5Wg/videos --playlist-end 99 -f mp4 -i

This gives the latest 99 videos posted by One&Only - already downloaded ones are skipped if found at the HD location - in mp4 format, and -i is added to avoid the whole download to stop, if a video gives an error.

To have that many videos in one place, you should point youtube-dl to another location: all videos at first end up on your user folder top, which you can't access that easily, the one in the command-line prompt, in my case C:\Users\aki>

So I paste this into the CL before the download command (I always use my own documents folders on any HD):

e:
cd E:\A0doc\yt\one

So much for the introduction of this powerful tool - it sometimes can get videos even from pages the browser won't open.

Now the 2nd subject: the videos neither of the 4 tools can access.

In my case I had the task to record the Sponavi live full basho broadcast: some 140 hours

I found no tool which really produced a video that showed what I could watch on my screen - working with my standard equipment.  After checking available free tools, I ended up with Activepresenter. It allows a satisfying amount of options and remembers them for the next session, also the selected screen part and the folder to save the result. And it produces a format (without having to export it) that can be played and edited with other standard tools, for me VLC and XMedia Recode - audio and video (matroska format) are recorded in separate files though.

I gave up on HD quality, full-screen and 24 frames per second. To have a reasonably smooth video, I recorded it just from the ordinary display part in the Firefox Developer Edition window, at 18 FPS with standard quality for sponavi display - but saved video quality up from 80 to 100%.

It was annoying to find out that by simply checking "record system sound", but not also selecting "none" as audio record option, I had the talk in my room mixed with the kokugikan sound - a reminder that our computers are always watching us and listening to us - whether we are using it consciously in a program for that or not.

Even in Japan on a local PC, with windows 8 I could watch and record Sponavi only by using Firefox Developer Edition. Already ordinary HD made the stream stop too often, let alone the Full HD now "available". Edge on windows10 could display HD without problems, but the recorded screen is blank - direct monitor output.

Another task for screen recording that emerged was to record manga on a tablet or mobile - for this thread namely the Kise story. No sound necessary, so the Telecine app was a satisfying choice (for Android). I used it also to record the whole of Hokuto-no-ken during the few hours it was available to me - 3 days were promised, the app started late the first day and the next day was my departure. Back here I could record the first 3 volumes of Hinomaru-zumo, which were free to read for a short time.

If you have problems to get a manga app or the Official NSK game app from Google Play, you can download it as apk file and install it with an APK Manager - at least those are made available by Google - also useful to install AdBlock for Firefox Android.

At last I'll mention a way to find video links not found by DownloadHelper. There are tools that log your whole network traffic - most won't work well at my place, so I use the Firefox Network Analysis (under Web Developer in the Extra menu). Promising links found there can be tried with youtube-dl. Usually these are just chunks of the whole video, but at least a few seconds at a time can be recorded this way.

Edited by Akinomaki
  • Like 3

Share this post


Link to post
Share on other sites

A way to record the vaughn NHK stream to a hq video while watching it in the browser (Firefox) at the same time by using youtube-dl with ffmpeg:

youtube-dl https://hls-ord-3a.vaughnsoft.net/den/live/live_sherming9/chunklist.m3u8 --referer "https://vaughnlive.tv/sherming9"

One stops the recording with ctrl-c. You need to do that before ffmpeg freezes, which happens after 2, 3 minutes if the browser doesn't keep the connection alive. You have to then rename the .part file to some proper .mp4 name and can start another recording.

If it freezes, ffmpeg doesn't react after one time ctrl-c - 2 times interrupts youtube-dl as well and the recording gets corrupted,
Recording may lead to more stops in the live view in Firefox, buffering is all that's shown then, but the recording should be OK. One command-line set example:
C:\***\youtube-dl "https://hls-ord-3a.vaughnsoft.net/den/live/live_sherming9/chunklist.m3u8" --referer "https://vaughnlive.tv/sherming9"
rename chunklist-chunklist.mp4.part Aki17d01a.mp4

Vaughn changed server numbers each day, hls-ord-1a., hls-ord-2a. or hls-ord-3a.- but the playlists disappeared from the Firefox network analysis now. 1 and 3 seem to work, 3 with better quality.

The recoding may stop properly by itself, to be sure to have it continue, this set to paste into the cmd-line window is better:

C:\***\youtube-dl "https://hls-ord-3a.vaughnsoft.net/den/live/live_sherming9/chunklist.m3u8" --referer "https://vaughnlive.tv/sherming9"
rename chunklist-chunklist.mp4 Aki17d01a.mp4
rename chunklist-chunklist.mp4.part Aki17d01a.mp4
C:\***\youtube-dl "https://hls-ord-3a.vaughnsoft.net/den/live/live_sherming9/chunklist.m3u8" --referer "https://vaughnlive.tv/sherming9"
rename chunklist-chunklist.mp4 Aki17d01b.mp4
rename chunklist-chunklist.mp4.part Aki17d01b.mp4
...

 

Without rename, a properly ended recording causes youtube-dl to exit at once with "already downloaded", and an interrupted .part recording gets deleted by the new recording. Careful to change the next name to rename to in the lines you paste, else the old recording is gone as well.

Edited by Akinomaki

Share this post


Link to post
Share on other sites

With all the scandals going on and now the riji elections, sumo is in the wide (news) shows every day. These variety news shows are for news programs what the tabloids are for proper newspapers, but they are a bit more reliable than gossip - and they often show a nice video report. The guests are the real nuisance there (but the casters as well), fortunately they also invite usually 1 or 2 sumo experts or sumo world related people: lately Daishi and Ishinriki appear somewhere nearly every day, Kyokushuzan was everywhere as he was on a visit.

Most wide shows can be found with a YT search, those on Dailymotion with the Google video search, some are regularly on 9tsu. Videos from these 3 can be downloaded with youtube-dl - and especially 9tsu shouldn't be visited directly: it's infested with the worst adware pop-ups imaginable (I download daily the new video index pages and extract only the contents lists). The sumo content can be found quickly in the downloaded videos with the super fast-forward of VLC (shift-arrow-right or faster with alt or ctrl).

I list here some of the major wide shows, which have their reports also mentioned in the sumo news - sometimes because of NSK protests about the reports or now a fake agreement to use NSK material on Wide!Scramble yesterday.

I post the YT search link, sorted by upload date. Useful filter settings are also Duration: long to only show the whole programs and Features: HD to only show the better quality videos - unfortunately the small screen ones can't be removed with this setting, one look at the video is often necessary to eliminate them. You can use the content of the YT search box, add the date like 180130 for today and paste it into the Google video search to search also for Dailymotion videos.

The full shows usually get erased after a short while, to keep the channels alive a bit longer - recently the channels last about a week, but get replaced at once.

Weekdays:

Miyane-ya  13:55-15:55 (Yomiuri TV = NiTele/NTV) https://www.youtube.com/results?q=ミヤネ屋&sp=CAISAhAB

Sukkiri 8-10:25 (NiTele) https://www.youtube.com/results?search_query=スッキリ&sp=CAISBBABGAI%3D

Viking 11:55-13:45  (Fuji TV) https://www.youtube.com/results?q=バイキング&sp=CAI%3D

Goody 13:45-15:50  (Fuji TV) https://www.youtube.com/results?sp=CAISAhAB&q=直撃LIVEグッディ

Tokudane 8-9:50 (Fuji TV) https://www.youtube.com/results?sp=CAISAhgC&search_query=とくダネ!

NonStop 9:50-11:25 (Fuji TV) https://www.youtube.com/results?sp=CAISAhgC&search_query=ノンストップ

GOGO Smile13:55-15:49  (TBS) https://www.youtube.com/results?q=ゴゴスマ&sp=CAISAhAB

HiruObi 10:25-13:55 (TBS) https://www.youtube.com/results?q=ひるおび&sp=CAISAhAB

Vivit 8-9:55 (TBS) https://www.youtube.com/results?search_query=ビビット&sp=CAI%3D

MorningShow 8-9:55 (Asahi TV) https://www.youtube.com/results?sp=CAISAhgC&search_query=羽鳥慎一モーニングショー

Wide!Scramble 10:25-13:40 (Asahi TV): https://www.youtube.com/results?search_query=ワイド+スクランブル&sp=CAISAhgC

not often on YT and more of a proper news program:

Nsta 15:49 -19:00  (TBS) https://www.youtube.com/results?sp=CAISAhAB&q=Nスタ

AsaChan (TBS) https://www.youtube.com/results?search_query=あさチャン&sp=CAISAhgC

Proper NHK news:

Shibu5ji https://www.youtube.com/results?sp=CAI%3D&search_query=シブ5時

Ohayou Nippon  https://www.youtube.com/results?sp=CAI%3D&search_query=おはよう日本

 

Weekend

Wide-na-Show (FujiTV) https://www.youtube.com/results?search_query=ワイドナショー&sp=CAISAhgC

Bankisha (NTV) https://www.youtube.com/results?sp=CAI%3D&search_query=バンキシャ - full to be found on 9tsu

Takeshi's NewsCaster (TBS) https://www.youtube.com/results?search_query=情報7daysニュースキャスター&sp=CAISAhgC

Sunday Japon (TBS) https://www.youtube.com/results?search_query=サンデー・ジャポン&sp=CAI%3D

Akko ni o-makase ! (TBS) https://www.youtube.com/results?search_query=アッコにおまかせ!&sp=CAISBBABGAI%3D

Edited by Akinomaki
  • Thanks 3

Share this post


Link to post
Share on other sites

Vivaldi browser now offers a vertical read mode for asian languages.

https://vivaldi.com/blog/vivaldi-1-14-three-years-of-continuous-innovation/#pk_campaign=release-notes&pk_kwd=1.14

I tried it with https://jp.vice.com and it works. Well, i still can't read japanese, but it is possible to switch from horizontal to vertical.

Share this post


Link to post
Share on other sites

This has probably been answered a thousand times but I’m unable to find YouTube channel with Juryo coverage. Can somebody share some links or tag me on a post with them? Thanks in advance. 

Share this post


Link to post
Share on other sites
Posted (edited)

This is one of the best tips ever to get free sumo coverage

On 09/07/2018 at 07:50, inhashi said:

FujiTV  https://fujitv.live/fujitv
It's free, realtime,  can start watching at around 13:00 JST (get most of Makushita joi on) 
The catch is registration is required with a valid e-mail address.

I use the following link https://temp-mail.org/en/ to obtain a temporary e-mail address.  Once confirming verification via the temp e-mail one can start watching channel 017 NHK BS1 from 13:00 - 15:18  then switch to channel 001 NHK from 15:18 - the end of the day.  The stream is solid with no to very little interruption if any.  
Once the trial viewing runs out, it's just a matter of obtaining a different temp email at the link above.  The old email address can be deleted or will expire after one day.

On 12/07/2018 at 05:44, Asojima said:

live access to over 50 Japanese channels, and has about 2 weeks of VOD available for each channel.  For those of us who tend to sleep through the live matches, it offers a full replay of Makushita thru Makuuchi upon wakeup.  It should also offer replays /access to program alerts made on the SF between bashos.

Select the day of the finished program from the selected channel - mouse scrolling works best to get to earlier days not listed yet.

Click on a finished program in the list to display the video for it (may take a few seconds). A bit more than 5 minutes after the end of a program a VOD (replay) is available for it. By moving the mouse cursor below, you can show the replay time position and change it. While the position is displayed, arrow keys etc. do erratic  jumps back and forth: right/left about 10s, pgdwn/-up 1 min.

You can watch it in a browser or install the app:

If you use an attached (TV) screen on e.g. a notebook, the full screen view of the app won't show on it, only on the notebook screen itself - the TV screen just freezes - even so if you just put the app itself into full screen.

You can download the VOD to the p2ps_video_cache folder the app creates - it creates that on install without asking, on some attached drive, if there is one. Download with the icon in the upper right corner of the video or with the <Rec> button next to the title in the program list, slightly reduced quality of about 500MB for an hour of video, with a very slow download. The download list, with the progress of the download and the downloaded videos to play, is on the right of the top home button of the app.

After logout, this downloaded video stops after a minute just like the live broadcast, but it remains for your next trial version login. You can only play these videos with the app, from the download list, but - Inhashi found the way to download a proper .flv video file with the browser version of FujiTV.live - I had avoided that one because the interface is awful compared to the app. FujiTV has allowed the download option of Adobe flash player:

  • start the TV program from the schedule -  VODs are marked with 再生 next to the name
  • right click on the video window - preview size is OK in Firefox, other browsers may need full screen - and select "download" from the Flash player pop-up window

 

How long you can do this remains to be seen.

I will put more program announcements into the thread from now on, with channel, day, start time and title, so you can easily find the program, and the start time of the interesting part for longer programs.

On 07/08/2018 at 19:24, Akinomaki said:

Funny that FujiTV.live has now lost all FujiTV channels (BS and CS included) - the channel numbers I gave in other posts have changed with that.

Next gone are the NiTere and TBS channels

Edit: all channels gone now

Edited by Akinomaki
  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites

I made an update of the post about the papers with pics - especially for Nikkan, who abandoned their --nsogp large pics, but -w1300 is even larger, and available for all old pics as well.

On 2/19/2016 at 14:44, Akinomaki said:

Nikkan http://www.nikkansports.com/battle/sumo/news/

hide their extra big pics,
No picture information is shown in the page information media tab (in Firefox) - and also the code of the pic page is useless. You find them in the code of the articles, e.g.
cache2.nipc.jp/battle/sumo/news/img/sp-koshogiku160217mt-w200_0.jpg for small and
battle/sumo/news/img/sp-koshogiku160217mt-w300_0.jpg for a medium size pic.
change it to -w500 to get the large pic shown on site.
But that doesn't give the real big one: change to -w1300 to get it.
Not all pics are available in that size, in that case w500 is the biggest.

 

I'd also like to mention again my personal tool to get (and filter) news pages and pics and to produce the pics overview. I made an update to have it now work with the freely available aria2 for download of the pages, pics and files. If there is interest, send me a PM: I can share the visual basic code and configuration files and give information on how to use it (meaning I will then start to make a help file).

  • Thanks 1

Share this post


Link to post
Share on other sites

Wide Show update. The programs are much longer available on MioMio - the last month is shown in this search, older ones can be found by adding to the search link URL the date, in the format +yy+mm+dd

You have just one pretty good quality video for each day, virtually no NHK (but that also less and less on YT), and shows that can't be found on YT,

esp. Nsta 15:49 -19:00  (TBS) http://search.miomio.tv/?keyword=N+スタ

and AsaChan (5:25-8:00 TBS) http://search.miomio.tv/?keyword=あさチャン

If HD is not available on YT, MioMio is always the best - for sumo minor ones in italics/small

Weekday shows

GOGO Smile13:55-15:49  (TBS) http://search.miomio.tv/?keyword=ゴゴスマ

Goody 13:45-15:50  (Fuji TV) http://search.miomio.tv/?keyword=グッディ

HiruObi 10:25-13:55 (TBS) http://search.miomio.tv/?keyword=ひるおび

Miyane-ya  13:55-15:55 (Yomiuri TV = NiTele/NTV) http://search.miomio.tv/?keyword=ミヤネ屋

MorningShow 8-9:55 (Asahi TV) http://search.miomio.tv/?keyword=羽鳥慎一モーニングショー

NonStop 9:50-11:25 (Fuji TV) http://search.miomio.tv/?keyword=ノンストップ

Sukkiri 8-10:25 (NiTele) http://search.miomio.tv/?keyword=スッキリ

Tokudane 8-9:50 (Fuji TV) http://search.miomio.tv/?keyword=とくダネ

Viking 11:55-13:45  (Fuji TV) http://search.miomio.tv/?keyword=バイキング

Vivit 8-9:55 (TBS) http://search.miomio.tv/?keyword=ビビット

Wide!Scramble 10:25-13:40 (Asahi TV) http://search.miomio.tv/?keyword=ワイド+スクランブル

The interesting shows vary from day to day, usually they are mentioned in the sumo news (ceek search)

Weekend

Wide-na-Show (FujiTV) http://search.miomio.tv/?keyword=ワイドナショー

Takeshi's NewsCaster (TBS) http://search.miomio.tv/?keyword=ニュースキャスター

Bankisha (NTV) http://search.miomio.tv/?keyword=バンキシャ
 

Sunday Japon (TBS) http://search.miomio.tv/?keyword=サンデー・ジャポン

Akko ni o-makase ! (TBS) http://search.miomio.tv/?keyword=アッコにおまかせ

This is mainly suited for download of the show with youtube-dl, which is pretty fast now, sometimes faster than from YT, but for watching directly I prefer YT. MioMio is inaccessible from time to time.

The display "Video transcoding" indicates that the video is not (yet) available (or never has been), a display of "MioMio" that the video is gone.

Contents on MioMio may be available for about 10 months  - some like most anime are not at all, even though the page is available and in the search result.

On Pandora maybe for longer - to look for a likely a better quality video, copy the part after keyword= to the Pandora search, e.g. for Bankisha http://www.pandora.tv/search/searchResult?query=バンキシャ

Pandora takes quite some time though to download with youtube-dl.

The long availability makes it practical now, to, after I checked earlier days, post an overview of the wide shows with interesting sumo contents - they feature it regularly now

Edited by Akinomaki

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now