New user's registration have been closed due to high spamming and low trafic on this forum. Please contact forum admins directly if you need an account. Thanks !

Automatically download torrent files using rsstorrent

Discuss development on Bubba
Post Reply
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Automatically download torrent files using rsstorrent

Post by AzP »

Hi guys

I was lacking software that could automatically download torrent files based on a regular expression (or something like it) to my B3. The torrent download is great, but it kind of defeats the purpose for me, since I still had to log in to my torrentsite, download the torrent, and manually put it on the torrent-folder on the B3.

Therefore I wrote a small program in python that monitors an RSS-feed and downloads all torrent files that match the files you have stated that you would like to download. It works for me on the B3, but you have to start it using screen for now. The init-script does not work yet. But I'm working on it.

The config file is pretty self explanatory, but I have hard coded the script to only work with torrentbytes.net for the time being. This is also something that I need to take care of, hopefully during this weekend.

If you have an account at torrentbytes, there is a thread on the forum telling you have to get the RSS-feed from it (get the one with your hashed password and username in it), and you should then be ready to add it to the rsstorrent config-file and then be ready to roll!

Hope somebody else finds this useful. If you do, please throw me a few comments about what to improve, etc. The things that has been taking most of my development time is actually deciding on how the script should behave, and how the user should be able to control/configure/use it.

If you don't find it useful, please tell me why as well. Perhaps we can find space and time for adding some more features. But my idea is to keep it simple and have it do one thing pretty well.

I created a repository at gitHub, so either you clone it to your bubba (to easily get fixes and changes) using git, or you can just download the files manually from the page.
https://github.com/AzP/rsstorrent

If you want to run it in a screen session, follow the tutorial at [url=http://wiki.excito.org/wiki/index.php/T ... k_Tutorial[The Excito wiki[/url]. But the gist of it:

Code: Select all

1. Log-in using SSH
2. Run 'screen -S rsstorrent' (use whatever name you want for the session)
3. Start the script (./rsstorrent.py or python rsstorrent.py)
4. Press ctrl+A ctrl+D
Now the session should be running in the background. Check it is by entering

Code: Select all

screen -ls
That should show the name of your session. Reconnect to it the next time you SSH into the box by running

Code: Select all

screen -r rsstorrent
(I posted this in the Howto section first, but figured this is more correct. Hope the double-post is ok)
johannes
Posts: 1470
Joined: 31 Dec 2006, 07:12
Location: Sweden
Contact:

Re: Automatically download torrent files using rsstorrent

Post by johannes »

Nice! If you don't mind, a wiki page with this howto would be appreciated! (Or if you think it's too much work, we can copy your guide to the wiki)?

http://wiki.excito.org/wiki/index.php/T ... nd_How-tos

Get an account here:

http://wiki.excito.org/wiki/index.php/S ... estAccount

Thanks!
/Johannes (Excito co-founder a long time ago, but now I'm just Johannes)
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

Jag slänger gärna upp en wiki när jag har tid! Kanske ikväll, kanske imorgon ;)
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

There you go, I initially copied this guide to the wiki, and did some minor fixes, and then I can update it more later as the script improves.
johannes
Posts: 1470
Joined: 31 Dec 2006, 07:12
Location: Sweden
Contact:

Re: Automatically download torrent files using rsstorrent

Post by johannes »

Wonderful, thanks! :)
/Johannes (Excito co-founder a long time ago, but now I'm just Johannes)
Tompa
Posts: 81
Joined: 03 Feb 2007, 12:50

Re: Automatically download torrent files using rsstorrent

Post by Tompa »

Hi,

Don't know if you've come across the tool FlexGet but if not I can recommend it. It handles all sorts of automatic downloads including filters using regexp.

http://flexget.com/

/Tompa
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

Oh, if only I'd found that before I started rolling my own ;)
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

Big updates on the code this last week!

* Rewrote config handling to use ConfigParser
* Improved logging (not outputs to ~/.rsstorrent/rsstorrent.log as default)
* Fixed crash bugs caused by bad handling of non-resolved connection

I still haven't done anything about the init-scripts, I would love some help from someone regarding those. I tried writing some init-stuff, but kind-of failed on Gentoo. I'd also like to get it working on debian since that's what the B3 is based on.

Another solution, which would greatly simplify my rsstorrent-script, is to use a cron-job to launch it at specified intervals. That would mean that the code wouldn't have to handle being a daemon and run forever, but at the same time I wanted the code to be "self-contained" after install. Just install, edit your config-file, and then you're ready to go.
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

Another "release" has been made as of yesterday. I've updated the script to run in proper Daemon mode, and use pid-files to keep track of itself. I also wrote the init script for the B3, but I forgot to commit it and push it to github, so sadly you can't get a hold of it right now.
But this means that rsstorrent can now be added to automatically start at boot! It's definitely a big improvement in my point of view since I restart my B3 every now and then, and it's a pain to have to start a screen and run it inside that.

I'd really like some feedback from the Excito team regarding the init script, it's the first one I've written (except the similar one for Gentoo) and I believe there are improvements to be made. It still has one issue, and that is that the pid-file can't be used to stop the daemon right now. Yes, that means that you are unable to use

Code: Select all

/etc/init.d/rsstorrent stop
or

Code: Select all

service rsstorrent stop
which is of course a shame. But when that has been fixed I think it's nearing perfection ;)
  • Features planned ahead:
  • Re-read (while running) the config file after it has been changed and supply log output regarding the new search patterns.
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

The correct init script has now been pushed to GitHub, so just get it from there and try it out!
AzP
Posts: 60
Joined: 04 Mar 2011, 08:09

Re: Automatically download torrent files using rsstorrent

Post by AzP »

Recent updates:

* Support for monitoring several sites is now working
* It is possible to poll these sites at different intervals (specified in minutes)
* Improved error handling
* Daemon mode might be a bit dodgy, it hasn't been tested with new support for several sites. Especially regarding shutting the daemon down
* More options at start (desregard cache, clean cache, dry run (don't download anything), etc)
* Improved logging

and much more (like code cleanup, etc)! Check it out!

https://github.com/AzP/rsstorrent
Post Reply