TorrentFlux.com Hey, OppTupacShakur, you have 11 messages, 0 are new.
05-07-2007, 12:38:52
Home Help Search Profile Logout donate chat
News:


+  TorrentFlux Forums
|-+  Code Hacks and Tools
| |-+  TorrentFlux 2.1 Hacks
| | |-+  [REQ] Autmatic RSS-Download
« previous next »
Pages: 1 2 [All] Go Down Reply Notify of replies Send this topic Print
Author Topic: [REQ] Autmatic RSS-Download  (Read 1099 times)
somlioy
Newbie
*
Offline Offline

Posts: 7



View Profile Personal Message (Offline)
« on: 02-12-2007, 10:27:57 » Reply with quote

I'd love if someone could make an script for Automaticly download from RSS TF Love
Report to moderator   Logged
jav
Newbie
*
Offline Offline

Posts: 15



View Profile Personal Message (Offline)
« Reply #1 on: 02-24-2007, 04:36:10 » Reply with quote

Me too!!  Grin
Report to moderator   Logged

Gentoo Linux
Lighttpd
PostgreeSQL
Torrentflux 2.1
total pwnage
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #2 on: 03-25-2007, 01:33:35 » Reply with quote

I hacked the 2 year old rss-torrentflux perl script yesterday and it seems to be working fine. have sent a patch to the maintainer yesterday but no response.

So if anyone is interested. drop a line here
Report to moderator   Logged
jav
Newbie
*
Offline Offline

Posts: 15



View Profile Personal Message (Offline)
« Reply #3 on: 03-25-2007, 04:44:41 » Reply with quote

I'm intrested, can you give me a link?
Report to moderator   Logged

Gentoo Linux
Lighttpd
PostgreeSQL
Torrentflux 2.1
total pwnage
emipa606
Newbie
*
Offline Offline

Posts: 2



View Profile Personal Message (Offline)
« Reply #4 on: 03-25-2007, 22:28:18 » Reply with quote

VERY interested
Can you post the code?
Report to moderator   Logged
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #5 on: 03-29-2007, 11:38:27 » Reply with quote

I sent a patch to the maintainer but he has not responded to me. So.. I've no idea if he just hasn't seen it or it's not good.

Took me a while to un-obfuscate it. (I'm  a perl newbie who just got a book and started reading)

=================== PATCH==================

--- /tmp/rss-torrentflux0.9c.pl 2007-03-24 17:23:52.000000000 +0800
+++ rss-torrentflux0.9c.pl.mine 2007-03-24 17:27:48.000000000 +0800
@@ -25,7 +25,7 @@
 use strict;

 use vars qw/$torrentflux_url $torrentflux_user
-    $torrentflux_password @rssfeeds @regexp @seentorrents/;
+    $torrentflux_password @rssfeeds @regexp $downloc @seentorrents/ ;

 ##########################################################################
 #################### Options you really should set Smiley ####################
@@ -51,6 +51,7 @@
 #           '^24');                  # Matches every Title beginning with 24
 @regexp = ();

+$downloc = '~/tf_rss_dl/';
 ##########################################################################
 ############# You shouldn't need to edit anything below here #############
 ##########################################################################
@@ -68,6 +69,8 @@
                      'n|nostart' => \$nostart,
                      ) and !$help;

+# Enable this to view the coming/going of LWP
+#use LWP::Debug qw(+);
 use LWP::Simple;
 use XML::RSS;
 use WWW::Mechanize;
@@ -75,8 +78,12 @@
 my $mech = WWW::Mechanize->new();
 my $rss = new XML::RSS;

-my $response = $mech->get($torrentflux_url);
-die ($response->status_line) unless ($response->is_success());
+sub connect_home() {
+       my $response = $mech->get($torrentflux_url);
+       die ($response->status_line) unless ($response->is_success());
+       }
+
+connect_home();

 for ($mech->forms()) {
     if ($_->attr('name') eq 'theForm') {
@@ -90,7 +97,7 @@
 die ('Could possibly not login to TorrentFlux!') unless ($mech->form_name('form_url'));

 # Get list of downloaded torrents
-if (open(TORRENTS, '<', $ENV{HOME}.'/.seentorrents')) {
+if (open(TORRENTS, '<', $downloc.'/.seentorrents')) {
     push(@seentorrents,$_) while(<TORRENTS>);
     chomp(@seentorrents);
     close(TORRENTS);
@@ -107,8 +114,12 @@
    ITEM: foreach my $item (@{$rss->{'items'}}) {
        my $title = $item->{'title'};
        my $url = $item->{'link'};
+          my $desc = $item->{'description'};
        my $filename = $url;
-       $filename =~ s!^.*/!!;
+
+#         print "title --> $title\n url--> $url\n desc-->$desc\n filename-->$filename\n\n";
+
+          $filename =~ s!^.*/!!;
        for (@regexp) {
           if ($title =~ /$_/i) {
               # Check if download has already started
@@ -116,13 +127,28 @@
                   next ITEM if ($filename eq $_);
               }
               print "Found new Match: $title\n" if ($verbose);
-              # Add URL to Torrentflux
-              $mech->submit_form(form_name=>'form_url',
-                                 fields => {url_upload => $url,});
+
+                  $title =~ s/ /_/g;  # Replace empty spaces with underscores (Globally)
+                  $title =~ s/\(/_/g;  # Replace open bracket with underscores (Globally)
+                  $title =~ s/\)/_/g;  # Replace closing bracket with underscores (Globally)
+                  $title =~ s/-/_/g;  # Replace hypens with underscores (Globally)
+                  $title =~ s/__/_/g;  # Replace double hypens with underscores (Globally)
+                  my $lfile = $downloc.$title.'.torrent';
+                  my $status = getstore($url,$lfile);
+           if ($verbose) {
+                       print("Successfully Downloaded $title\n") if is_success($status);
+                  }
+
+                  # Add URL to Torrentflux
+              $mech->submit_form(form_name=>'form_file',
+                                 fields => {upload_file => $lfile,});
               if ($mech->content() !~ /ERROR/) {
                   # Start Download
                   if (!$nostart) {
-                      my $response = $mech->get($torrentflux_url.'index.php?torrent='.$filename                                                                                                     );
+                      my $response = $mech->get($torrentflux_url.'startpop.php?torrent='.$title                                                                                                     .'.torrent');
+                          $mech->submit();
+                          #print "$response->status_line\n";
+                          #print "$response->is_success()\n";
                       die ($response->status_line) unless ($response->is_success());
                   }
                   push(@seentorrents, $filename);
@@ -130,12 +156,14 @@
               } else {
                   print " Failed to add torrent to TorrentFlux. Will try again next time.\n" if                                                                                                      ($verbose);
               }
+
+                  connect_home();
           }
        }
    }
  }

 # Update ~/.seentorrents
-open(TORRENTS, '>',$ENV{HOME}.'/.seentorrents') or die 'Could not write to ~/.seentorrents';
+open(TORRENTS, '>',$downloc.'/.seentorrents') or die "Could not write to $downloc/.seentorrents                                                                                                     ";
 print TORRENTS join("\n",@seentorrents);
 close(TORRENTS);
=========== END PATCH===========

I've only tested it on eztvefnet.org's RSS Feeds. So, not sure if it will work at other places.

Oh yeah. the 5 lines of regex can be collapsed into 1 line, but I'm lazy to go look for that (other) patch I did. Majority of the regexes are needed because when you upload to TF, the file gets stripped of any offending '()' and "-" etc. So it gave stumped me for a while on why it would successfully upload into TF but fails to start it. (Cos the uploaded name and the RSS's name are different!!)

Oh.. one other thing, it doesn't remove the DL'ed files, you might need to add code if you want to automatically delete it after getting it into TF.

Report any fixes and stuff.

Been working fine since Last Sat.
Report to moderator   Logged
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #6 on: 04-03-2007, 09:36:12 » Reply with quote

C'mon.. Don't tell me no one tried it and has nothing good/bad to say???

Report to moderator   Logged
v-v
Newbie
*
Offline Offline

Posts: 2



View Profile Personal Message (Offline)
« Reply #7 on: 04-05-2007, 08:27:54 » Reply with quote

Hello,
I've tried TorrentFlux for the first time today and I'm very interested in an rss feed automation but I have some problems patching the rss-torrentflux.pl
Code:
Hunk #1 succeeded at 25 with fuzz 1.
Hunk #3 succeeded at 69 with fuzz 2.
Hunk #6 succeeded at 114 with fuzz 2.
Hunk #7 FAILED at 127.
Hunk #8 FAILED at 156.
2 out of 8 hunks FAILED

Could you please upload or paste the whole rss-torrentflux.pl file you modify?

Thank you very much

Vedran Smiley
Report to moderator   Logged
heion
Newbie
*
Offline Offline

TF Base: Linux 2.x
Posts: 4



View Profile Personal Message (Offline)
« Reply #8 on: 04-06-2007, 05:52:38 » Reply with quote

I get the same hunks failed when trying to patch rss-torrentflux 0.9c as well.

Any clues yet as to why?

Regards,
Heion

Code:
Hunk #1 succeeded at 25 with fuzz 1.
Hunk #3 succeeded at 69 with fuzz 2.
Hunk #6 succeeded at 114 with fuzz 2.
Hunk #7 FAILED at 127.
Hunk #8 FAILED at 156.
2 out of 8 hunks FAILED

EDIT: I edited the failed hunks manually. It's close to working now, I think.
« Last Edit: 04-06-2007, 10:59:22 by heion » Report to moderator   Logged
heion
Newbie
*
Offline Offline

TF Base: Linux 2.x
Posts: 4



View Profile Personal Message (Offline)
« Reply #9 on: 04-06-2007, 11:02:56 » Reply with quote

The perl script now successfully downloads torrent files, but fails to add them to torrentflux.

Code:
Use of uninitialized value in substitution (s///) at rss-torrentflux0.9c.pl line 122.
Line 122 looks like this:
Code:
$filename =~ s!^.*/!!;

Any tips as to what could be wrong?


EDIT: Fixed it by changing the line to this:
Code:
$filename =~ s!^-*!! if defined($filename);
« Last Edit: 04-06-2007, 22:40:55 by heion » Report to moderator   Logged
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #10 on: 04-06-2007, 23:14:51 » Reply with quote

The perl script now successfully downloads torrent files, but fails to add them to torrentflux.

Code:
Use of uninitialized value in substitution (s///) at rss-torrentflux0.9c.pl line 122.
Line 122 looks like this:
Code:
$filename =~ s!^.*/!!;

Any tips as to what could be wrong?


EDIT: Fixed it by changing the line to this:
Code:
$filename =~ s!^-*!! if defined($filename);

Firstly, this line should _not_ even be there. The patch removes this line. (indicated by the "-" at the beginning)
Report to moderator   Logged
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #11 on: 04-06-2007, 23:17:33 » Reply with quote

Could you please upload or paste the whole rss-torrentflux.pl file you modify?

Here are the BITS after the comment that you shouldn't edit anything below this line.

---------------->8------------------->8---------------------

use Getopt::Long;

my ($verbose, $help, $nostart);

die "Usage: $0 [-n] [-v] [-h]
\t-n\tDon't start download, just add URL to TorrentFlux
\t-v\tBe verbose
\t-vv\tBe Very verbose
\t-h\tDisplay this help screen"
    unless GetOptions('v|verbose' => \$verbose,
              'h|help' => \$help,
              'n|nostart' => \$nostart,
              ) and !$help;

# Enable this to view the coming/going of LWP
#use LWP::Debug qw(+);
use LWP::Simple;
use XML::RSS;
use WWW::Mechanize;

my $mech = WWW::Mechanize->new();
my $rss = new XML::RSS;

sub connect_home() {
    my $response = $mech->get($torrentflux_url);
    die ($response->status_line) unless ($response->is_success());
    }

connect_home();

for ($mech->forms()) {
    if ($_->attr('name') eq 'theForm') {
    $mech->form_name('theForm');
    $mech->set_fields(username => $torrentflux_user,
              iamhim   => $torrentflux_password);
    my $response = $mech->click();
    die ($response->status_line) unless ($response->is_success());
    }
}
die ('Could possibly not login to TorrentFlux!') unless ($mech->form_name('form_url'));

# Get list of downloaded torrents
if (open(TORRENTS, '<', $downloc.'/.seentorrents')) {
    push(@seentorrents,$_) while(<TORRENTS>);
    chomp(@seentorrents);
    close(TORRENTS);
}

 FEED: for my $feed (@rssfeeds) {
     my $feedcontent = get($feed);
     if (!$feedcontent) {
     warn "Could not get RSS-Feed $feed";
     next FEED;
     }
     $rss->parse($feedcontent);

   ITEM: foreach my $item (@{$rss->{'items'}}) {
       my $title = $item->{'title'};
       my $url = $item->{'link'};
       my $desc = $item->{'description'};
       my $filename = $url;

#      print "title --> $title\n url--> $url\n desc-->$desc\n filename-->$filename\n\n";

       $filename =~ s!^.*/!!;
       for (@regexp) {
       if ($title =~ /$_/i) {
           # Check if download has already started
           for (@seentorrents) {
           next ITEM if ($filename eq $_);
           }
           print "Found new Match: $title\n" if ($verbose);

#          $title =~ s/ /_/g;  # Replace empty spaces with underscores (Globally)
#          $title =~ s/\(/_/g;  # Replace open bracket with underscores (Globally)
#          $title =~ s/\)/_/g;  # Replace closing bracket with underscores (Globally)
#          $title =~ s/-/_/g;  # Replace hypens with underscores (Globally)
#          $title =~ s/__/_/g;  # Replace double hypens with underscores (Globally)
           $title =~ s/[-()\s]+/_/g;
           my $lfile = $downloc.$title.'.torrent';
           my $status = getstore($url,$lfile);
           if ($verbose) {
            print("Successfully Downloaded $title\n") if is_success($status);
           }

           # Add URL to Torrentflux
           $mech->submit_form(form_name=>'form_file',
                  fields => {upload_file => $lfile,});
           if ($mech->content() !~ /ERROR/) {
           # Start Download
           if (!$nostart) {
               my $response = $mech->get($torrentflux_url.'startpop.php?torrent='.$title.'.torrent');
               $mech->submit();
               #print "$response->status_line\n";
               #print "$response->is_success()\n";
               die ($response->status_line) unless ($response->is_success());
           }
           push(@seentorrents, $filename);
           print " Successfully added torrent to TorrentFlux\n" if ($verbose);
           } else {
           print " Failed to add torrent to TorrentFlux. Will try again next time.\n" if ($verbose);
           }

           connect_home();
       }
       }
   }
 }

# Update ~/.seentorrents
open(TORRENTS, '>',$downloc.'/.seentorrents') or die "Could not write to $downloc/.seentorrents";
print TORRENTS join("\n",@seentorrents);
close(TORRENTS);

------------>8----------------->8--------------------

Well at least some of you are responding... :-)

it's really been working great for me.
Report to moderator   Logged
v-v
Newbie
*
Offline Offline

Posts: 2



View Profile Personal Message (Offline)
« Reply #12 on: 04-07-2007, 05:55:36 » Reply with quote

Nice, It's working very well Smiley

I tested it on torrentleech and works just fine Smiley

Thank you very much for this great thing Smiley

Vedran
Report to moderator   Logged
myjunk
Newbie
*
Offline Offline

Posts: 6



View Profile Email Personal Message (Offline)
« Reply #13 on: 04-07-2007, 06:22:55 » Reply with quote

Nice, It's working very well Smiley

I tested it on torrentleech and works just fine Smiley

Thank you very much for this great thing Smiley

Vedran


Torrentleech you say.. hmm.. I must take a look at the rss feeds and see how it pans out. I know (at least I think) that it won't work on mininova feeds due to the change in the location between the rss and the actual link.

I have an idea how to get that done, but there's not a lot of need since my needs has been fulfilled.

You're welcomed. I've sent the patch to the original author, but he has not responded (and I doubt he will)
Report to moderator   Logged
heion
Newbie
*
Offline Offline

TF Base: Linux 2.x
Posts: 4



View Profile Personal Message (Offline)
« Reply #14 on: 04-13-2007, 06:21:55 » Reply with quote

Thanks for patching things up, it's working just fine now.  Smiley
I've started liking perl after working with this script.
Report to moderator   Logged
joewen7
Newbie
*
Offline Offline

Posts: 6



View Profile Personal Message (Offline)
« Reply #15 on: 04-29-2007, 23:10:47 » Reply with quote

Hi, I'm not really sure how to use this perl script in the first place, as there doesn't seem to be any other place which documents the usage of this script anymore. Huh?

Is this script suppose to the run manually? like "$ perl rss-torrentflux.pl" each time you want to run it?

I hope somebody can elaborate the usage of this script to me. Thanks.
Report to moderator   Logged
heion
Newbie
*
Offline Offline

TF Base: Linux 2.x
Posts: 4



View Profile Personal Message (Offline)
« Reply #16 on: 04-30-2007, 07:54:06 » Reply with quote

Yes, it's supposed to run manually. You can have it run every minute/hour/day or what not, if you put it in a cron job. Try starting it manually like this: "$ perl rss-torrentflux.pl -v" to see if it works.
Report to moderator   Logged
Pages: 1 2 [All] Go Up Reply Notify of replies Send this topic Print 
« previous next »
Jump to:  


Powered by MySQL Powered by SMF 1.1.1 | SMF © 2006, Simple Machines LLC Powered by PHP