Juiced's RSS crawler minipack v 0.0.1

Site Message

(Message will auto close in 2 seconds)

Logged in as: OPPZeroCool ( Log Out )

My Controls · View New Posts · My Assistant · My Friends · 1 New Messages

If you would like to advertise your Hosting, Services or Products on Filesoup, please contact Geeker for a quote.

Juiced's RSS crawler minipack v 0.0.1

Options

juiced View Member Profile Add as Friend Send Message Find Member's Topics Find Member's Posts	May 19 2007, 07:06 AM Post #1
Vegetable Group: Member Posts: 38 Joined: 7-May 07 Member No.: 452,851	Juiced's RSS Crawler mini-pack version 0.0.1 What does it do? -Crawls the rss feeds for mininova, meganova, torrentspy, btjunkie, thepiratebay and downloads found torrents. What does it not do? -Drop info into DB -Get torrent metadata Ok, so here is the first mini-pack release of my rss crawler project. Keep in mind this is version 0.0.1 so don't expect it to be perfect and do everything you could ever want. It's just the basic code for the beginning of this project. The meganova cralwer still picks up a bit of excess crap. You will have to alter the $start variable and the while statement based on your needs to determine what specific RSS feeds will be crawled so I want no lame comments about how it's not crawling all the feeds or some nonsense because you decide. Also, if anyone feels like writing the code to have the torrents drop into a DB, get scrape/metadata info (maybe have it integrated into t-xore or something) by all means go ahead. Do whatever you want and do a re-release, all I ask is you keep and do not alter the project info message at the top of each of the php pages. Future releases will include crawlers for more sites, more advanced features, and an easy to use interface. JuicedsRSSCrawlerminipack.zip ( 3.64k ) Number of downloads: 16

phrostwave View Member Profile Add as Friend Send Message Find Member's Topics Find Member's Posts	May 19 2007, 07:11 AM Post #2
DJ Taktikz / STR Records Group: Member Posts: 52 Joined: 7-September 06 Member No.: 411,795 System Specs: (show/hide) 64bit x86 based AMD 'san diego' 3700+anthlon processor (2.41G), nforce4-a754, 2gb of ram, SATA, windows xp sp2	excellent work juiced, this will definately be a project i'm gonna start working on asap! This post has been edited by phrostwave: May 19 2007, 07:16 AM -------------------- may 15, 2007 releases - ak nova 1.4 (snapshot), ak tracker 1.0.1 (snapshot). t-xore 0.4 preconfigured with ibitzy, registration captcha (ive seen it on 18 sites so far!), admin search, user search, and tons of other stuff by me and various other filesoup members! bNova stylesheet for ak nova 1.4 look for ak nova 1.7 soon! lots of features like forums, dhtml/ajax, two mass scrapes, nicer admin area, plus a ton of other stuff!!

juiced View Member Profile Add as Friend Send Message Find Member's Topics Find Member's Posts	May 19 2007, 08:00 AM Post #3
Vegetable Group: Member Posts: 38 Joined: 7-May 07 Member No.: 452,851	Cool, good to hear

SumoTorrent View Member Profile Add as Friend Send Message Find Member's Topics Find Member's Posts	May 19 2007, 08:55 AM Post #4
Vegetable Group: BT Community Leader Posts: 42 Joined: 26-February 07 Member No.: 447,369	Interesting. I coded myself sor SUMO a generic RSS fetcher (not crawler as it actually crawls nothing, it just get the URL for the enclosed content or to the details page), then I have a bunch of crawlers multi threaded that extract the torrents from various sources on the web.

juiced View Member Profile Add as Friend Send Message Find Member's Topics Find Member's Posts	May 19 2007, 11:34 AM Post #5
Vegetable Group: Member Posts: 38 Joined: 7-May 07 Member No.: 452,851	Some of the download code is messed up at the moment. This line of code CODE $filelink = file_get_contents("$actualLink/download.torrent"); must be altered for each website depending on how the site is set up... I'll put a fixed version up tomorrow sometime. I totally spaced this somehow hah. EDIT: Haven't been able to finish them all but here is the fixed mininova rss crawler (download code corrected). mininova.php ( 1.47k ) Number of downloads: 6 This post has been edited by juiced: May 19 2007, 09:33 PM