Uploader – Janhouse.lv

Auto-uploader v4

Uploader v4 is out and it is miles ahead of v3 in terms of stability, ease of setup and ease of use.

Even though I released it a few months ago, there were some things I wanted to add and test before I announce it to the public. It now performs better than I expected so I figured I should write about it here.

The new start/stop/status script. — The new start/stop/status script in action.

Setup

Statically compiled – no need for compiling

The time it takes to do the initial setup has been reduced a lot. There is no need to install any dependencies anymore because Uploader is now available as a statically compiled i686 binary that should run on most Linux servers.

I also made statically compiled rTorrent (with the headless patch), tmux, lighttpd and other tools that are used with the uploader. You can use these versions to save some time going through dependency hell and compiling, or you can use your own binaries if you wish to. (Since statically compiled binaries might be of interest to some of you, especially in case of rTorrent, I’ll talk about this in a separate post. Sources are on my Github page.)

This is good news because it is possible to run Uploader on shared seedboxes. So far I have tested it only on some shared boxes with SSH access but it should be possible to use it without ssh as long as there is some way to execute the binary.

Better structured, completely portable

In previous versions it took some time to change paths in every configuration file to make it work after moving it to other location. The new version uses relative paths and some cool scripts to make sure it runs no matter where you move it.

Just move “tuper4/” folder to a new location/server and run it without doing any manual changes. Yes, it is that simple.

Easy to migrate

If you are still using the previous version it is easy to migrate to v4 using the new migrate script. Just run a single command and your’e all done.

Automatic updating

V4 will receive critical updates automatically by default. This means that when a download source or some API changes and breaks some functionality, it will fix itself as soon as update is available.

Update status messages on the server during testing.

Usability and features

Now, when deploying and updating is made easy I have had more time to focus on new features and usability improvements.

Uploader is now easier to use by novice users and at the same time more advanced features have been added for the more experienced users who need more control over how things work.

Features

Multiple functions were fixed and new features were added to the core of the Uploader. Among them are:

Easier to view and understand debugging messages;
Better dupe detection;
Smart IMDB title search. This allows automatic IMDB info and poster retrieval;
Fixed TVRage info lookup for TV shows;
Updated GiantBomb game info retrieval functions;
Better BBCode converter and image uploading functions to include screenshots and other images in the description;
Fixed RAR and ZIP extraction functions that allow automatic creation of new unpacked releases.
Better handling of unicode, messed up encoding and ascii art.

Some of the non-core improvements:

Added new debugging tool called “logstream” that uses websockets to show realtime message output of the Uploader. It also shows process groups and allows to view output of each single task;
Modified browser plugin to use websockets to show realtime status of the task being run;
New version of web panel is being worked on;
New script to easily start, stop and see the status of the Uploader and all the related processes (rTorrents, lighttpd).

Better support

Making it easier to set up and update gives more time for supporting users and helping them find answers to their questions.

Also wiki and forum has improved and it is easier to find answers than it was before.

Music release info search results for Waffles upload form.

Existing users

And as it has always been, existing users still get the access to the latest version and it probably will stay this way in the future.

Summary

As years have passed, Uploader is still being developed and version 4 is a big step forward. It is completely different experience to how it was couple years ago. Making these changes makes it better for you, saves more time for me and it also means that it costs less. So, when you need the ultimate tool to automate it, you know where to look. 🙂

Automatic movie screenshot maker for Auto Uploader

I have finally found some time to make new features for the Auto Uploader.

So, the Thumbnailer is finally here.

Features:

Creates video thumbnails (single shots and joined timelapse);
Creates thumbnails even if video file is packed inside .rar archive (Linux only);
Skips sample files;
Extracts basic information about video file (resolution, codec, bitrate, etc.);
Automatically uploads created images to public image hosting sites and creates BBCode;
Can be fine tuned any way you want.

I am providing this as an external module for the Uploader.

If you want to get it, feel free to contact me. 🙂

Generating and uploading screenshots:

Generating screenshots:

I am going to upload a better video once I set up the new version on some tracker.

Following screenshot demonstrates BBCode generated by the Uploader:

Upgrades to announcement system

Some of you know that there are times when release announcements for certain download sources stop appearing for some reason. Mostly it is because the announcer gets banned from the announcement channel or there are some other problems with the feed.
I promised that I would make it possible for you to help with announcing. And so I did! 🙂

I made some changes so you could help with passing announcements from your box to the tuper-server, that then sends them out to every connected user.
This way you will get the announcements sooner than before (at least for some sources) and they will keep coming even if some boxes get problems.

Those who want to help out, please contact me. I will give your uploader account permission to send release announcements and help you with setting up the necessary scripts.

Also, to make sure that releases get announced only once, I am now storing announcement history, so you will be able to check the history easily.

FTP and file feature for auto uploader

Few days ago FTP feature was added to the uploader. It means that any files/folders can be easily added to the uploader’s database. This allows using FTP servers and other sources to auto upload to your tracker (or maybe even upload to some other FTP servers).

So basically you can make it work in any way you want now. FTP to Tracker, Tracker to Tracker, Tracker to FTP, FTP to FTP or even make it work with direct files on the web.

Those of you who already have the access to the new Wiki for the Uploader, can look there for information on setting it up.

I’ll keep making this new FTP thing better so any feedback is welcome in the bugtracker’s forum. 🙂

From the shiny features, automatic screenshot grabber script for videos is left to port to V3.

The loop

It is interesting that my “torrent auto uploader” has become one of the best (or maybe even the best) and most accessible tool in this field.

And it is interesting that lately more and more people want to use my uploader for downloading from sites that are already using it.

Let’s see if it gets more interesting after I test and add support for downloading from FTP servers.

P.S. Yes, I know, some of you were waiting long time for this. 🙂

Auto-uploader v3

A bit later than I expected but it is available for testing in the new tuper-v3 repository on my SVN server. Use your old usernames and passwords to get access to it.

I did rewrite most of the code and changed the way it works.

The new auto-uploader client (announce-client) process keeps running in the background and gets announcements from main auto-uploader server. This means that there is no need for IRSSI anymore (unless you really want to use it).

Possibly it is also slightly faster since most of the needed modules are kept loaded while announce-client process is running.

Multiple commands are executed asynchronously in separate worker processes.

Announce-client locally listens for incoming connections so you can still pass commands to it manually. (Detailed documentation about communication with announce-client will be added to repository later this week.) You can use this to make your own custom extensions.

Next updates:

Better support for uTorrent. (Teach Eliminator to use uTorrent API.)

FTP auto-upload (or should I say auto-download from FTP, auto-upload to tracker).

Setup wizard for Windows users.

Update “Uploader” section in my webpage, make it show v3 changelog.

Web panel for Uploader

Since some people prefer shiny user interfaces over command line, I have added web administration panel to auto uploader.

Since it is really early über pre-alpha version, it doesn’t have many options yet but it will change in time.

The idea is to create admin panel that allows you to manage Uploader using only your browser and to forget about terminal commands.

uploader-web-panel-alpha-version — Alpha version of web panel for Uploader

As you can see in the screenshot above, you can already search, remove, upload and redownload releases.

Web panel is built using PHP and jQuery. To make it work, make sure you have installed Sqlite3 for PHP (it comes installed by default from PHP 5.3.x). You should be able to install it on Debian/Ubuntu by using sudo apt-get install php5-sqlite.

It is available through SVN repository, inside trunk/web_panel/.