.. highlight:: bash Newznab on Ubuntu 11.10 ======================= This guide covers installing Newznab+ on a fresh copy of Ubuntu 11.10 64-bit server and assumes a basic knowledge of using Linux via the command line. TODO: Install and configure Memcache? Install Ubuntu -------------- Grab a copy of Ubuntu from their `website`_ and install it. Once the install is complete, log in, fire up a terminal window, do a full upgrade and reboot (entering your password when prompted):: sudo apt-get update sudo apt-get -y upgrade sudo apt-get -y dist-upgrade sudo reboot You'll probably want to install SSH if you don't have it already:: sudo apt-get install -y ssh Additionally, we're going to be compiling some code from source, so we'll need the tools to do so:: sudo apt-get install build-essential checkinstall .. _website: http://www.ubuntu.com/download/server/download Install Newznab --------------- Once Ubuntu is installed and updated, we can begin installing Newznab. Before we get started though, we need to decide where to install Newznab. Typically, web-related stuff goes in ``/var/www/``, so lets put Newznab in ``/var/www/newznab``. Let's create the directory and set it writeable by our user for now:: sudo mkdir -p /var/www/newznab sudo chmod 777 /var/www/newznab Now we can begin installing the prerequisites. Prerequisites ~~~~~~~~~~~~~ Newznab as a few dependencies, so let's start with installing the required software. According to the install docs, Newznab needs the following software: 1) PHP 5.2+ 2) GD Imaging Library w/PHP integration 3) PEAR 4) Database: MySQL (or Percona) 5) Web Server: Apache (or Nginx) Its worth mentioning a few things at this point. The first is that the default version of `MySQL`_ that comes with Ubuntu is kind of old (version 5.1). Therefore, we'll also cover installing a higher-performance version (`Percona`_ version 5.5) instead. Additionally, `Apache`_ is listed as a requirement, however it is possible to use a different web server instead. Therefore, we'll go over how to use the blazing-fast `Nginx`_ as well for those who don't want to use Apache. .. _MySQL: http://www.mysql.com/ .. _Percona: http://www.percona.com/ .. _Apache: http://www.apache.org .. _Nginx: http://www.nginx.org The following software is optional, but we'll also cover installing it and setting it up: 1) Unrar 2) FFmpeg 3) Lame 4) MediaInfo 5) Sphinx By the end of this guide we'll have a fully working Newznab+ install with all optional components working as well. PHP +++ Let's start by installing PHP and required PHP extensions:: sudo apt-get install -y php5 php5-dev php-pear php5-gd php5-mysql php5-curl The install docs also say that PHP needs a few tweaks. Using your favorite text-editor (I'm using ``nano``) open up ``php.ini``:: sudo nano /etc/php5/cli/php.ini Scroll down and look for ``register_globals``, make sure it is set to:: register_globals = Off Next, look for ``max_execution_time``, and set it to (2 minutes):: max_execution_time = 120 Next, look for ``memory_limit``, and either set it to ``1024M``, or ``-1`` for no limit:: memory_limit = 1024M While you're here, you should also set PHP's ``date.timezone`` (a list of available timezone settings can be found `here`_):: date.timezone = Europe/London That should do it for PHP for the time being (we'll come back to it when we configure the web server). .. _here: http://uk.php.net/manual/en/timezones.php Database ++++++++ You have a few options here, we'll cover two: 1) Use MySQL 5.1 2) Use Percona 5.5 I'd recommend that you use Percona, but the choice is yours. Pick one and follow *one* of the sections below (**do not** install MySQL and Percona). **MySQL 5.1** If you decided to use MySQL, simply install it via ``aptitude``:: sudo apt-get install mysql-server-5.1 mysql-client-5.1 libmysqlclient-dev **Percona 5.5** Installing Percona requires a little extra work upfront, but in the long run it is generally worth it as Percona is a high-tuned fork of MySQL. The first step is to add Percona into aptitude:: gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A gpg -a --export CD2EFD2A | sudo apt-key add - That retrieves and installs the GPG keys needed to add Percona's aptitude repository. Now we need to tell aptitude about the repositories:: sudo sh -c "echo \"\n#Percona\" >> /etc/apt/sources.list" sudo sh -c "echo \"deb http://repo.percona.com/apt lenny main\" >> /etc/apt/sources.list" sudo sh -c "echo \"deb-src http://repo.percona.com/apt lenny main\" >> /etc/apt/sources.list" Now update and install it:: sudo apt-get update sudo apt-get install -y percona-server-client-5.5 \ percona-server-server-5.5 \ libmysqlclient-dev It'll ask for a password to use for the MySQL root user, so pick one and remember it. Web Server ++++++++++ OK, now for the web server. Once again, you have multiple choices and we'll cover two: 1) Apache 2) Nginx Apache is "easier" and generally more flexible, but it also tends to be a system hog. So, if you can't make up your mind it is probably best to take the easy route and install Apache. If you feel like squeezing the most out of your machine and you are a little more skilled with configuring software, Nginx might be a good choice. Again, pick one and follow the steps below (don't install both). **Apache** First install it:: sudo apt-get install -y apache2 You'll also need to configure the ``php.ini`` for Apache, so follow the section above about configuring PHP (from above), but this time, edit the file ``/etc/php5/apache2/php.ini``:: sudo nano /etc/php5/apache2/php.ini Let's create the site configuration file for Apache. Open up a new file:: sudo nano /etc/apache2/sites-available/newznab Here is a template you can use for this file. You should change settings where appropriate for your setup:: ServerAdmin webmaster@localhost ServerName localhost # You might want to change this # These paths should be fine DocumentRoot /var/www/newznab/www ErrorLog /var/log/apache2/error.log LogLevel warn Now we need to disable the default Apache settings, enable the one we just created for Newznab and enable ``modrewrite``:: sudo a2dissite default sudo a2ensite newznab sudo a2enmod rewrite sudo service apache2 restart **Nginx** First install it:: sudo apt-get install -y nginx Now, for Nginx there are multiple ways to serve PHP files. Probably the best is ``php-fpm``, which is basically a daemon that runs and serves PHP to Nginx. So, let's go ahead and install that now:: sudo apt-get install -y php5-fpm You'll also need to configure the ``php.ini`` for FPM, so follow the section above about configuring PHP (from above), but this time, edit the file ``/etc/php5/fpm/php.ini``:: sudo nano /etc/php5/fpm/php.ini Just for good measure, restart the daemon:: sudo /etc/init.d/php5-fpm restart Let's create the configuration file for Nginx. Open a new file:: sudo nano /etc/nginx/sites-available/newznab Here is a template you can use for this file. You should change settings where appropriate for your setup:: server { # Change these settings to match your machine listen 80 default_server; server_name localhost; # Everything below here doesn't need to be changed access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; root /var/www/newznab/www/; index index.html index.htm index.php; location ~* \.(?:ico|css|js|gif|inc|txt|gz|xml|png|jpe?g) { expires max; add_header Pragma public; add_header Cache-Control "public, must-revalidate, proxy-revalidate"; } location / { try_files $uri $uri/ @rewrites; } location @rewrites { rewrite ^/([^/\.]+)/([^/]+)/([^/]+)/? /index.php?page=$1&id=$2&subpage=$3 last; rewrite ^/([^/\.]+)/([^/]+)/?$ /index.php?page=$1&id=$2 last; rewrite ^/([^/\.]+)/?$ /index.php?page=$1 last; } location /admin { } location /install { } location ~ \.php$ { include /etc/nginx/fastcgi_params; fastcgi_pass 127.0.0.1:9000; # The next two lines should go in your fastcgi_params fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; } } Let's make sure that the Nginx logs directory exists and is writeable:: sudo mkdir -p /var/log/nginx sudo chmod 755 /var/log/nginx Now, let's disable the ``default`` Nginx site handler and enable our ``newznab`` configuration:: sudo unlink /etc/nginx/sites-enabled/default sudo ln -s /etc/nginx/sites-available/newznab /etc/nginx/sites-enabled/newznab Finally, let's restart Nginx:: sudo service nginx restart Extras ++++++ unrar ''''' Unrar is used to extract files from releases. Installing unrar is trivial:: sudo apt-get install -y unrar FFmpeg '''''' Once again, the version of FFmpeg in Ubuntu's repository is old and crusty, so lets build a newer one and also add x264 support. Start by grabbing the source code we need:: wget http://ffmpeg.org/releases/ffmpeg-0.8.7.tar.gz wget ftp://ftp.videolan.org/pub/videolan/x264/snapshots/last_stable_x264.tar.bz2 sudo apt-get install -y libfaac-dev libjack-jackd2-dev \ libmp3lame-dev libopencore-amrnb-dev \ libopencore-amrwb-dev libsdl1.2-dev libtheora-dev \ libva-dev libvdpau-dev libvorbis-dev libx11-dev \ libxfixes-dev texi2html yasm zlib1g-dev \ libdirac-dev libxvidcore-dev Let's build ``x264`` first:: mkdir x264 tar --strip-components=1 -jxf last_stable_x264.tar.bz2 -C x264 cd x264 ./configure --enable-static make sudo checkinstall --pkgname=x264 --pkgversion="3:$(./version.sh | \ awk -F'[" ]' '/POINT/{print $4$5}')" \ --backup=no --deldoc=yes --fstrans=no --default Now, onto ``FFmpeg``:: cd ../ tar xvfz ffmpeg-0.8.7.tar.gz cd ffmpeg-0.8.7/ ./configure --enable-gpl --enable-libfaac --enable-libmp3lame \ --enable-libopencore-amrnb --enable-libopencore-amrwb \ --enable-libtheora --enable-libvorbis --enable-libx264 \ --enable-nonfree --enable-postproc --enable-version3 \ --enable-x11grab --enable-libdirac --enable-libxvid make sudo checkinstall --pkgname=ffmpeg --pkgversion="5:0.8.7" --backup=no \ --deldoc=yes --fstrans=no --default hash x264 ffmpeg ffplay ffprobe Lame '''' Lame is used for processing audio samples:: sudo apt-get install -y lame MediaInfo ''''''''' MediaInfo is used to gain information about various types of media files found in releases. In order to install it we simply need to add the MediaInfo ``aptitude`` repository:: sudo apt-get install -y python-software-properties sudo add-apt-repository ppa:shiki/mediainfo sudo apt-get update sudo apt-get install -y mediainfo Sphinx '''''' Sphinx is used for full-text searching. It is insanely fast and if you really want your Newznab+ install to fly, it is highly recommended. Unfortunately the version in Ubuntu's ``aptitude`` repository is horribly old, so we'll need to build a newer version. Let's download the source and extract it:: wget http://sphinxsearch.com/files/sphinx-2.0.2-beta.tar.gz tar xvfz sphinx-2.0.2-beta.tar.gz cd sphinx-2.0.2-beta Download and extract ``libstemmer_c``:: wet http://snowball.tartarus.org/dist/libstemmer_c.tgz tar --strip-components=1 -zxf libstemmer_c.tgz -C libstemmer_c Configure it:: ./configure --prefix=/usr/local --with-libstemmer Now we're ready to compile Sphinx. For this step you can speed up the compilation on a multi-core system. If you have a 4-core system, for example, you can do (replace ``j4`` with the number of cores your machine has):: make -j4 Once that finally finished, install it:: sudo checkinstall --pkgname=sphinx --pkgversion="2.0.2-beta" --backup=no \ --deldoc=yes --fstrans=no --default Now we have a nice new version of Sphinx installed in ``/usr/local``. Binaries are installed in ``/usr/local/bin``. Newznab Source ~~~~~~~~~~~~~~ Finally, we can now begin installing Newznab! We'll be grabbing the latest and greatest version, so we'll need Subversion installed first:: sudo apt-get install -y subversion Now we can check out the Newznab code:: svn co svn://svn.newznab.com/nn/branches/nnplus /var/www/newznab At this point we might as well set the permissions on a couple of directories as well:: sudo chmod 777 /var/www/newznab/www/lib/smarty/templates_c sudo chmod 777 /var/www/newznab/www/covers/movies sudo chmod 777 /var/www/newznab/www/covers/anime sudo chmod 777 /var/www/newznab/www/covers/music sudo chmod 777 /var/www/newznab/www sudo chmod 777 /var/www/newznab/www/install sudo chmod 777 /var/www/newznab/nzbfiles/ Configure Newznab ----------------- Now that Newznab is installed, we need to configure it. Configuration is done via a web browser. Run Installer ~~~~~~~~~~~~~ It is now time to configure Newznab. This is done via a web-based installer. Open up ``http://localhost/install`` in a web browser (or whatever the address/IP address is of your server) and follow the guided steps. Enable Groups ~~~~~~~~~~~~~ Head over to ``/admin/group-list.php`` in your web browser and pick some groups to index by clicking "activate" on a few groups. Set Paths and Options ~~~~~~~~~~~~~~~~~~~~~ Make sure to set your ``newznab ID``. We need to let Newznab know where all the extra software is that we installed earlier, so head over to ``/admin/site-edit.php`` in your browser, scroll down to the "3rd Party Application Paths" section and update the fields: Unrar Path ``/usr/bin/unrar`` Mediainfo Path ``/usr/bin/mediainfo`` Ffmpeg Path ``/usr/local/bin/ffmpeg`` Lame Path ``/usr/bin/lame`` If you'd like to enable audio previews, check ``Save audio preview``. It is worthwhile to do rar checking, so set ``Check For Passworded Releases`` to ``Deep``. That's it for configuration right now, but don't close your browser yet as we'll be coming back to the configuration page when configuring Sphinx. Indexing -------- Back at the command line its time to fire up the binaries and releases update scripts:: cd /var/www/newznab/misc/update_scripts The ``update_scripts`` folder contains a lot of scripts. The most important ones are ``update_binaries.php`` and ``update_releases.php``. If you have any experience with ``screen`` or ``tmux``, it is highly recommended that you use one of these to run the update scripts, as it will allow you to monitor the update process, observe and resolve issues; this is especially important for newcomers to Newznab. With that said, Newznab also ships with an init-sytle script that can be installed to make Newznab run more or less as a daemon that will start and stop with startup and shutdown, respectively. **Screen or tmux** If you want to go the ``screen`` or ``tmux`` route, you'll need to pick one and install it:: # Install screen... sudo apt-get install -y screen # ...or tmux sudo apt-get install -y tmux In the ``nix_scripts`` directory there is a useful script called ``newznab_screen.sh`` that runs ``update_binaries.php`` and ``update_releases.php``, in addition to a few other scripts, continuously and automatically. First, we need to modify it however, so lets change dir and make a copy:: cd /var/www/newznab/misc/update_scripts/nix_scripts cp newznab_screen.sh newznab_screen_local.sh Now open ``newznab_screen_local.sh`` in a text editor and modify ``NEWZNAB_PATH`` near the top to point to our installation path:: nano newznab_screen_local.sh Set ``NEWZNAB_PATH``:: export NEWZNAB_PATH="/var/www/newznab/misc/update_scripts" Now we can run the script via ``screen``:: screen sh newznab_screen_local.sh You should see the script download headers for the groups that you have enabled and then run various stages that will attempt to group and catalogue the headers. For now, just leave the script running and detach from ``screen`` by typing ``cntl a d``. Sphinx ------ As mentioned previously, Sphinx is a fast full-text indexer. By default, it is disabled in Newznab, so go ahead and enable it by visiting ``/admin/site-edit.php`` and setting "Use Sphinx" to "Yes". While there, you'll also notice that there are a few other configuration options for Sphinx. By default, Sphinx will index all of the release information, however, there are three other optional indexes: NZB contents, NFO contents and release files. Enabling these optional indexes will add increased processing time, so you will likely want to experiment to see what combination works best for your hardware. For now, you don't have to enable any of the optional indexes. To get Sphinx running, we need to generate a ``sphinx.conf`` file. To do this we'll use the ``nnindexer.php`` script in ``misc/sphinx``:: cd /var/www/newznab/misc/sphinx ./nnindexer generate The script will print out the location of the ``sphinx.conf`` file, which by default will be ``/var/www/newznab/sphinx/sphinx.conf``. This path needs to be entered into the "Sphinx Configuration Path" setting located at ``/admin/site-edit.php``. Now we need to start the search daemon and create the indexes and restart the daemon:: ./nnindexer.php daemon ./nnindexer.php index full all ./nnindexer.php index delta all ./nnindexer.php daemon --stop ./nnindexer.php daemon Summary of Installed Software ----------------------------- * PHP v5.3.6 * Pear v1.9.2 * MySQL v5.1.58 or * Percona v5.5.17 * Apache v2.2.20 or * Nginx 1.0.5 * FFmpeg v0.8.7 * MediaInfo v0.7.50 * Lame v3.98.4 * unrar v4.00-beta3 * Sphinx 2.0.2-beta