Showing posts with label HandBrake. Show all posts
Showing posts with label HandBrake. Show all posts

Friday, May 1, 2009

Monitoring a Directory to Automatically Invoke HandBrake

I've seen this question pop up a lot on HandBrake's forums and IRC channel, so I thought I'd make an entry about it here (Mac users skip down to the bottom for your directions):

Many folks have expressed interest in being able to specify a directory for HandBrake to 'watch' for new files that it would then automatically attempt to convert with predefined settings. I think most people are wanting this for use with devices, such as iPods, PS3s and AppleTVs, which require specific settings for videos to work. While HB doesn't support this functionality on its own (and the devs don't sound too interested in adding it), you can accomplish much the same thing in Ubuntu Linux using HandBrakeCLI and a little shell scripting.
WARNING: I'm a novice at scripting and there is definitely a more effective and elegant way of doing this. If you have a suggestion, please leave a comment! Similar steps will also work on other platforms/distros, so feel free to leave a comment about your successes or failures.

First, we'll need to install a utility to enable monitoring of directories:
sudo aptitude install inotify-tools
Next, we'll make some new directories in our home folder to hold our scripts and conversions. In a terminal, type:
cd $HOME ; mkdir HandBrake ; mkdir HandBrake/convert
Navigate to the newly created HandBrake directory:
cd HandBrake
and type:
gedit monitor.sh
This is where we'll write our script to monitor the 'convert' directory and invoke another script to do the actual conversion:
#!/bin/bash

inotifywait --monitor -e moved_to -e create ~/HandBrake/convert | while read dir;
do
(~/HandBrake/convert.sh)

done
Save, exit and--again--type:
gedit convert.sh
Here we will create our conversion script (be sure to put your desired file extension and preset in place instead of the bracketed reminders):
#!/bin/bash
for file in ~/HandBrake/convert/*
do HandBrakeCLI -v -i "$file" -o "$file".converted.[FILE-EXTENSION-GOES-HERE] --preset [PRESET-NAME-GOES-HERE] ;

#uncomment next line to delete original
#rm $file

done
Save and exit, then type:
chmod +x *.sh
to make both scripts executable.

Now, you can start monitoring by typing:
sh ~/HandBrake/monitor.sh
or you can set the script to run as a startup item where it will run continuously in the background starting the next time you log on.

Henceforth, any file you move or copy into the 'convert' directory will automatically convert to the desired format. This script will only work on one file at a time (i.e., you have to wait for the encoding to finish before dropping in the next file to convert). Also of note: HB will choke if the file is weird in any way--e.g. no audio track--and you'll have no way of knowing it if the script is running in the background, since it won't print any output.

Good luck and let me know if you run into any problems.

Bulk Encoding on Macs


Update (06/01/09): There's been a lot of clamoring on the Mac board of the HandBrake forums asking for bulk input of files to be converted using a preset. The devs have no interest in adding such a feature at this time because of the tremendous support headache it could cause, but you Mac users can do scripting to accomplish the same thing.

Just like the Linux users, open a Terminal (Applications > Utilities > Terminal) and type:

cd Desktop ; nano convert.sh
then paste in this (shift+ctrl+v; be sure to put your desired file extension and preset in place instead of the bracketed reminders):
#!/bin/bash
for file in ./*
do ./HandBrakeCLI -v -i "$file" -o "$file".converted.[FILE-EXTENSION-GOES-HERE] --preset [PRESET-NAME-GOES-HERE] ;

#uncomment next line to delete original
#rm $file

done
Save and exit (ctrl+x), then type:
chmod +x *.sh
to make the script executable. Now, just put the script into a folder with your HandBrakeCLI binary and you should be able to invoke the script (navigate to its directory in the Terminal by typing cd [space after cd] and then dragging your conversion folder onto the Terminal window and hit 'Enter,' then type ./convert.sh) and automatically convert any files within the directory using the chosen preset. I would recommend just keeping a folder around that you use for conversions and keep the script and HandBrakeCLI binary in there at all times, then you can just drop in the files you want to convert, start the script and go along your merry way.

UPDATE (12/20/2013): An anonymous reader shared his script, which sounds much more robust than mine:
inotifywait -r --monitor --quiet -e moved_to -e close_write --format '%w%f' /mnt/public/convert/video/android-hq/ | while read -r FILE; do echo "File copy detected" echo "Starting HandBrake..." (sleep 15 && /usr/bin/HandBrakeCLI -v -i "$FILE" -o /mnt/public/convert/output/video/"$(echo "$FILE" | cut -c38- | rev | cut -c4-| rev)android-hq.mkv" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --width 720 --loose-crop --decomb --markers --turbo --two-pass --vfr --subtitle 1 --native-language eng && sleep 15 && rm -rf "$FILE")& done
UPDATE (9/27/2014): The same user (name is teeedubb, apparently) is back with an update. This was posted in the comments but got mangled, so I tried to piece it back together:
I'm back - I have revisited the above script because it had some glaring faults - it would start a handbrakecli process for each file, which when encoding a seasons worth of tv shows it would bring my pc to its knees, it didnt handle files in sub directories, didnt handle multiple profiles and probably would have given undesirable results if a file extension had more than 3 character, plus it was kinda messy. New version below solves these issues: It queues encodes using task-spooler (tsp package in ubuntu), works with sub-directories in the watch folder (and deletes any empty subdirectories within watch folders) and supports multiple profiles. Options are 'watch directory', 'output directory' and 'task-spooler slots' (concurrent jobs) which are set through the variables. You can set multiple handbrake profiles which the script uses based on which directory the files are copied into, if files are copied into the root watch directory the first profile is used. Options for profiles are: 'name', profile name which will be appended to the transcoded file (this needs to match the profile directory and be fairly unique - script searches the input file location for profile name, so a profile called 'android' could confuse the script when transcoding the movie 'android cop'), 'handbrake settings', settings for handbrake to use (I'm pretty sure you could use --preset XXXX here as well), 'file extension', file extension for handbrake to use on output file and 'delete source file', whether or not to delete the source video file. You can create extra profiles by adding variables beginning with PRESET2 etc and adding elif entries to the if/then statement in the script. By default the script attempts to delete any empty subdirectories *and its ancestors* when completing a transcode, so you need to ensure you profile directories are not empty - I have done this by creating a hidden file in each profile dir and making the file un-deletable (eg: profile1 dir is /mnt/public/convert/video/android-hq, run the command touch /mnt/public/convert/video/android-hq/.android-hq && sudo chattr +i /mnt/public/convert/video/android-hq/.android-hq).
and here's the script:
#!/bin/bash
#script to watch a directory for incoming files and pass them onto HandBrakeCLI

WATCH_DIR="/mnt/public/convert/video/"
OUTPUT_DIR="/mnt/public/convert/output/video/"
TASK_SPOOLER_SLOTS=2

##HANDBRAKE PROFILE 1
PRESET1_NAME="android-hq"
PRESET1_HANDBRAKE_SETTINGS=" -e x264 -x weightp=0:cabac=0 -b 650 --audio 1 --aencoder faac --ab 96 --mixdown stereo --gain 3 --width 720 --loose-crop --decomb --markers --turbo --two-pass --vfr --subtitle 1 --native-language eng"
PRESET1_FILE_TYPE="mkv"
PRESET1_DELETE_SOURCE="yes"

###########################

ts -S $TASK_SPOOLER_SLOTS
inotifywait --recursive --monitor --quiet -e moved_to -e close_write --format '%w%f' "$WATCH_DIR" | while read -r INPUT_FILE; do

PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"

if [[ $(echo "$WATCH_DIR" | grep -i "$PRESET1_NAME") ]] ; then
PRESET_NAME="$PRESET1_NAME"
HANDBRAKE_SETTINGS="$PRESET1_HANDBRAKE_SETTINGS"
FILE_TYPE="$PRESET1_FILE_TYPE"
DELETE_SOURCE="$PRESET1_DELETE_SOURCE"
fi
FULL_FILE_NAME=$(echo ${INPUT_FILE##*/})
OUTPUT_FILE=$(echo ${FULL_FILE_NAME%.*})
tsp bash -c 'nice -n 19 /usr/bin/HandBrakeCLI -v -i "$0" -o "$1""$2"-"$3"."$4" "$5" && if [[ "$6" = yes ]] ; then sleep 15 ; rm -f "$0" ; fi ; rmdir -p "$(dirname "$0")"' "$INPUT_FILE" "$OUTPUT_DIR" "$OUTPUT_FILE" "$PRESET_NAME" "$FILE_TYPE" "$HANDBRAKE_SETTINGS" "$DELETE_SOURCE"
done

Monday, April 13, 2009

My PPA Repository for Filthy Pants (HandBrake Repository)

After much more time and effort than I care to admit, I finally got Canonical's build servers to successfully build deb binaries for HandBrake in my PPA repository.

From here on out, I will no longer distribute packages through Rapidshare, which some people didn't like, and will exclusively use the PPA repository. I will regularly build development packages of HandBrake with both the GTK and QT4 GUIs (the CLI version will also be included with the GTK package).

To access my PPA repository through apt (so updates happen properly and so forth), hop on a terminal and type:
sudo gedit /etc/apt/sources.list
and add these lines at the bottom:
#Filthy Pants PPA
deb http://ppa.launchpad.net/hunter-kaller/ppa/ubuntu jaunty main
deb-src http://ppa.launchpad.net/hunter-kaller/ppa/ubuntu jaunty main
Save and close, then type:
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0x065fda2d23cfea71e753d04703e1fe8b2796dec2
After that, type:
sudo aptitude update
and you're all set.

Wednesday, March 4, 2009

New Directions for Building HandBrake SVN

Update (5/15/09): I have working binaries available in my PPA repository. Directions for adding it to your package manager are available here.

A couple of months ago, the HandBrake devs implemented some major changes to the build procedure for HandBrake from the project's SVN repository. I'm not really sure why they did this, but it seems to have cut a couple of dependencies, which is nice*. Here's the new method that worked for me on Ubuntu 8.10 Intrepid Ibex and 9.04 Jaunty Jackalope (adapted from their build instructions in the readme).

Step 1: download the dependencies
In a terminal, type:
sudo aptitude install subversion build-essential m4 wget autotools-dev yasm autoconf intltool libtool libbz2-dev zlib1g-dev libglib2.0-dev libdbus-glib-1-dev libgtk2.0-dev libhal-dev libhal-storage-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev automake1.9 libnotify-dev libwebkit-dev
Update (6/18/09): as of svn 2550, libgtkhtml-3.14-dev was replaced by libwebkit-dev.
Users of OpenSuSE may need to download the additional dependencies zypper and in to build the CLI, as well as gtkhml2 and gtkhtml2-devel to build the GUI. Users of Red Hat/Fedora or derivatives may need to install the package groups "Development Tools," "Development Libraries," "X Software Development," and "GNOME Software Development," as well as zlib-devel, bzip2-devel, dbus-glib-devel, hal-devel, gtkhtml3-devel, gstreamer-devel, and gstreamer-plugins-base-devel.

Step 2: download the source from HandBrake's SVN repository
in a terminal, type:
svn checkout svn://svn.handbrake.fr/HandBrake/trunk hb-trunk
This will download a bunch of stuff and finish by telling you which revision you have checked out.

Step 3: switch to your newly created directory
in a terminal, type:
cd hb-trunk
Step 4: prepare the source for building
in a terminal, type:
./configure --launch
This command should take care of most of the remaining steps, i.e. making a scratch directory and compiling the source for both the CLI and GUI interfaces.

Step 5: install the GTK GUI
in a terminal, type:
cd build/gtk ; sudo make install
In you experienced any errors during Steps 4 or 5, you can attempt to run through the build functions manually:
In a terminal, type:
rm -rf build ; mkdir build ; cd build
then:
../configure ; make
and finally:
cd gtk ; make ; sudo make install


*Interestingly, the build process no longer invokes/requires jam, which had always been a vestigial requirement left over from HandBrake's roots in the venerable BeOS.

Thursday, February 19, 2009

HandBrake Qt4-native GUI Updated

Update (5/15/09): the code has improved a lot lately and you can download updated binaries from my PPA repository. Directions for adding the repo to your package manager are available here.

Gonza, a developer from the HandBrake forums, added some updates to his native Qt4 HandBrake GUI, known as qtHB, to bring it a little closer to the current code state. I haven't had a chance to do any extensive testing with it, but I wanted to go ahead and post some deb binaries so any KDE/Kubuntu users can give it a shot.

32-bit qtHB (built on a Core2Quad Kentsfield)

These binaries are more up-to-date, but wouldn't package properly, so you'll have to download the qt4-core dependency (and maybe others) manually (sudo aptitude install qt4-core). Run it by double-clicking or by navigating to its directory and typing ./qtHB into a terminal.
32-bit qtHB binary
64-bit qtHB still to come

Leave me a comment if you run into any issues with any of it, or if you have any requests (rpm packages, etc).

If you're looking for CLI or GTK GUI builds of the latest code, check out my post here.

Monday, November 24, 2008

HandBrake with Live Video Previews

The HandBrake team has released version 0.9.3, which includes official, sanctioned builds of the GTK GUI in both 32- and 64-bit formats. This new version also brings a number of improvements that have been enjoyed by those of us using the SVN builds for some time to the rest of the HB users. However, new features are constantly being added, and the codec pool is always being updated, so I will continue to post the latest bleeding-edge builds here on my site, starting with svn1952, which includes live video previews (!), as well as a more recent version of the x264 and ffmpeg codecs.

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

Here's how the live preview looks. You can really get in there and see the direct impact of your changes. This leads to better cropping and a better sense for your end product. (this is my Mac x-forwarding GHB from my Ubuntu box. As you can see, the live previews even work through that):

For those of you who wish to compile on your own hardware (recommended for those with newer Core 2 Duo CPUs), the process is exactly the same as my previous SVN instructions, except for the addition of 2 new dependencies: libgstreamer0.10-dev and libgstreamer-plugins-base0.10-dev. I've updated the instructions there to reflect the change.

Monday, September 29, 2008

How To Compile HandBrake With New Psy Options Enabled

Update: It looks like the new x264 version is committed to the SVN repo now, so you should be able to just perform Steps 1, 2, and 5 (i.e., a normal SVN checkout and compile) to get the new psychovisual options.

Update (5/15/09): I have working binaries available in my PPA repository. Directions for adding it to your package manager are available here.

Here are directions to download and install the latest development code for HandBrake that can take advantage of the revolutionary new psychovisual additions to the x264 codec (known as psychovisual rate distortion [psy-rd] and psychovisual trellis [psy-trellis]).

Step 1: check out the latest source code from SVN:
svn co svn://svn.handbrake.fr/HandBrake/trunk HandBrake
Step 2: navigate to the new directory
cd HandBrake/contrib
Step 3: edit the download location for the x264 codec in your favorite text editor (I prefer nano):
nano version_x264.txt
then, either delete or comment out (put a # in front of it) the address that is already there and replace it with this:
http://download.m0k.org/handbrake/contrib/x264-r979-6d4af8d.tar.gz
Step 4: return to the HandBrake directory:
cd ..
If you just want to use the CLI version (Linux users have no choice), you're almost finished and can go straight to Step 5. If you are using OS X or Windows, though, you can update the GUI interfaces to reflect these new options by applying the appropriate patch (for OS X or for Windows)

To apply the patch, move it into the HandBrake directory and type into a terminal:
patch -p0

Step 5: compile as usual:
make
That should get you all set. The new options require a subme value of ≥6, which should be the new default (and should be reflected in the GUIs and presets). If you have any questions or concerns, leave me a comment and I'll try to help.

Friday, September 19, 2008

HandBrake With New Adv. x264 psy-rd

Update: I just finished conducting some tests using live, film-grained content and the details are posted below the cartoon data.

x264 just committed some fancy new features, known as psychovisual rate distortion and psychovisual trellis, to their git repository. These features are supposed to produce a subjectively superior picture by maintaining detail and minimizing blurriness during the encoding process (I guess; read more about it here, at the blog of Dark Shikari, a developer for the x264 codec).

However, if you read the posts at the Doom9 forums linked in his blog post, they mention that Peak Signal To Noise Ratio (PSNR)--basically the only objective measure of picture distortion available to me--will always be negatively affected by the new features, and freeze-frames will also look worse. This makes the new features sound a little like voodoo to me, since I tend to favor objective measures over subjective ones. Regardless, I decided to try it out.

I used a bitrate of 250 in an mp4 container with these adv. x264 settings:
ref=5:mixed-refs:bframes=6:bime:weightb:b-rdo:me=umh:subme=7:
filter=0,-1:trellis=2:threads=2
For the psy encoding, I followed those settings with these psy-specific options (first number sets the psychovisual rate distortion [value from 0.1-1] while the second controls the psychovisual trellis [again, 0.1-1]):
psy-rd=1,1
If I understood things correctly, psy-rd only works with a subme setting ≥6, btw, so keep that in mind if you want to use the psy settings. Here are a couple of screenshots from an episode of Futurama encoded using otherwise identical settings (v0.9.2 on the left [no psy], svn 1734m [psy-enabled] on the right):

As you can see, it's a pretty noticeable difference favoring the psy shot, especially in a few key areas. Of note, you can see significantly less blocking on the dark spots on the cloud in the psy shot, and the top platform against the red sky is much sharper.

Now for the numbers:

As expected, Avg. PSNR fell from 42.602 to 41.557 in the psy encode, and Global PSNR fell from 40.655 to 39.641.

Dark Shikari and the Doom9 guys said that encoding speeds would be relatively unchanged, and mine fell only slightly from approx. 20 fps to approximately 18 fps. However, despite using the exact same bitrate option, my filesize increased from 56.6MB to 58.1MB, a difference which may partially explain the picture improvements.

Apparently, the psy options really shine in retaining small details, which are necessarily absent from cartoon sources, so I intend to do another comparison soon using some live-action sources, including at least one with visible film grain.

Stay tuned.

Another potential concern to keep in mind is that, as usual, QuickTime can have trouble with these new options, especially using the Perian codec pack. It can make strange graphical glitches and generally look like shit. Stick to VLC if possible.

Update: I conducted similar tests to those performed on the cartoon source using the Kubrick classic, Dr. Strangelove. This source has some serious film grainage thanks to its age, so we can really see what psy-rd is all about.

I've taken my clips and dropped them into an animated gif, so click on the picture to see the comparison:

The first thing that really jumped out at me is that the psy version is much darker overall, with a much higher contrast ratio (i.e., the blacks are blacker, whites whiter, and the picture looks less gray). It's arguable whether the detail is better in these still-shots with psy-rd added, but it looks much sharper in motion.
Now the difference between the 2 encodings really comes out. Again, the psy-enabled shot is darker overall, with a higher contrast ratio, and the film grain is highly evident. In contrast, the non-psy shot looks as though a smoothing filter has been run over it, steamrolling the details.
Here, just as in the previous comparison, the differences are stark and easily noticeable. Hell, you can almost read the letter through the backside of the page!

Now for the numbers:
As in the earlier experience, avg. PSNR fell from 43.616 in the non-psy encoding to 42.602, while global PSNR fell from 43.371 to 42.347. Again, frames per second remained relatively stable between the 2 encodes.

Conclusions:
After the somewhat underwhelming cartoon-based comparison, I was undecided as to whether I would upgrade to the psy-enabled version, but this comparison has definitely sold me on it. In certain cases that have a lot of small, fine details (such as with Dr. Strangelove), the psychovisual analyses can create a 250 kbps video that rivals an 800-1000 kbps xvid encoding. This means smaller file sizes and faster streaming, which is always a good thing.

Let me know if you have any questions or comments.

For directions on compiling your own binary with the psy options enabled, visit my post here.

Tuesday, September 2, 2008

How to Compile HandBrake GTK GUI from SVN

UPDATE (3/4/09): These directions no longer work on code checked out from the SVN. I've left them here for future reference (and for folks attempting to build the stable 0.9.3 code), but updated build instructions are available here.

Update (5/15/09): I have working binaries available in my PPA repository. Directions for adding it to your package manager are available here.

This information has been gleaned from my and others' posts on the ubuntu forums. I have reproduced it here for ease of access and to avoid having to read all 14+ (at the time of this writing) pages of that thread. This process requires at least a standard Intrepid Ibex (8.10) Ubuntu installation, but it can certainly be adapted for other distros (users of OpenSuSE will need to download the additional dependencies zypper and in to build the CLI, as well as gtkhml2 and gtkhtml2-devel to build the GUI). If you just want precompiled binaries, see my post here.

Update (11/9/08): there's a new dependency added as of the last couple of days: libgtkhtml3.14-dev. I added it to the list, but be sure you install it if you've suddenly run into problems. Update (11/24/08): 2 more dependencies--libgstreamer0.10-dev and libgstreamerplugins-base0.10-dev. Update (2/2/09): another dependency--libdbus-glib-1-dev

Step 1: download the dependencies (in a terminal, type):
sudo aptitude install libgtk2.0-dev libglib2.0-dev libhal-storage1 libhal-storage-dev libhal-dev automake build-essential jam libtool subversion zlib1g-dev libbz2-dev lib64bz2-1.0 libbz2-1.0 xmlto texinfo subversion gfortran doxygen libsdl1.2-dev gfortran-multilib gcc-multilib g++-multilib libesd0-dev libgtk1.2-dev libfftw3-dev electric-fence linux-kernel-devel libgtkhtml3.14-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev yasm intltool libdbus-glib-1-dev
Step 2: install libdvdcss2 from the medibuntu repository (all on one line):
sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list
then type (again, all on one line):
wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update
and finally:
sudo aptitude install libdvdcss2
Step 3: check out the latest HandBrake code from the project's SVN repository:
svn co svn://svn.handbrake.fr/HandBrake/trunk HandBrake
Step 4: navigate to the newly created HandBrake directory:
cd HandBrake
Step 5: compile the HandBrakeCLI binary:
make
Step 6: navigate to the gtk directory:
cd gtk
Step 7: prepare the source to compile the GUI against the newly built CLI binary:
./autogen.sh
Step 8: compile and install the GUI:
make && sudo make install
This should add all the relevant menu shortcuts and so forth on a standard Ubuntu install. Leave me a comment if you have any questions/problems and I'll try to help.

32-bit HandBrake GTK GUI and Yasm deb Binary for Ubuntu

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

Download HandBrake GTK GUI (GHB) 32-bit deb
Download HandBrake GTK GUI (GHB) 64-bit deb
Download Yasm 0.7.1 32-bit deb
(they're compressed into a tarball, but the debs are inside)

Update: These binaries include a new version of the x264 codec that no longer requires as many options to do the same thing. From Dark Shikari (a main x264 developer):
x264 is removing b-rdo and bime options this week. GUI makers, take note, and prepare patches. bime will be enabled at subme >=5 automatically. New subme options will be organized as follows:
subme6: RD on I/P frames
subme7: RD on all frames
subme8: RD refinement on I/P frames
subme9: RD refinement on all frames

new subme 6 = old subme 6
new subme 7 = old subme 6 + b-rdo
new subme 8 = old subme 7 + b-rdo
new subme 9 = didn't exist, RD refinement in B-frames is completely new.

RD refinement in B-frames consists of qpel-RD as in P-frames, and also RD-bime for bidir blocks. Overall the speed cost for this option should be less than old subme6->7 (new subme7->8).

This is a first step on the road to decreasing the number of unnecessary options in x264.
I finally got my 32-bit Hardy Heron build environment set up, so first thing I did was crank out a couple of debs for the yasm assembler and HandBrake's official GTK GUI (known as GHB) from SVN.

Yasm is version 0.7.1 and GHB is version svn 1911.

Leave me a comment if these do/don't work.

Monday, August 11, 2008

How to Compile QT4 GUI For HandBrake

This is a guide for compiling the QT4 GUI for HandBrake that was updated by gonza from the HandBrake forums. I posted this guide on the Ubuntu forums, but I wanted to post it here as well so it could be updated as needed.

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

install git if you don't already have it (sudo aptitude install git-core), along with the dependencies for HandBrakeCLI, then type
git clone git://repo.or.cz/HandBrake.git
navigate to the newly created HandBrake directory:
cd HandBrake
and add the bonne/qhandbrake branch (all on one line):
git checkout --track -b bonne/qhandbrake origin/bonne/qhandbrake
You might need to type git pull just to make sure you're up to date. Then, compile HandBrakeCLI as normal:
make
navigate to the qt4 directory:
cd qt4
and then compile the GUI (dependencies include qt4-core and qt4-dev-tools, I think):
qmake && make
You should be left with a binary named qtHB. This GUI is feature-complete (as far as I can tell), including video previews, a working queue, advanced x264 options, presets, and so on. You can run it by typing ./qtHB into a terminal, or you can copy it into your /usr/bin directory like any other program.

Unfortunately, qtHB *refuses* to bundle into a proper deb binary, but I believe you can download and run my precompiled binaries as long as you have the proper qt4 dependencies installed.
Download 64-bit qtHB
Download 32-bit qtHB

HandBrake GUI Roundup

I previously posted all of this information in a thread at the Ubuntu forums, but I wanted to post it here, too, so I could update it as new versions/information arise. This is a list of all of the GUIs/frontends (that I know of) for HandBrake, along with information regarding features/limitations and development progress for each.

1. GHB is the official GTK GUI that is included with the HandBrake source code (currently just available in the SVN repository). It was written by a swell guy known as JohnAStebbins from the HandBrake forums and it's nearly on par with the OS X GUI in features. However, some people have trouble getting it to compile. On Ubuntu, it requires at least Hardy. (you can download 32-bit and 64-bit deb binaries from here or you can compile from source using HandBrake's official SVN repository [directions here]).

2. qtHB (aka qHandBrake) is an early, yet promising GUI based on the official QT4 HandBrake code that languished in SVN until a HandBrake forums user known as gonza updated it recently. It is still in early development and not yet included in the official SVN code, but it is nearly as complete as GHB and significantly easier to compile (fewer dependencies). (You can read about it here or download and compile it using my instructions here).

3. HandBrakeGTK (aka RippedWire) is a port of a Windows CLI wrapper that requires mono-devel to run. It has a bug that causes it to not pass advanced x264 options to the CLI, which is a total deal-breaker for me. However, its queue feature works well, so it can be useful for batch rips if you don't mind using baseline AVC. It also works well with other codecs, such as xvid, that don't suffer from the adv. x264 opts bug. Another advantage is that you can easily update the HandBrakeCLI backend without affecting the graphical frontend. RippedWire does not appear to be in active development. (read about its history here or download it here. 64-bit users can try this one, which I built from the author's mono project files on my 64-bit system [just install mono-devel, move the executable into the same directory as your HandBrakeCLI binary and type mono HandBrakeGTK.exe]).

4. gHandBrake is a GTK CLI wrapper written from scratch by a guy that goes by Derrik81787. I later added x264 options to it, but I'm a terrible programmer and it ended up really unstable. Derrik seems to have dropped the project because his site for it is gone (Update: he's too busy with classes to work on it now but plans to revisit development in the future), but I would love to see someone tweak the code for better stability (the big issue appears to be with malloc, which I know nothing about; leave a comment if you'd like to pitch in). The major advantage to this frontend is that it is really easy to compile and it has very simple dependencies, unlike the official GTK GUI. (read about it here or download it here).

If you have anything to add, or if you have any corrections, please leave me a comment.

Friday, July 4, 2008

Comparison of x264/h.264 Advanced Options (With Pics)

x264 is a popular (relatively) new codec that's extremely efficient and capable of producing a quality picture even at very low bitrates. Since it has largely replaced DivX/Xvid as the codec of choice for online video, users need to know how to maximize their video quality when using this new codec. The very best quality can only be achieved by tweaking the codec's advanced options, which is a daunting task. Therefore, I have conducted a side-by-side comparison of many of the most useful options so you can decide which ones work best for you (click the pictures to see full-size).
Note: the differences are often quite subtle. If you have trouble spotting them, try looking at the reflection on the fingernails, the quality of the teeth, the edge of the index finger against the black background, and the color gradients on the skin. These are the places where differences will be most apparent.
Some caveats: every video is different and different options work better for different sources. I tried to pick a test video that would be fairly representative of what you would encounter in normal use but your mileage may vary and I make no guarantees about anything. However, I tried to be as transparent and systematic as possible so you can feel free to reproduce my testing if you see fit***. In the comparisons, I will frequently refer to "peak signal-to-noise ratio" (PSNR; higher is better), so if you're interested, you can read up on it at the Wikipedia.

I used HandBrake for my testing, but the results should be equally applicable to mencoder and maybe other programs that support advanced x264 options. Extensive text-based discussions of these advanced options are available at the official HandBrake and Mencoder sites, so I will try not to overlap with them too much. You can think of my comparisons as a visual adjunct to these discussions.

2-Pass Encoding (-2; not really an x264 option, but very important)

The singlemost beneficial thing you can do for a variable bitrate encode is switching from single-pass encoding (the default) to 2-pass encoding. This means the encoder spends the first pass analyzing the file and figuring out where the most bits will be needed (e.g., high-motion scenes) and then it attempts to make the quality similar throughout the file. The drawback to this strategy, of course, is that it takes ~2x as long to finish, but the benefit is large (0.988 dB) and easily noticeable with the naked eye:


Subpixel Motion Estimation (subme / subq)

Subpixel motion esimation (subq) is the next-most-important option. HandBrake defaults to a setting of 4, but I recommend using a setting of 7, which is slow but provides an increase in PSNR of 0.292 dB beyond the default. Using a setting of 1 looks terrible and provides a PSNR hit of -0.325, though it has the benefit of being much faster than the default:



Trellis (trellis)

The trellis option provides the next largest benefit (0.158 dB) at the cost of a somewhat slower encode. Enabling this option also makes a striking difference in fine, hard-edged details, such as on-screen text, as demonstrated by the MPAA rating screen:



I've seen examples online of people using a setting of 2, which they claim improves the quality, but that just isn't supported by these benchmarks, so don't bother using a setting other than 1. Update: turns out using a value of 2 does do something (at the cost of slower encodes), but not unless your subme value is 6+.

The maximum subme value increased from 7 to 9 since I originally wrote this article. I used the new maximum setting for these encodings and they all turned out pretty similar with this highly complex setting, but you can still see some pretty noticeable improvements with each trellis bump. With trellis=0, we can see some blocking and a weird artifact on her tooth; the average encoding speed with this setting was 24.427734 fps. Trellis=1 is a little better but still has some discoloration, while the average encoding speed was inexplicably faster at 24.567919 fps. Trellis=2 is a little smoother than either of the other settings, but it had approximately 20% slower encoding speed, at 19.798990 fps, which may not be worthwhile for all applications.
PSNR was a real mystery to me in this comparison. The average PSNR actually fell as trellis use increased, despite some pretty clear visual improvements. Global PSNR also appears wonky, as it dips slightly with trellis=1 and then increases slightly more with trellis=2. These results really showcase the danger of relying on PSNR for assessment, as it is clearly an imperfect measurement.


Reference Frames (refs)

Next up, we have the number of reference frames. The default is 1, but increasing it to 3 provides an increase in PSNR of .079 dB and increasing it to a value of 5 provides a further increase in PSNR of .043 dB. Unfortunately, diminishing returns sets in quickly and ramping the value up to 12 provides only a further benefit of .034 dB at significant detriment to encoding speed:



Also, be wary of adding too many reference frames, as Quicktime may struggle to play back videos with large amounts. Furthermore, the number of reference frames interacts with the subpixel motion estimation setting to determine how much longer your encoding will take, so experiment to see what works best for you.

Mixed Reference Frames (mixed-refs)

Using >1 reference frames unlocks the ability to use mixed reference frames, which provides a modest improvement in PSNR of .035 dB:



B-Frames (bframes)

In my experience, the b-frames value is roughly as important as the reference frames value with respect to PSNR. Going from the default 0 b-frames to 1 b-frame provides a noticeable 0.12 dB benefit in PSNR. However, diminishing returns are apparent immediately whereby upping the value to 4 provides an additional benefit of only .069 dB, and ramping it up to 16 b-frames actually worsens the picture by -.033 dB:



According to the HandBrake wiki, you can use more b-frames in animated content (~10-15).

Weighted B-Frames and Pyramidal B-Frames

Once you enable ≥1 b-frame, you can also benefit from weighted b-frames and/or pyramidal b-frames. These options provide modest benefits to PSNR of .016 and .01 dB, respectively:




Be aware: enabling pyramidal b-frames is a "high-profile" feature that will totally bork Quicktime playback. Don't use it if you plan on watching your video on a Mac.

Motion Estimation Method (me)

This option determines the method used to estimate motion in your video. Options include hex (hexagon; the default), umh (uneven multi-hexagon; the highest quality), dia (diamond; not shown), and esa (exhaustive; not shown and not to be used). Umh is the highest quality but slightly slower than regular ol' hex; it provides a noticeable .159 dB improvement in PSNR:




Also included in the previous comparison is the motion estimation range (me-range) option, which provides a modest .048 dB improvement in PSNR at the cost of slower encodes.

Analysis (analyse)

The analysis method makes very little difference and I recommend sticking with the default. Switching the value to 'all' provides a miniscule .004 dB increase in PSNR (too small to see).

After enabling the 'all' analysis, you can also enable the use of 8x8 DCT analysis option, which, strangely, hurts PSNR by .07 dB:



No Fast P-Skip (no-fast-pskip)

The HandBrake wiki says this option helps "with blocking on solid colors like blue skies," but I didn't really see any benefit of it in that regard. On the other hand, it reduced the PSNR by a fairly substantial (considering the lack of benefit) -.048 dB, so it's up to you whether or not to use it:



Disabling P-skip might also have more of an impact on animated content, which generally features larger patches of solid colors than live content.

Deblocking Filter (filter)

This option tweaks x264's built-in deblocking filter to smooth out edges--and details--from the picture. It's kind of like using a PhotoShop filter on each frame, with positive values acting like a smoothing filter and negative values acting like a sharpening filter. The default, 0,0, is almost always the best, according to the HandBrake wiki, but I prefer a bit of smoothing and loss of detail (around 2,2). A lot of people (apparently) prefer using negative values, but I think that looks like crap and adds a lot of noise to edges:



Either way, you take a PSNR hit of approximately -.067 dB.

CABAC vs CAVLC

CABAC, the default option for HandBrake, is good for every use except when playback on iPod 5.5G and AppleTV are necessary. If you turn it off (cabac=0), HandBrake will use CAVLC instead, which is visibly detrimental and imparts a tremendous PSNR hit of -1.542(!!):



Direct Prediction (direct)

The HandBrake wiki makes this option sound really important, but changing the values had absolutely no effect on PSNR or appearance in my experience, so I suggest just leaving it alone:



Turbo First Pass (-T; again, not an advanced x264 option, but very important)

The turbo option significantly speeds-up the first pass of a 2-pass encoding by passing faster options for the first pass and slower, higher-quality options for the second pass. The HandBrake wiki mentions that the turbo option may slightly reduce quality, but my experience was exactly the opposite. Due to the nature of the option, I compared an encoding with a variety of slow, high-quality options with and without the turbo option included. Interestingly, this resulted in an increase in PSNR of .054 dB in the turbocharged encode:



Bidirectional Refinement (bime)

The next option, bidirectional refinement, depends on several other options that I couldn't readily identify, so I did a similar comparison to what I used with the turbo comparison (i.e., a variety of slow, high-quality options with and without bime). This yielded a PSNR benefit of .014 with bime turned on, which is probably unnoticeable, but worthwhile if you're looking for the best quality possible:



Threads (threads)

The threads option just allows you to specify the number of threads HandBrake will use while encoding. This makes no difference on single-core processors, but makes a huge difference on multicore processors. HandBrake defaults to automatically decide the "optimum" number, but I find I get slightly higher processor utilization if I assign a value of 4 on my dual-core AMD X2 4000+.

Deinterlacing (-d; not an x264 option, but it's pretty important so I figured I'd throw it in)

Interlaced video (as opposed to 'progressive') has a bunch of crazy-looking lines--known as scanlines or combing--in some frames and it makes videos look awful. If you want to get rid of them, HandBrake has a really great built-in deinterlacing filter that fixes things right up. I tried comparing "-d slowest" with plain ol' "-d" and they were exactly the same, so I've only included a comparison with and without the vanilla "-d" option:


Of note, deinterlacing does not affect PSNR, though it does significantly impair picture quality and introduces 'jaggies.'

Update (5/4/09): HandBrake has recently introduced a new option known as Decombing, which analyzes each frame and detects combing and only runs the deinterlacing filter on those specific frames. This is a huge improvement over regular ol' deinterlacing because it only affects the necessary frames and leaves the other intact. Furthermore, you can leave this option on all the time (i.e., include it in custom presets) and it will only kick in when necessary.

Adaptive B-frames: (b-adapt)

If I understand correctly, this setting analyzes your file and determines the optimum number of b-frames, up to the maximum you specify in the b-frames field. This means you can crank your b-frames setting up to, say, 16 and it will only use as many as it feels are necessary (almost never more than 10, so 16 is overkill, but you get the idea). Within this setting, you have 3 choices: 'none,' 'fast,' and 'optimum.' 'Fast' is much better than 'none' and 'optimum' is very slightly better than 'fast.' Importantly, 'fast' only imparts a slightly longer encoding speed than 'none,' while going from 'fast' to 'optimum' will drop your encoding speed by 50%(!!!) for a largely unnoticeable difference. Therefore, I recommend using 'fast' unless you're a real stickler for whom time is no object. I hope to get a comparison of these options up here in the near future, so stay tuned.

My personal settings

Each of these options may not provide much benefit on their own, but the improvements certainly add up. My settings result in a high-quality rip (PSNR of 41.232 vs 35.933 with the default options; approximately 13% improvement) that remains (mostly) compatible with Quicktime but with a 60-80% worsening in encoding speed (from ~60 frames per second with default options to ~15 frames per second with my settings). Here is a comparison of the vanilla x264 options and my personal settings so you can decide for yourself if it's worthwhile for you:



I also use a higher bitrate (800-900 kb/s) when doing my actual encodings (not in this comparison), which obviously improves quality a great deal. Here are my x264 options:

ref=5:mixed-refs:bframes=4:bime:weightb:b-rdo
:me=umh:subme=9:filter=0,-1:trellis=2:threads=4
I also use the non-x264 options of -2 and -T. Update (5/4/09): I used to be all about the 2-pass encoding, but I now prefer constant quality with CRF enabled. It takes the same time as 2-pass overall but gives consistent quality that is slightly better throughout the file, in my experience. I hope to post a comparison sometime in the near future.
***My source video is a preview for Jackie Brown that appears on a Pulp Fiction collector's edition something-or-other. All comparisons used frame 460 (or 1120 for deinterlacing comparisons). For testing, I used a bitrate of 300 kb/s, which was intentionally low so the benefits of different options would hopefully be more visible. Nevertheless, x264 is so good at preserving quality that I had to zoom in 400% to get a good look at the details. So as not to introduce artifacts from upscaling and/or jpg compression, I used png (lossless) screencaptures at the native resolution for all initial adjustments (zooming, compositing, etc) and only switched to jpg (lossy) at the very end for posting online. I placed all streams into a matroska mkv container and used AC3 passthrough for the audio.

Tuesday, May 27, 2008

CPU-optimized Backend for gHandBrake

Edit: The original author of gHandBrake's site appears to be down and I haven't heard from him since I submitted code to add most of the advanced x264 options to the interface, so I decided to offer my updated version right here (64-bit binary plus source code) until further notice. If you need to compile it for a 32-bit system, I have a list of dependencies for HandBrake here, and precompiled binaries of Yasm Debian/Ubuntu systems here. If you have any problems with any of it, just let me know in the comments and I'll try to help.

You can also download the unmodified gHandBrake (fewer features, but more stable) here. This package includes both the source tarball and a precompiled 64-bit deb binary for Ubuntu/Debian users.

Original post: I've been trying out gHandBrake, the excellent new GTK frontend for HandBrake. It's very early in development, so it doesn't have many of the advanced options--such as advanced x264 options--implemented yet but it seems quite solid and will eventually be a great replacement for HandBrakeGTK/RippedWire, which is basically just an incomplete port of the Windows GUI (of note, HandBrakeGTK does not actually pass advanced x264 options to the commandline).

Unfortunately, the precompiled binaries offered by the author do not appear to utilize HandBrake's processor-specific optimizations, so I decided to offer an optimized 64-bit build of the backend that would:
Download link

To use it, just download and extract, then replace the old ghandbrake-backend with the newer version. In a console, you would type (assuming it was extracted to your desktop):
sudo mv -f ~/Desktop/ghandbrake-backend /usr/bin
Since switching to the updated backend, my average encoding speed went from ~40 fps to ~115 fps, but YMMV.

Of course, if you compiled gHandBrake yourself from source, you would already have the optimizations included as long as you had the yasm assembler installed when you compiled. If you want to go this route, I have a precompiled yasm binary that recognizes SSE3 instructions in my previous post.

Please leave me a comment if you have any questions or concerns.

Tuesday, May 13, 2008

HandBrake 0.9.2 and yasm 0.7.1 precompiled deb binary for 64-bit linux

Update (5/15/09): The binaries on this page are woefully out of date, but I have working binaries built from the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

Download: HandBrakeCLI-AMD64 0.9.2 (Thanks Alexander!)
Download: yasm-AMD64 0.7.1
yasm-i386_0.7.1 (it's in a tarball, but the deb is inside)

I just got finished doing some building for HandBrakeCLI. They used to provide a precompiled 64-bit binary on the official site, but they've stopped for some unknown reason, so I decided to provide a copy of my own.

This binary doesn't need to be installed. Just navigate to the directory that contains it and type
./HandBrakeCLI
followed by any desired options (for more information on using the command line with HandBrake, visit the HandBrake wiki).

One of the biggest pains in compiling HandBrake (other than the lack of a comprehensive list of dependencies) is the fact that, to get the most out of HandBrake's processor-specific optimizations, you have to have the yasm assembler installed. Unfortunately, the version in Ubuntu's repos only supports instructions up to SSE2.

To correct this, I had to download and compile a newer version of yasm (namely 0.7.0) before I built HandBrake. Once I got that going, HandBrake recognized my cpu's extensions and accelerated the encoding speed to approximately 3.5-4x faster than the stock, non-yasm compile.

This binary should recognize any extensions present on my cpu: an AMD X-2 4000+ cpu, which has all the usual goodies up to SSE3 (so you fancy-pants core2duo users are out of luck on the SSSE3). Edit: A fella named Alexander was kind enough to supply a build with additional support for SSSE3 and Cache_64. Thanks Alexander!

My binaries include the original source code, which I have not modified. If you have any problems with them, please leave a comment or drop me an email and I'll see if I can correct the problem.

Analytics Tracking Footer