HandBrake with Live Video Previews

The HandBrake team has released version 0.9.3, which includes official, sanctioned builds of the GTK GUI in both 32- and 64-bit formats. This new version also brings a number of improvements that have been enjoyed by those of us using the SVN builds for some time to the rest of the HB users. However, new features are constantly being added, and the codec pool is always being updated, so I will continue to post the latest bleeding-edge builds here on my site, starting with svn1952, which includes live video previews (!), as well as a more recent version of the x264 and ffmpeg codecs.

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

Here's how the live preview looks. You can really get in there and see the direct impact of your changes. This leads to better cropping and a better sense for your end product. (this is my Mac x-forwarding GHB from my Ubuntu box. As you can see, the live previews even work through that):

For those of you who wish to compile on your own hardware (recommended for those with newer Core 2 Duo CPUs), the process is exactly the same as my previous SVN instructions, except for the addition of 2 new dependencies: libgstreamer0.10-dev and libgstreamer-plugins-base0.10-dev. I've updated the instructions there to reflect the change.

How to run ZSNES Super Nintendo Emulator on 64-bit Ubuntu Linux

ZSNES is my favorite SNES emulator in the Windows world, but it can be a real pain to get going in 64-bit Linux. Most tutorials will suggest you set up a chroot jail and maintain a parallel set of 32-bit libs so that you can run the 32-bit version, but this is both a hassle and overly complex for many users.

I tried to compile my own binaries from source to avoid this, but I kept running into architecture errors and ended up throwing in the towel after a few hours of fiddling. Luckily, a ZSNES developer known as Nach has provided specialized precompiled binaries for a variety of architectures that worked a treat for me.

Just follow this link, scroll down a bit, and then download the binary that matches your architecture. I have an Athlon 64 X-2 4000+, so I selected the Athlon64-SSE3 binary.

Next, install libsdl-dev either through Synaptic or by typing sudo aptitude install libsdl-dev into a terminal.

Once downloaded and decompressed, you should find your binary inside, which you can run by double-clicking it or typing ./zsnes into a terminal. If you double-click and nothing happens, try running it through the terminal to spot any errors. I personally encountered this error:

Unable to poll /dev/input/event8. Make sure you have read permissions to it

repeated 9 times (1 for each /dev/input/event* 0-8).

I was able to get around this by running ZSNES as root (i.e., sudo ./zsnes), which is certainly not ideal, but I haven't found a way around it yet.

If you take this route, be aware that the default location for saved games will not be ~/.zsnes as it normally would be. Instead, it will be located in /root/.zsnes (since you're running as root). This in mind, you may have to copy your .srm files into this directory for them to be recognized. When I first got my copy to run, I couldn't get it to recognize any of my saved files, even though I tried changing the path for saved files under the preferences. However, once I copied the files into the /root/.zsnes directory, everything showed up just dandy.

Using the native 64-bit binary referenced above also had another unexpected (perhaps coincidental) positive effect of making my gamepad's d-pad work correctly. Using dfreer's zsnes32 binary worked well for me in most respects, but it just wouldn't recognize my Logitech Precision gamepad's d-pad. The buttons worked fine, but when it came time to assign the directions, it would just sit there dumbly, even though jscalibrator and cat /dev/input/js0 both showed plenty of action. This problem remained no matter how many kernel modules I loaded (usbhid, analog, etc.), until I tried Nach's binary. Now it works just fine. Go figure...

This information was written for Ubuntu 8.10 Intrepid Ibex, but it should be applicable to other distros as well.

Intrepid Ibex broke my Netatalk

I mostly use Macs, but all of my media files are stored on a home server that runs Ubuntu. I had been using Gutsy and sharing my files using netatalk, an open source implementation of Apple's AppleTalk protocol, and everything was working swimmingly.

However, as soon as the beta for the newest version of Ubuntu--known as Intrepid Ibex--was released, I upgraded to it and was shocked to find that I could no longer connect through netatalk using my Macs (they're running 10.5 Leopard). Every time I tried, it would fail with this error:

A volume failed to mount.

The volume [directory name] could not be mounted

I fiddled with configurations on both ends to no avail, so I decided to try a workaround: I uninstalled the version of netatalk from the Intrepid repos (sudo aptitude remove netatalk) and then downloaded outdated deb binaries for netatalk and its only other major dependency, libdb4.2, from the Gutsy repos.

Install (sudo dpkg -i [package name]) libdb4.2 first (it'll give you a warning about downgrading from a higher version), then netatalk, and then let the service start. You should now be able to connect to your shares again.

Leave me a comment if this doesn't fix your problem and I might be able to help.

How To Compile HandBrake With New Psy Options Enabled

Update: It looks like the new x264 version is committed to the SVN repo now, so you should be able to just perform Steps 1, 2, and 5 (i.e., a normal SVN checkout and compile) to get the new psychovisual options.

Update (5/15/09): I have working binaries available in my PPA repository. Directions for adding it to your package manager are available here.

Here are directions to download and install the latest development code for HandBrake that can take advantage of the revolutionary new psychovisual additions to the x264 codec (known as psychovisual rate distortion [psy-rd] and psychovisual trellis [psy-trellis]).

Step 1: check out the latest source code from SVN:

svn co svn://svn.handbrake.fr/HandBrake/trunk HandBrake

Step 2: navigate to the new directory

cd HandBrake/contrib

Step 3: edit the download location for the x264 codec in your favorite text editor (I prefer nano):

nano version_x264.txt

then, either delete or comment out (put a # in front of it) the address that is already there and replace it with this:

http://download.m0k.org/handbrake/contrib/x264-r979-6d4af8d.tar.gz

Step 4: return to the HandBrake directory:

cd ..

If you just want to use the CLI version (Linux users have no choice), you're almost finished and can go straight to Step 5. If you are using OS X or Windows, though, you can update the GUI interfaces to reflect these new options by applying the appropriate patch (for OS X or for Windows)

To apply the patch, move it into the HandBrake directory and type into a terminal:

patch -p0

Step 5: compile as usual:

make

That should get you all set. The new options require a subme value of ≥6, which should be the new default (and should be reflected in the GUIs and presets). If you have any questions or concerns, leave me a comment and I'll try to help.

HandBrake With New Adv. x264 psy-rd

Update: I just finished conducting some tests using live, film-grained content and the details are posted below the cartoon data.

x264 just committed some fancy new features, known as psychovisual rate distortion and psychovisual trellis, to their git repository. These features are supposed to produce a subjectively superior picture by maintaining detail and minimizing blurriness during the encoding process (I guess; read more about it here, at the blog of Dark Shikari, a developer for the x264 codec).

However, if you read the posts at the Doom9 forums linked in his blog post, they mention that Peak Signal To Noise Ratio (PSNR)--basically the only objective measure of picture distortion available to me--will always be negatively affected by the new features, and freeze-frames will also look worse. This makes the new features sound a little like voodoo to me, since I tend to favor objective measures over subjective ones. Regardless, I decided to try it out.

I used a bitrate of 250 in an mp4 container with these adv. x264 settings:

ref=5:mixed-refs:bframes=6:bime:weightb:b-rdo:me=umh:subme=7:
filter=0,-1:trellis=2:threads=2

For the psy encoding, I followed those settings with these psy-specific options (first number sets the psychovisual rate distortion [value from 0.1-1] while the second controls the psychovisual trellis [again, 0.1-1]):

psy-rd=1,1

If I understood things correctly, psy-rd only works with a subme setting ≥6, btw, so keep that in mind if you want to use the psy settings. Here are a couple of screenshots from an episode of Futurama encoded using otherwise identical settings (v0.9.2 on the left [no psy], svn 1734m [psy-enabled] on the right):

(Click to enlarge or click here to see the comparison anim00ted)

As you can see, it's a pretty noticeable difference favoring the psy shot, especially in a few key areas. Of note, you can see significantly less blocking on the dark spots on the cloud in the psy shot, and the top platform against the red sky is much sharper.

Now for the numbers:

As expected, Avg. PSNR fell from 42.602 to 41.557 in the psy encode, and Global PSNR fell from 40.655 to 39.641.

Dark Shikari and the Doom9 guys said that encoding speeds would be relatively unchanged, and mine fell only slightly from approx. 20 fps to approximately 18 fps. However, despite using the exact same bitrate option, my filesize increased from 56.6MB to 58.1MB, a difference which may partially explain the picture improvements.

Apparently, the psy options really shine in retaining small details, which are necessarily absent from cartoon sources, so I intend to do another comparison soon using some live-action sources, including at least one with visible film grain.

Stay tuned.

Another potential concern to keep in mind is that, as usual, QuickTime can have trouble with these new options, especially using the Perian codec pack. It can make strange graphical glitches and generally look like shit. Stick to VLC if possible.

Update: I conducted similar tests to those performed on the cartoon source using the Kubrick classic, Dr. Strangelove. This source has some serious film grainage thanks to its age, so we can really see what psy-rd is all about.

I've taken my clips and dropped them into an animated gif, so click on the picture to see the comparison:

The first thing that really jumped out at me is that the psy version is much darker overall, with a much higher contrast ratio (i.e., the blacks are blacker, whites whiter, and the picture looks less gray). It's arguable whether the detail is better in these still-shots with psy-rd added, but it looks much sharper in motion.

Now the difference between the 2 encodings really comes out. Again, the psy-enabled shot is darker overall, with a higher contrast ratio, and the film grain is highly evident. In contrast, the non-psy shot looks as though a smoothing filter has been run over it, steamrolling the details.

Here, just as in the previous comparison, the differences are stark and easily noticeable. Hell, you can almost read the letter through the backside of the page!

Now for the numbers:

As in the earlier experience, avg. PSNR fell from 43.616 in the non-psy encoding to 42.602, while global PSNR fell from 43.371 to 42.347. Again, frames per second remained relatively stable between the 2 encodes.

Conclusions:

After the somewhat underwhelming cartoon-based comparison, I was undecided as to whether I would upgrade to the psy-enabled version, but this comparison has definitely sold me on it. In certain cases that have a lot of small, fine details (such as with Dr. Strangelove), the psychovisual analyses can create a 250 kbps video that rivals an 800-1000 kbps xvid encoding. This means smaller file sizes and faster streaming, which is always a good thing.

Let me know if you have any questions or comments.

For directions on compiling your own binary with the psy options enabled, visit my post here.

How to Compile HandBrake GTK GUI from SVN

UPDATE (3/4/09): These directions no longer work on code checked out from the SVN. I've left them here for future reference (and for folks attempting to build the stable 0.9.3 code), but updated build instructions are available here.

Update (5/15/09): I have working binaries available in my PPA repository. Directions for adding it to your package manager are available here.

This information has been gleaned from my and others' posts on the ubuntu forums. I have reproduced it here for ease of access and to avoid having to read all 14+ (at the time of this writing) pages of that thread. This process requires at least a standard Intrepid Ibex (8.10) Ubuntu installation, but it can certainly be adapted for other distros (users of OpenSuSE will need to download the additional dependencies zypper and in to build the CLI, as well as gtkhml2 and gtkhtml2-devel to build the GUI). If you just want precompiled binaries, see my post here.

Update (11/9/08): there's a new dependency added as of the last couple of days: libgtkhtml3.14-dev. I added it to the list, but be sure you install it if you've suddenly run into problems. Update (11/24/08): 2 more dependencies--libgstreamer0.10-dev and libgstreamerplugins-base0.10-dev. Update (2/2/09): another dependency--libdbus-glib-1-dev

Step 1: download the dependencies (in a terminal, type):

sudo aptitude install libgtk2.0-dev libglib2.0-dev libhal-storage1 libhal-storage-dev libhal-dev automake build-essential jam libtool subversion zlib1g-dev libbz2-dev lib64bz2-1.0 libbz2-1.0 xmlto texinfo subversion gfortran doxygen libsdl1.2-dev gfortran-multilib gcc-multilib g++-multilib libesd0-dev libgtk1.2-dev libfftw3-dev electric-fence linux-kernel-devel libgtkhtml3.14-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev yasm intltool libdbus-glib-1-dev

Step 2: install libdvdcss2 from the medibuntu repository (all on one line):

sudo wget http://www.medibuntu.org/sources.list.d/hardy.list -O /etc/apt/sources.list.d/medibuntu.list

then type (again, all on one line):

wget -q http://packages.medibuntu.org/medibuntu-key.gpg -O- | sudo apt-key add - && sudo apt-get update

and finally:

sudo aptitude install libdvdcss2

Step 3: check out the latest HandBrake code from the project's SVN repository:

svn co svn://svn.handbrake.fr/HandBrake/trunk HandBrake

Step 4: navigate to the newly created HandBrake directory:

cd HandBrake

Step 5: compile the HandBrakeCLI binary:

make

Step 6: navigate to the gtk directory:

cd gtk

Step 7: prepare the source to compile the GUI against the newly built CLI binary:

./autogen.sh

Step 8: compile and install the GUI:

make && sudo make install

This should add all the relevant menu shortcuts and so forth on a standard Ubuntu install. Leave me a comment if you have any questions/problems and I'll try to help.

32-bit HandBrake GTK GUI and Yasm deb Binary for Ubuntu

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

Download HandBrake GTK GUI (GHB) 32-bit deb
Download HandBrake GTK GUI (GHB) 64-bit deb
Download Yasm 0.7.1 32-bit deb
(they're compressed into a tarball, but the debs are inside)

Update: These binaries include a new version of the x264 codec that no longer requires as many options to do the same thing. From Dark Shikari (a main x264 developer):

x264 is removing b-rdo and bime options this week. GUI makers, take note, and prepare patches. bime will be enabled at subme >=5 automatically. New subme options will be organized as follows:

subme6: RD on I/P frames

subme7: RD on all frames

subme8: RD refinement on I/P frames

subme9: RD refinement on all frames

new subme 6 = old subme 6

new subme 7 = old subme 6 + b-rdo

new subme 8 = old subme 7 + b-rdo

new subme 9 = didn't exist, RD refinement in B-frames is completely new.

RD refinement in B-frames consists of qpel-RD as in P-frames, and also RD-bime for bidir blocks. Overall the speed cost for this option should be less than old subme6->7 (new subme7->8).

This is a first step on the road to decreasing the number of unnecessary options in x264.

I finally got my 32-bit Hardy Heron build environment set up, so first thing I did was crank out a couple of debs for the yasm assembler and HandBrake's official GTK GUI (known as GHB) from SVN.

Yasm is version 0.7.1 and GHB is version svn 1911.

Leave me a comment if these do/don't work.

How to Hack Mario Kart Wii Online for Unlimited Items Including Blue Shells

Update: looks like it's easier to cheat nowadays. If you install the homebrew channel, you can enable cheats from there. I don't know the specifics, so feel free to leave a comment if you know any more than that.

Update (6/19/09): from reader Rosalina, you can indeed hack it from the homebrew channel:

Materials:
A Homebrew Channel on your Nintendo Wii
An SD Card
Ocarina in your SD Card
Legend of the Twilight Princess for Wii

Use Legend of the Twilight Princess game to get Homebrew Channel and then
Go to File and then open TXT file. Then select rmce01 is Mario Kart Wii's ID. Then go to add a code/comment. Type in as many as you want but make sure you call it what it is called in the Mario Kart Wii Instructions Manual. And make sure you put “Infinite” before the power you want (the most fun powers are Star, Mega Mushroom, Triple Red Turtle Shells, Triple Green Turtle Shells, Bullet Bil l(the rocket) and some time Triple Banana, Fake Item Box and Bomb-omb.) OK? Now type the codes in the code box {codes are at www.usbgecko.com/codes}. Insert the SD Card into the PC and then go to 'Export to GCT' and select your SD Card. Insert the SD Card into your Wii and go to the Homebrew Channel and press start. You will be able to hack Mario Kart on the Homebrew Channel. My guess to stop hacking Mario Kart is to take your SD Card out of your Wii.

This information is out of date, but I'll leave it here for posterity:
I did some digging and this guy who goes by the handle /dexter0 (aka ihack@you) appears responsible for this nonsense (46 pageviews when I accessed it), but I could be wrong. It appears this hack is accomplished using a USB Gecko attachment, which is pretty much like an Action Replay or a GameShark.

He basically just edits the database values that represent your item box status to choose what item you have at any point (resulting in unlimited items).

From /dexter0 (aka ihack@you):

These codes were hacked under NTSC-USA. And since they involve assembly hacking, they are not easily ported. So don't ask for PAL or JP codes.

[Write Char Offset]
D27E4DDC 00000002
3FE08000 93BF1500
3BE00000 00000000

[Always Have Item *Works anywhere] - Add Your Own Button Activator
48000000 809BEE20
DE000000 80008180
58010000 00000014
DE000000 80008180
48100000 80001500
DE000000 80008180
1400008C 000000XX
14000090 0000000Y
E0000000 80008000

...you need both of them for it to work. The Write Char Offset must come first! And don't click the apply button while a race is active, you'll crash them game.

Have fun. Try not to be a dick.

If you wish to report a cheater, drop a comment on my Wall of Shame page with their name and a description of their Mii (if available).

How to Enable Surround Sound in Ubuntu Linux

Update (04/21/10): The information in this original post is no longer applicable since Ubuntu switched to the PulseAudio sound infrastructure. I'm leaving it all here for reference, but here's how to do it now:

Step 1: open up a terminal (Applications > Accessories > Terminal or Alt+F2 and type 'gnome-terminal' into the command applet)

Step 2: at the command prompt, type alsamixer

Your prompt should change to something like this:

Step 3: Scroll all the way to the right (using the arrow keys) until you reach a section labeled "Channel." It will most likely say "2ch" over it. Press the 'up' arrow key until it says 6ch (for 5.1 surround) or 8ch (for 7.1 surround).

Exit the alsamixer program by pressing ctrl+C. You should be all set. You may have to go into your Sound Preferences and change either the 'Profile' setting on the 'Hardware' tab or fiddle with the devices in the 'Output' tab. I didn't have to do this, but you might.

If you have any problems, leave me a note in the comments and I'll try to help.

Update (10/08/08): I was using Gutsy when I wrote this post, but I have since upgraded to Intrepid Ibex (8.10) and it still works for me. YMMV

I've been hard at work lately trying to get my motherboard's (an Asus M2V-MX SE socket AM2) 6-channel surround sound working with Linux and it's been a surprisingly stubborn pain in my ass. However, I finally succeeded this past weekend and decided to document my process here for others to hopefully benefit from.

The first complication I ran into when trying to get surround sound working is that my mobo does not have the normal multichannel output jacks. That is, instead of having the blue, orange, and black plugs that are in most multichannel setups, it instead just has what appears to be standard blue, pink, and green jacks that are usually used for stereo output, mic-input, and line-in, respectively.

What is poorly documented, though, is that the pink and green jacks can double as the orange and black plugs when enabled by software. It's a stupid way of doing things, IMHO, but low-end mobo manufacturers apparently do it to save costs.

Anyway, in Windows, enabling this hidden functionality was accomplished using the Realtek Sound Manager program that was installed with my sound driver. However, as with damn-near all driver utilities, it does not have a Linux equivalent.

Instead, open the Alsa-mixer GUI by double-clicking on the speaker icon in the upper-right of the screen (a.k.a., the Volume Control applet):

Then select 'Preferences' from the 'Edit' pull-down menu:

From here, you'll want to scroll down a bit and check the 'Channel Mode' box, and you'll probably want to check the 'Surround,' 'Front,' 'LFE,' and 'Center' boxes as well:

When you go back to the main applet, click on the 'Options' tab, which should now have an option to choose the number of channels:

Select '6ch' for a 5.1 setup or '8ch' in a 7.1 setup. This should unlock the hidden functionality of those pink and green jacks. On my system, the pink jack converts to the orange function and the green converts to the black function, but YMMV.

You could stop here, but I suggest doing one more step to make sure everything is configured properly. Open up a terminal and type:

speaker-test -Dplug:surround51 -c6 -twav

This will make a creepy disembodied voice play through each channel in succession so you can be sure your speakers are plugged into the right jacks.

Btw, this command is for a 5.1 / 6-channel surround card. A 7.1 system will use this command instead:

speaker-test -Dplug:surround71 -c8 -twav

For some people (like me), the system automatically duplicates the front channels' sound to the back channels for listening to stereo sources, such as mp3s, normal non-DVD video files, etc. Other people, however, will need to do a bit more to make this happen. If you're one of these people, open up a terminal and type:

gedit ~/.asoundrc

Whether this file is blank or already has some stuff in it, just skip down to the bottom and copypasta this in (courtesy of the Gentoo wiki):

pcm.!dmix {
type dmix
ipc_key 1024
slave {
pcm "hw:0,0"
channels 6
period_size 512
buffer_size 1024
}
}
pcm.!default {
type plug
slave.pcm "dmix"
slave.channels 6
route_policy duplicate
}

You may have to change "hw:0,0" to something else, depending on your sound card's location. When you restart, surround signals will still work like before, but stereo signals should be duplicated. If, on the other hand, you have no sound or your sound is weird, just reopen the ~/.asoundrc file and delete the part you added (or try one of the other configurations from the aforementioned Gentoo wiki). No big deal.

Let me know in the comments if this works for you or not or if you have any questions.

How to Compile QT4 GUI For HandBrake

This is a guide for compiling the QT4 GUI for HandBrake that was updated by gonza from the HandBrake forums. I posted this guide on the Ubuntu forums, but I wanted to post it here as well so it could be updated as needed.

Update (5/15/09): I have working binaries of the latest code available in my PPA repository. Directions for adding it to your package manager are available here.

install git if you don't already have it (sudo aptitude install git-core), along with the dependencies for HandBrakeCLI, then type

git clone git://repo.or.cz/HandBrake.git

navigate to the newly created HandBrake directory:

cd HandBrake

and add the bonne/qhandbrake branch (all on one line):

git checkout --track -b bonne/qhandbrake origin/bonne/qhandbrake

You might need to type git pull just to make sure you're up to date. Then, compile HandBrakeCLI as normal:

make

navigate to the qt4 directory:

cd qt4

and then compile the GUI (dependencies include qt4-core and qt4-dev-tools, I think):

qmake && make

You should be left with a binary named qtHB. This GUI is feature-complete (as far as I can tell), including video previews, a working queue, advanced x264 options, presets, and so on. You can run it by typing ./qtHB into a terminal, or you can copy it into your /usr/bin directory like any other program.

Unfortunately, qtHB *refuses* to bundle into a proper deb binary, but I believe you can download and run my precompiled binaries as long as you have the proper qt4 dependencies installed.
Download 64-bit qtHB
Download 32-bit qtHB

HandBrake GUI Roundup

I previously posted all of this information in a thread at the Ubuntu forums, but I wanted to post it here, too, so I could update it as new versions/information arise. This is a list of all of the GUIs/frontends (that I know of) for HandBrake, along with information regarding features/limitations and development progress for each.

1. GHB is the official GTK GUI that is included with the HandBrake source code (currently just available in the SVN repository). It was written by a swell guy known as JohnAStebbins from the HandBrake forums and it's nearly on par with the OS X GUI in features. However, some people have trouble getting it to compile. On Ubuntu, it requires at least Hardy. (you can download 32-bit and 64-bit deb binaries from here or you can compile from source using HandBrake's official SVN repository [directions here]).

2. qtHB (aka qHandBrake) is an early, yet promising GUI based on the official QT4 HandBrake code that languished in SVN until a HandBrake forums user known as gonza updated it recently. It is still in early development and not yet included in the official SVN code, but it is nearly as complete as GHB and significantly easier to compile (fewer dependencies). (You can read about it here or download and compile it using my instructions here).

3. HandBrakeGTK (aka RippedWire) is a port of a Windows CLI wrapper that requires mono-devel to run. It has a bug that causes it to not pass advanced x264 options to the CLI, which is a total deal-breaker for me. However, its queue feature works well, so it can be useful for batch rips if you don't mind using baseline AVC. It also works well with other codecs, such as xvid, that don't suffer from the adv. x264 opts bug. Another advantage is that you can easily update the HandBrakeCLI backend without affecting the graphical frontend. RippedWire does not appear to be in active development. (read about its history here or download it here. 64-bit users can try this one, which I built from the author's mono project files on my 64-bit system [just install mono-devel, move the executable into the same directory as your HandBrakeCLI binary and type mono HandBrakeGTK.exe]).

4. gHandBrake is a GTK CLI wrapper written from scratch by a guy that goes by Derrik81787. I later added x264 options to it, but I'm a terrible programmer and it ended up really unstable. Derrik seems to have dropped the project because his site for it is gone (Update: he's too busy with classes to work on it now but plans to revisit development in the future), but I would love to see someone tweak the code for better stability (the big issue appears to be with malloc, which I know nothing about; leave a comment if you'd like to pitch in). The major advantage to this frontend is that it is really easy to compile and it has very simple dependencies, unlike the official GTK GUI. (read about it here or download it here).

If you have anything to add, or if you have any corrections, please leave me a comment.

How To X-Forward On a Mac

If you're a terminal-jockey and you use a Mac, you've probably come across the term 'X-forwarding' before, but very few places will actually tell you what it is or how to do it.

The X Windowing System is an open source window manager that's used to draw windows in most free operating systems, including GNU/Linux, the *BSDs, and Solaris. It was ported to OS X in the form of the XQuartz project and is distributed with Tiger and Leopard install DVDs. You can also download it here if you're using Leopard (10.5), here if you're using Tiger (10.4), or here if you're using Panther (10.3).

Once it's installed, you can do a lot of cool stuff. For instance, you can run many more open source programs than before (check out Fink and Darwinports), and (more importantly for me) you can X-forward from any other computer that is currently running X.

What that means is that you can open an instance of an X-native program from another computer using your Mac and it will seem (to you) as though it were running on your Mac (i.e., you will have a window appear on your desktop that you can move, minimize, etc., but it uses the other computer's resources). It ends up being sort of like a remote desktop--such as VNC--but faster and without affecting the experience of local users. For example, I use X-forwarding to run programs from my Linux media center while my wife watches videos.

To do it, the remote computer you want to X-forward from has to have an SSH client--such as OpenSSH--installed. Once that is in place, open the Terminal.app on your Mac (located in Applications > Utilities by default) and type:

ssh -X username@target.IP.address.here

You have to use an account that already exists on the remote computer. It will then ask for the account's password. You will then be logged-in to the remote machine via your local terminal. Your command prompt will also change to reflect this:

Now, you can run programs via this terminal and any windows will be drawn on your local computer (i.e., your Mac) instead of the remote computer.

Here is what it looks like when I forward the Linux QT4 GUI for HandBrake to my Mac:

As you can see, the widgets look a little weird, and the font is clunky, but it's fully functional and has much lower latency than other remote access methods. Also, thanks to the ssh connection (also known as an 'ssh tunnel'), your connection cannot be spied upon, even if you X-forward across the Internet. Leave me a comment if you have any questions or run into any problems.

Comparison of x264/h.264 Advanced Options (With Pics)

x264 is a popular (relatively) new codec that's extremely efficient and capable of producing a quality picture even at very low bitrates. Since it has largely replaced DivX/Xvid as the codec of choice for online video, users need to know how to maximize their video quality when using this new codec. The very best quality can only be achieved by tweaking the codec's advanced options, which is a daunting task. Therefore, I have conducted a side-by-side comparison of many of the most useful options so you can decide which ones work best for you (click the pictures to see full-size).

Note: the differences are often quite subtle. If you have trouble spotting them, try looking at the reflection on the fingernails, the quality of the teeth, the edge of the index finger against the black background, and the color gradients on the skin. These are the places where differences will be most apparent.

Some caveats: every video is different and different options work better for different sources. I tried to pick a test video that would be fairly representative of what you would encounter in normal use but your mileage may vary and I make no guarantees about anything. However, I tried to be as transparent and systematic as possible so you can feel free to reproduce my testing if you see fit***. In the comparisons, I will frequently refer to "peak signal-to-noise ratio" (PSNR; higher is better), so if you're interested, you can read up on it at the Wikipedia.

I used HandBrake for my testing, but the results should be equally applicable to mencoder and maybe other programs that support advanced x264 options. Extensive text-based discussions of these advanced options are available at the official HandBrake and Mencoder sites, so I will try not to overlap with them too much. You can think of my comparisons as a visual adjunct to these discussions.

2-Pass Encoding (-2; not really an x264 option, but very important)

The singlemost beneficial thing you can do for a variable bitrate encode is switching from single-pass encoding (the default) to 2-pass encoding. This means the encoder spends the first pass analyzing the file and figuring out where the most bits will be needed (e.g., high-motion scenes) and then it attempts to make the quality similar throughout the file. The drawback to this strategy, of course, is that it takes ~2x as long to finish, but the benefit is large (0.988 dB) and easily noticeable with the naked eye:


Subpixel Motion Estimation (subme / subq)

Subpixel motion esimation (subq) is the next-most-important option. HandBrake defaults to a setting of 4, but I recommend using a setting of 7, which is slow but provides an increase in PSNR of 0.292 dB beyond the default. Using a setting of 1 looks terrible and provides a PSNR hit of -0.325, though it has the benefit of being much faster than the default:



Trellis (trellis)

The trellis option provides the next largest benefit (0.158 dB) at the cost of a somewhat slower encode. Enabling this option also makes a striking difference in fine, hard-edged details, such as on-screen text, as demonstrated by the MPAA rating screen:



I've seen examples online of people using a setting of 2, which they claim improves the quality, but that just isn't supported by these benchmarks, so don't bother using a setting other than 1. Update: turns out using a value of 2 does do something (at the cost of slower encodes), but not unless your subme value is 6+.

The maximum subme value increased from 7 to 9 since I originally wrote this article. I used the new maximum setting for these encodings and they all turned out pretty similar with this highly complex setting, but you can still see some pretty noticeable improvements with each trellis bump. With trellis=0, we can see some blocking and a weird artifact on her tooth; the average encoding speed with this setting was 24.427734 fps. Trellis=1 is a little better but still has some discoloration, while the average encoding speed was inexplicably faster at 24.567919 fps. Trellis=2 is a little smoother than either of the other settings, but it had approximately 20% slower encoding speed, at 19.798990 fps, which may not be worthwhile for all applications.

PSNR was a real mystery to me in this comparison. The average PSNR actually fell as trellis use increased, despite some pretty clear visual improvements. Global PSNR also appears wonky, as it dips slightly with trellis=1 and then increases slightly more with trellis=2. These results really showcase the danger of relying on PSNR for assessment, as it is clearly an imperfect measurement.

Reference Frames (refs)

Next up, we have the number of reference frames. The default is 1, but increasing it to 3 provides an increase in PSNR of .079 dB and increasing it to a value of 5 provides a further increase in PSNR of .043 dB. Unfortunately, diminishing returns sets in quickly and ramping the value up to 12 provides only a further benefit of .034 dB at significant detriment to encoding speed:



Also, be wary of adding too many reference frames, as Quicktime may struggle to play back videos with large amounts. Furthermore, the number of reference frames interacts with the subpixel motion estimation setting to determine how much longer your encoding will take, so experiment to see what works best for you.

Mixed Reference Frames (mixed-refs)

Using >1 reference frames unlocks the ability to use mixed reference frames, which provides a modest improvement in PSNR of .035 dB:



B-Frames (bframes)

In my experience, the b-frames value is roughly as important as the reference frames value with respect to PSNR. Going from the default 0 b-frames to 1 b-frame provides a noticeable 0.12 dB benefit in PSNR. However, diminishing returns are apparent immediately whereby upping the value to 4 provides an additional benefit of only .069 dB, and ramping it up to 16 b-frames actually worsens the picture by -.033 dB:



According to the HandBrake wiki, you can use more b-frames in animated content (~10-15).

Weighted B-Frames and Pyramidal B-Frames

Once you enable ≥1 b-frame, you can also benefit from weighted b-frames and/or pyramidal b-frames. These options provide modest benefits to PSNR of .016 and .01 dB, respectively:




Be aware: enabling pyramidal b-frames is a "high-profile" feature that will totally bork Quicktime playback. Don't use it if you plan on watching your video on a Mac.

Motion Estimation Method (me)

This option determines the method used to estimate motion in your video. Options include hex (hexagon; the default), umh (uneven multi-hexagon; the highest quality), dia (diamond; not shown), and esa (exhaustive; not shown and not to be used). Umh is the highest quality but slightly slower than regular ol' hex; it provides a noticeable .159 dB improvement in PSNR:




Also included in the previous comparison is the motion estimation range (me-range) option, which provides a modest .048 dB improvement in PSNR at the cost of slower encodes.

Analysis (analyse)

The analysis method makes very little difference and I recommend sticking with the default. Switching the value to 'all' provides a miniscule .004 dB increase in PSNR (too small to see).

After enabling the 'all' analysis, you can also enable the use of 8x8 DCT analysis option, which, strangely, hurts PSNR by .07 dB:



No Fast P-Skip (no-fast-pskip)

The HandBrake wiki says this option helps "with blocking on solid colors like blue skies," but I didn't really see any benefit of it in that regard. On the other hand, it reduced the PSNR by a fairly substantial (considering the lack of benefit) -.048 dB, so it's up to you whether or not to use it:



Disabling P-skip might also have more of an impact on animated content, which generally features larger patches of solid colors than live content.

Deblocking Filter (filter)

This option tweaks x264's built-in deblocking filter to smooth out edges--and details--from the picture. It's kind of like using a PhotoShop filter on each frame, with positive values acting like a smoothing filter and negative values acting like a sharpening filter. The default, 0,0, is almost always the best, according to the HandBrake wiki, but I prefer a bit of smoothing and loss of detail (around 2,2). A lot of people (apparently) prefer using negative values, but I think that looks like crap and adds a lot of noise to edges:



Either way, you take a PSNR hit of approximately -.067 dB.

CABAC vs CAVLC

CABAC, the default option for HandBrake, is good for every use except when playback on iPod 5.5G and AppleTV are necessary. If you turn it off (cabac=0), HandBrake will use CAVLC instead, which is visibly detrimental and imparts a tremendous PSNR hit of -1.542(!!):



Direct Prediction (direct)

The HandBrake wiki makes this option sound really important, but changing the values had absolutely no effect on PSNR or appearance in my experience, so I suggest just leaving it alone:



Turbo First Pass (-T; again, not an advanced x264 option, but very important)

The turbo option significantly speeds-up the first pass of a 2-pass encoding by passing faster options for the first pass and slower, higher-quality options for the second pass. The HandBrake wiki mentions that the turbo option may slightly reduce quality, but my experience was exactly the opposite. Due to the nature of the option, I compared an encoding with a variety of slow, high-quality options with and without the turbo option included. Interestingly, this resulted in an increase in PSNR of .054 dB in the turbocharged encode:



Bidirectional Refinement (bime)

The next option, bidirectional refinement, depends on several other options that I couldn't readily identify, so I did a similar comparison to what I used with the turbo comparison (i.e., a variety of slow, high-quality options with and without bime). This yielded a PSNR benefit of .014 with bime turned on, which is probably unnoticeable, but worthwhile if you're looking for the best quality possible:



Threads (threads)

The threads option just allows you to specify the number of threads HandBrake will use while encoding. This makes no difference on single-core processors, but makes a huge difference on multicore processors. HandBrake defaults to automatically decide the "optimum" number, but I find I get slightly higher processor utilization if I assign a value of 4 on my dual-core AMD X2 4000+.

Deinterlacing (-d; not an x264 option, but it's pretty important so I figured I'd throw it in)

Interlaced video (as opposed to 'progressive') has a bunch of crazy-looking lines--known as scanlines or combing--in some frames and it makes videos look awful. If you want to get rid of them, HandBrake has a really great built-in deinterlacing filter that fixes things right up. I tried comparing "-d slowest" with plain ol' "-d" and they were exactly the same, so I've only included a comparison with and without the vanilla "-d" option:


Of note, deinterlacing does not affect PSNR, though it does significantly impair picture quality and introduces 'jaggies.'

Update (5/4/09): HandBrake has recently introduced a new option known as Decombing, which analyzes each frame and detects combing and only runs the deinterlacing filter on those specific frames. This is a huge improvement over regular ol' deinterlacing because it only affects the necessary frames and leaves the other intact. Furthermore, you can leave this option on all the time (i.e., include it in custom presets) and it will only kick in when necessary.

Adaptive B-frames: (b-adapt)

If I understand correctly, this setting analyzes your file and determines the optimum number of b-frames, up to the maximum you specify in the b-frames field. This means you can crank your b-frames setting up to, say, 16 and it will only use as many as it feels are necessary (almost never more than 10, so 16 is overkill, but you get the idea). Within this setting, you have 3 choices: 'none,' 'fast,' and 'optimum.' 'Fast' is much better than 'none' and 'optimum' is very slightly better than 'fast.' Importantly, 'fast' only imparts a slightly longer encoding speed than 'none,' while going from 'fast' to 'optimum' will drop your encoding speed by 50%(!!!) for a largely unnoticeable difference. Therefore, I recommend using 'fast' unless you're a real stickler for whom time is no object. I hope to get a comparison of these options up here in the near future, so stay tuned.

My personal settings

Each of these options may not provide much benefit on their own, but the improvements certainly add up. My settings result in a high-quality rip (PSNR of 41.232 vs 35.933 with the default options; approximately 13% improvement) that remains (mostly) compatible with Quicktime but with a 60-80% worsening in encoding speed (from ~60 frames per second with default options to ~15 frames per second with my settings). Here is a comparison of the vanilla x264 options and my personal settings so you can decide for yourself if it's worthwhile for you:



I also use a higher bitrate (800-900 kb/s) when doing my actual encodings (not in this comparison), which obviously improves quality a great deal. Here are my x264 options:

ref=5:mixed-refs:bframes=4:bime:weightb:b-rdo
:me=umh:subme=9:filter=0,-1:trellis=2:threads=4

I also use the non-x264 options of -2 and -T. Update (5/4/09): I used to be all about the 2-pass encoding, but I now prefer constant quality with CRF enabled. It takes the same time as 2-pass overall but gives consistent quality that is slightly better throughout the file, in my experience. I hope to post a comparison sometime in the near future.

***My source video is a preview for Jackie Brown that appears on a Pulp Fiction collector's edition something-or-other. All comparisons used frame 460 (or 1120 for deinterlacing comparisons). For testing, I used a bitrate of 300 kb/s, which was intentionally low so the benefits of different options would hopefully be more visible. Nevertheless, x264 is so good at preserving quality that I had to zoom in 400% to get a good look at the details. So as not to introduce artifacts from upscaling and/or jpg compression, I used png (lossless) screencaptures at the native resolution for all initial adjustments (zooming, compositing, etc) and only switched to jpg (lossy) at the very end for posting online. I placed all streams into a matroska mkv container and used AC3 passthrough for the audio.

Mac Version of Spore CC Uses Code From WINE

Instead of making an actual port of the Spore Creature Creator for Mac, it appears EA has simply bundled WINE code (see the 'transgaming' folder inside the app bundle) in with a Windows-compatible .exe file (it's something called Cider that they talked about at some developer conference a while back). This is pretty depressing for the whole "gaming on Macs" crowd, but it brings up some interesting thoughts about getting it to work on Linux.

This may not reflect much on the standard WINE install, since Transgaming has a spotty history of contributing back to the WINE community, but I'm hoping I can use this info to get it going on my Gutsy box. I'll make a new post if I can figure anything new out.

(Edit: I ran into dll hell for a while, but I got that part worked out. Now it starts up mostly error-free except I get an error that says "A required security module cannot be activated. This program cannot be executed (8008)." If anyone has a clue how to get past this, let me know.)

Update: no luck with the 8008 error, but I looked at the MD5 checksums for the windows .exe and the .exe file that's bundled into the Mac install and, although they both are 16.6 MB, their checksums are different:

3bcbb5ce269c4a38e73344fcb4c87949 for the Mac version

and

eb2834d31b2cb572a4f22947e000127e for the Windows version

[Let me know if you guys come up with different checksums]

This suggests that Cider does something (possibly at the compile time) that's making them different. I would assume it's something similar to the Darwine SDK that lets you compile Windows executables from source for use on Macs, but who knows. This in mind, you may have better or worse luck when trying to run the different versions with WINE since they're not exactly the same.)

For anyone interested, here's a list of the the .dll files that Transgaming included in the Mac version, which may give a clue to what is needed to run it in Linux:

cg.dll
cgD3D9.dll
d3d8.dll
d3d9.dll
d3drm.dll
ddraw.dll
dinput.dll
dinput8.dll
dmusic.dll
mfc42.dll
mfc71.dll
msvcirt.dll
msvcp70.dll
msvcp71.dll
msvcp80.dll
msvcr70.dll
msvcr71.dll
msvcr80.dll
msvcrt.dll
opengl32.dll
psapi.dll
squish_nosse.dll
squish.dll
stdole2.tlb
stdole32.tlb

Likewise, here's a list of the dll overrides they're using in their transgaming config file (located at Resources/Preferences/config in the Mac bundle. You can also find the system.reg file, which holds a lot of interesting info, in that folder):

[dlloverrides]
"commdlg" = "builtin, native"
"comdlg32" = "builtin, native"
"oleaut32" = "builtin, native"
"ver" = "builtin, native"
"version" = "builtin, native"
"shell" = "builtin, native"
"shell32" = "builtin, native"
"shfolder" = "builtin, native"
"shlwapi" = "builtin, native"
"lzexpand" = "builtin, native"
"lz32" = "builtin, native"
"comctl32" = "builtin, native"
"commctrl" = "builtin, native"
"advapi32" = "builtin, native"
"crtdll" = "builtin, native"
"mpr" = "builtin, native"
"winspool.drv" = "builtin, native"
"d3d8" = "builtin, native"
"d3d9" = "builtin, native"
"d3drm" = "builtin, native"
"ddraw" = "builtin, native"
"dinput" = "builtin, native"
"dinput8" = "builtin, native"
"dmusic" = "builtin, native"
"dsound" = "builtin, native"
"opengl32" = "builtin, native"
"msvcrt" = "native, builtin"
"rpcrt4" = "native, builtin"
"msvideo" = "builtin, native"
"msvfw32" = "builtin, native"
"quartz" = "builtin, native"
"mcicda.drv" = "builtin, native"
"mciseq.drv" = "builtin, native"
"mciwave.drv" = "builtin, native"
"mciavi.drv" = "native, builtin"
"mcianim.drv" = "native, builtin"
"msacm.drv" = "builtin, native"
"msacm" = "builtin, native"
"msacm32" = "builtin, native"
"midimap.drv" = "builtin, native"
"psapi" = "builtin, native"
"wininet" = "builtin, native"
"dbghelp" = "native, builtin"

Also located in the Mac bundle is a folder called MacOS that includes a pair of Unix executables called 'cider' and one called 'cidernoui.' I tried running these in my Linux terminal and it made my text go wonky in that session. Very strange.

Another file, Resources/transgaming/c_drive/Program Files/EA Games/Spore/Data/Config/ConfigManager.txt may be of interest.

Please comment if you have any info about that, or anything I've mentioned so far. Please correct me if I've gotten anything wrong.

- Hunter K.

HandBrake Dependencies List

Update (8/24/10): I wrote this post a long time ago and the dependencies have changed quite a bit since then, so here's an updated list. I'll leave the old post untouched for posterity. These are intended for use on a Debian-based system and can be pasted into an 'apt-get install' command:

build-essential autotools-dev libtool libgudev-1.0-dev intltool autoconf yasm libbz2-dev zlib1g-dev libwebkit-dev libnotify-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev wget subversion python python-gtk2-dev

You'll need to add libdvdcss2 to this list to work with copy-protected DVDs.

Original post: Here is a complete list of dependencies required to compile HandBrake from source. This is mostly for my own reference for whenever I reinstall my OS, but hopefully others will be able to benefit from it as well since it's so difficult to find a complete list elsewhere:

autoconf
automake
g++
g++-multilib
gcc
gcc-multilib
jam
libgcc
libtool
make
makedev
yasm (or nasm if you don't care about cpu extensions)
zlib1g-dev
build-essential (if you're on Ubuntu or a derivative)
dpkg-dev
libdvdcss2

Once you have these installed, just navigate to your source directory, type ./configure, then jam. The build process takes a long time and downloads a lot of things for codecs etc. When it's finished, you'll have a binary called HandBrakeCLI in your source folder, which can be run from its current location or moved to wherever is most convenient for you.

Fix for Netatalk nbp_rgstr: Connection timedout error

So, I had been playing around with VMWare Server in my Gutsy installation and when I restarted, my Netatalk daemon was failing with an error: nbp_rgstr: Connection timedout.

Luckily, I found this old forum post, which had the solution: Just go into your atalkd.conf file and uncomment the line that says eth0.

I'll walk you through it:

Open a terminal, then type:

sudo nano /etc/netatalk/atalkd.conf

Then delete the # sign in front of the word 'eth0'. Pretty simple!

Apparently, netatalk will just bind to whatever network interface is laying around, so it can sometimes get confused and bind to the crippled one used by VMWare Server. This fix just tells it to look for eth0 from now on instead of picking one willy-nilly.

CPU-optimized Backend for gHandBrake

Edit: The original author of gHandBrake's site appears to be down and I haven't heard from him since I submitted code to add most of the advanced x264 options to the interface, so I decided to offer my updated version right here (64-bit binary plus source code) until further notice. If you need to compile it for a 32-bit system, I have a list of dependencies for HandBrake here, and precompiled binaries of Yasm Debian/Ubuntu systems here. If you have any problems with any of it, just let me know in the comments and I'll try to help.

You can also download the unmodified gHandBrake (fewer features, but more stable) here. This package includes both the source tarball and a precompiled 64-bit deb binary for Ubuntu/Debian users.

Original post: I've been trying out gHandBrake, the excellent new GTK frontend for HandBrake. It's very early in development, so it doesn't have many of the advanced options--such as advanced x264 options--implemented yet but it seems quite solid and will eventually be a great replacement for HandBrakeGTK/RippedWire, which is basically just an incomplete port of the Windows GUI (of note, HandBrakeGTK does not actually pass advanced x264 options to the commandline).

Unfortunately, the precompiled binaries offered by the author do not appear to utilize HandBrake's processor-specific optimizations, so I decided to offer an optimized 64-bit build of the backend that would:
Download link

To use it, just download and extract, then replace the old ghandbrake-backend with the newer version. In a console, you would type (assuming it was extracted to your desktop):

sudo mv -f ~/Desktop/ghandbrake-backend /usr/bin

Since switching to the updated backend, my average encoding speed went from ~40 fps to ~115 fps, but YMMV.

Of course, if you compiled gHandBrake yourself from source, you would already have the optimizations included as long as you had the yasm assembler installed when you compiled. If you want to go this route, I have a precompiled yasm binary that recognizes SSE3 instructions in my previous post.

Please leave me a comment if you have any questions or concerns.