Friday, June 20, 2014

True Hq2x Shader Comparison With xBR

Background
I was shocked to learn recently that the shader I and others have long called Hq2x is/was actually a misnamed port of another shader entirely! guest.r originally put out a 2.0 scale shader with a suffix "HqFilter," which stands for a part of the color blending code. Over time, this shader was ported to a million different emulators (including the official Metal Slug PC releases) and, at some point, the name got confused with another popular emulation upscaling algorithm, Hq2x. As CPU filters have been supplanted over time with GPU shaders, no one seemed to notice that no true shader port of the classic Hqnx algorithm--in any scale--existed in any language. EDIT: looks like there was an attempt to port it back in 2005 but it never caught on because it was incomplete and had some bugs, but it's something.

Fast-forward to a few weeks ago, when a shader programmer named Armada brought this up in RetroArch's IRC channel. He shared some pics of the "Hq4x" shader in the common-shaders repo (which itself was based on the old XML shader in bsnes' gitorious repo, which in turn was based on guest.r's ePSXe shader) and an identical pic taken using a CPU-filter version of Hq4x. The differences were obvious and indisputable:
I started digging through old forum posts and it turns out that several people had noticed this over the years, but no one ever posted any comparison pics or were able to backup their suspicion in any way.

Armada and I did find that there was an old DOSBox renderer called OpenGL-Hq that was at least a step in the right direction, being a hardware-accelerated implementation of the Hqnx algorithm, and it was helpful insofar as it demonstrated how the algorithm can use an external lookup texture for the detection. However, it was not particularly applicable to modern shader language, so Armada set out to port directly from the CPU filter's C implementation.

After some intense work, which included creating a program to generate the LUTs that Hqnx bases its calculations on, Armada completed his shader port (also copied into the common-shaders repo) and it works beautifully! Incidentally, the requirement for LUTs means that a true Hqnx port wasn't even possible until fairly recently, as SSNES/RetroArch is/was the first emulator (that I know of, at least) to support LUTs in shaders.

Comparison Shots
If you've read over my previous comparison of Hyllian's xBR vs Hqnx, xBR won by a landslide in pretty much every comparison, which is no surprise because it wasn't really an apples-to-apples comparison. That in mind, here are updated pics that show a true comparison between the two algorithms (2xBR shader first, Hq2x shader right after and Hq2x CPU filter third):


The first thing you'll notice in those Super Mario World shots is that Hq2x does a great job of killing the jaggies. Much better, in fact, than ScaleHQ from the other comparison, and almost as good as xBR. There are a couple of rough edges (Yoshi's nose is a good example), but Hq2x is also *very* fast, so reasonable tradeoffs here. Hq2x does completely ignore the light texture blobs in the ground, leaving them as hard-edged rectangles, while xBR turns them into ovals. You can also see some slight lingering differences between the Hq2x CPU filter and the Hq2x shader, where the shader actually does a significantly better job of handling various detections.








On these digitized shots from Earthworm Jim 1 and 2, though, the comparison sort of falls apart. xBR is able to spot the jaggies and smooth them out while Hq2x doesn't spot any patterns at all, due to them being outside of its LUT's detection matrix. In fact, Hqnx's inability to work on antialiased images is one of the reasons Hyllian developed xBR in the first place.

It's also worth looking at how the algorithms differ in handling text. For this comparison, I included the two extremes of xBR's corner detection, with the 'a' variant as the most rounded and the 'c' variant as the most square:
 

In this comparison, Hq2x is essentially indistinguishable from xBR's 'c' variant, insofar as the text is concerned. The xBR 'a' variant is of course substantially more bubbly, which may be desirable for some games.

Conclusion
My previous comparison wasn't really a fair fight, and I apologize to Mr. Steppin for misrepresenting his algorithm. This is a much better comparison of the algorithms, and in ideal conditions, Hq2x is almost identical to xBR in smoothing while running much faster. However, in other cases--particularly digitized artwork--limitations in Hq2x's pattern detection can leave some images completely unsmoothed.

The speed of Hq2x makes it attractive for certain use-cases, such as mobile, where performance is still of the utmost importance and xBR either doesn't hit full speed at all or else drains your battery. xBR, on the other hand, can handle a greater variety of images and is more likely to produce a pleasing image with the digitized artwork that became more common in the PSX era.

25 comments:

Anonymous said...

hey, this is off topic but your ppa said that you should be contacted through the blog. I noticed that there's no trusty build for libretro-pcsx and I was wondering if you could do something about that

(TIA, I like ur blog posts about shaders 2)

Hunter K. said...

@Anon
Yeah, libretro-pcsx-ReARMed isn't really functional. It has a ton of bugs. :(

Is there a reason you want to use it instead of mednafen-psx?

Anonymous said...

because it supports images other than bin/cue, which are a bit of a pain to find. (none of my disks survived the 90s)

Hunter K. said...

Ah, yeah, the bin/cue thing is a hassle, but you can use most images with it as long as you supply a cue sheet (e.g., iso/cue, img/cue, etc.). They're pretty easy to make and I think there are some scripts/utilities that can help.

Here's my homemade cue sheet for Armored Core (I only had an img file):
FILE "Armored Core.img" BINARY
TRACK 01 MODE2/2352
INDEX 01 00:00:00

ClassicGamer said...

is there a way to add a scanline shader to the xbr shader? I ask because in games like dkc series on snes for instance the rigging and ropes in the first level of dkc 2 are ridiculously smooth and nice looking but the rest of the game suffers from xbr doing it's thing. a scanline shader on top of the xbr shader may help... thoughts?

Hunter K. said...

Yeah, that's totally doable. I'm a big fan of scanlines after xBR. One combo I particularly like is xBR with scanlines and some light blur on top.

Which emulator are you using? If RetroArch, just add another pass after the xBR pass(es) and put a scanline shader in the new spot and it should work just fine.

ClassicGamer said...

I'm using Higan. I've tried to do some research on quark folder shaders but I haven't been able to find a way to combine them.

Hunter K. said...

Ah, ok, it's actually pretty easy there, too, you just need to add the subsequent passes to the folder's manifest.bml. So, add scanline.fs to the shader folder, then open the manifest and add at the bottom:
program
fragment: scanline.fs

Here's the one I described with the scanlines and blur:
https://github.com/hizzlekizzle/quark-shaders/tree/master/2xBR-Halation-Scanlines.shader

ClassicGamer said...

oh wow! Thank you for the quick education on quark shader operation! I was actually favoring the xbr.mdapt.hiresBlend shader and managed to add scanlines to it using your instructions! I hope this will be useful to others out there!

I've compared the settings in the scanlines.fs in different filters and am wondering what should I change to darken the scanlines maybe...twice as much?

Hunter K. said...

Try playing with the -0.15 in the last line. IIRC, increasing that to -0.3 should make them twice as dark. I could be wrong, but that's a good place to start. :)

ClassicGamer said...

once again you amaze me with your knowledge and experience! perhaps I can pick your brain again on this problem...

regarding scanlines - everything looks normal on my pc monitor but on my tv the scanlines have 'rows'...seemingly horizontal rows of varying intensity.

Hunter K. said...

The issue I think you're referring to is a result of scanlines on non-integer scale factors. The only reliable way to deal with it is to always use integer scaling (I forget what it's called in higan; it's the option with black bars at the top and bottom). It sucks not getting to use the full resolution, but it's just a fact of life :/

ClassicGamer said...

I see... I managed to eliminate the problem by setting the tv screen size at 'screen fit' instead of 16:9 (which it was set at), wide fit or 4:3.

Anonymous said...

Hi, I'm on mobile now and currently can't find a better way to reach you. Do you believe you could (or somebody has) make a version of xbr for dosbox? I'd love to play daggerfall using that filter.

ClassicGamer said...

Excellent post by the way! I love reading about the technicalities and changes out there from when I first used Zsnes back in the 90's! Things have come a long way and I'm excited to see where they go! It's just a matter of time until Nintendo applies custom filters to their VC games!

Hunter K. said...

For DOSBox, it looks like these SVN builds include xBRZ, which is a CPU filter based on Hyllian's shaders. It also seems to support HLSL shaders, which should work with Hyllian's HLSL xBR ports.

Glad you like the post :)

ClassicGamer said...

I've been playing around with RetroArch PC v1.0.0.2 and using the Mednafen PSX emulator. All is well except I decided to spice up the video and use xbr-mlv4-multipass as a shader preset. then I added on a scanline pass and it looks awesome windowed but if I go full screen the scanlines are spaced apart due to the larger screen size.

Am I missing a setting or something to make the scanlines look the same as when in windowed mode?

Hunter K. said...

Hmm. It shouldn't be necessary. Can you post a screenshot so I can see exactly what you mean?

ClassicGamer said...

this is in windowed mode when I first start up retroarch.
[IMG]http://i58.tinypic.com/qreql2.png[/IMG]
http://i58.tinypic.com/qreql2.png

this is fullscreen mode achieved by pressing "f"
[IMG]http://i60.tinypic.com/6572hk.png[/IMG]
http://i60.tinypic.com/6572hk.png

Benjamin Hodgetts said...

Do you happen to know of anywhere to get the recent xBR or more preferably the xBRZ shaders in XML format? (for FS-UAE as an example).

I can't seem to find any anywhere.

Hunter K. said...

@Benjamin Hodgetts
AFAIK, Hyllian doesn't do XML ports anymore, and I think xBRZ is only available in CPU-filter format, rather than GPU-shader formats.

It's not terribly difficult to port a more modern version to the older XML, fixed-function format, however, and you could use one of the cg2glsl-converted versions to handle most of the heavy lifting.

guest.r said...

In good old times of pixel shader 2.0 adapters i wrote s shader named with a suffix "HqFilter" which stand for a part of some color blending code.

Looks preety decent if you use it only for double scaling, otherwise it needs a second shader pass to resample the image.

The name "Hq" was used, i guess, because of the "blending" part of the code, a feature similar to the official Hq algorithm.

Nowdays it's a bit archaic/obsolete, but i discovered an other shader of mine named "4xSoft" is popular on the scene. It's really really better feature wise since it can be used on top of the CPU scaling algorithms, is even faster compared with the disputed "Hq" shader and doesen't need a fixed resolution to look nice.

Anyway, kudoz to xBRZ, Mr. Stepin's Hq etc. ...

Hunter K. said...

@guest.r
Hey, nice to hear from you :) I've been a fan of your work for years.

Your shaders have been ported so many times into so many different formats (often without your credits included, unfortunately) that I wasn't sure where the naming conventions came from. Thanks for the explanation. I updated the post to reflect.

Equality Equation said...

I apologize for leaving this here, but I couldn't find any other way to reach you (I tried finding you IRC on freenode or whatever, but I lacked the ability to figure it out).

I was wishing you'd be able to convert one of the newer, stronger xBR filters over to the Kega Fusion RPI format. I've been looking all over for an RPI of a newer version that's 5xbr, but I haven't found one made.

If you could create this for me, I'd extremely appreciate it. Although it may seem small, I can clearly tell a difference between x4 and x5, plus the difference in versions stands out as well.

Hyllian said...

Hi Hunterk. Nice comparison!

I'll add some info about the development of xBR and the differencies of it and the xBRZ variant (probably many people don't know why there are two names for the same algorithm) and answer the question about kega plugin made by Equality Equation:

1) I began the development of xBR in 2011, as I was a fan of the HqNx and wasn't satisfied by its performance over anti-aliased games.

2) It was developed firstly for Kega Fusion in plugin format (cpu filter) and after some months in shader format for Retroarch (when it was called ssnes at the time).

3) There are two main points in the algorithm that can be fine tuned: the formula used to measure the distance between colors and, second, the way the algorithm treats corners. I decided to offer three variants of the second one: a, b and c. But, it isn't limited to them.

4) In 2012, the author of xBRZ, Zenju, asked me to research other ways to treat corners and try new ways to calculate distances between colors. He rewrote the code in C++ and introduced his new findings. So, his xBRZ is the same algorithm of xBR, only with different corner detection and color distance measures. It's essentially a variant of the same xBR algorithm. It was forked from my 3.5 version, if I remember well.

4) Since 2012 I continued evoluting the algorithm in single pass shader format until version 3.8 (there's a 4.0, but it was a failure and its ideas where transfered and better developed in the multipass versions). The algorithm continued the same, the evolution was only to get more speed and some other cosmetic changes.

5) About the kega question, the Kega plugin specs is limited to 4x scale and it's hardcoded in the Kega closed sources. I can't make a 5xBR to it until the author evolute its plugin specs.

Analytics Tracking Footer