Thursday, January 19, 2017

Shaders for Sharpest Pixels

As readers of my blog can probably guess, I usually favor scanline and CRT effects on my retro games but there are a lot of folks who prefer sharp-edged pixels. For many of these folks, integer scaling on the Y-axis and 1:1 pixel aspect ratio (PAR) with nearest neighbor (NN) sampling is gospel, but that can lead to weird display aspect ratios (DAR) on many systems, including S/NES, where the PAR is naturally non-square*. This issue is further exacerbated if you want to run your games at 4:3, which is the nominal aspect ratio of CRT displays and is probably most like what people saw playing retro games in their natural habitat.

If you get away from 1:1 PAR/integer with NN sampling, you end up with a lot of problems caused by uncertainty regarding where in the texel (that is, the texture's pixel) the upscaled pixel is actually coming from. This manifests as "shimmering" during what should be smooth scrolling and ugly, uneven pixel sizes on what should be smooth slopes:
The solution for this is to anti-alias the pixel edges, weighting the resulting pixels against their neighbors, and there are a handful of shaders that do precisely this. For my comparison shots, I've zoomed in much further than I usually do and I'm using an NES shot to accentuate the effects. I also limited the interpolation to the X-axis to make it easier to see the effect.

PIXELLATE

Originally written by Fes, this is the OG sharp anti-aliasing shader and has been ported everywhere. It takes four texture samples a small distance from the current pixel and averages them together.
Since the averaging is happening in gamma-adjusted colorspace, it favors dark pixels just a bit, so I added a runtime option to the slang port to do the interpolation in linear colorspace instead:
This avoids the darkening but also leads to "floating" pixels sometimes when an upscaled pixel is flanked by light pixels, as occurs behind/below Mario's ear. So, pros and cons /shrug.

AANN

jimbo1qaz took another stab at the same concept and wareya added some bits to allow for interpolating in "pseudo-perceptual" colorspace. It ends up being slightly darker than pixellate via gamma curve, surprisingly enough:

SHARP-BILINEAR

Themaister took a different approach with this one. It prescales the image to a desired integer scale (I added a default option to automatically choose the largest integer that would fit on the screen) using NN scaling and then use bilinear scaling to go the rest of the fractional scale factor. In this shader, there's no averaging of multiple samples, so the gamma status doesn't matter. It ends up landing somewhere between AANN and pixellate via gamma curve:
This shader is very lightweight because it only samples the texture once and then leverages the GPU's own scaling hardware, which works essentially for free. Note: this effect is the exact same concept as the "prescale" option that appears in a variety of emulators.

There are a few other shaders worth mentioning that I didn't include here, like Inigo Quilez' "improved texture filtering" (we just call it 'quilez' in the repos), which is significantly sharper than plain bilinear scaling while still providing evenly sized pixels, and aliaspider's scaling swiss army knife, GTU, which can produce a similar anti-aliasing effect when pushed to a very high internal resolution.

These shaders are available in Cg, GLSL/slang and quark/GLSL formats.

*This statement is hilariously contentious and causes some people to flip their shit.

7 comments:

  1. Thanks for this great post! Lately I found myself preferring simple shaders like these instead of crt-shaders to have this pure and lightweight look that a simple linedoubler like say an OSSC would give you. I've also liked the option that mednafen has had for ages where you can choose only to interpolate between pixels horizontally.

    ReplyDelete
  2. Why do you scale to the largest integer that will fit and go bilinear the rest of the way, rather than going to the next largest integer and then scaling down with bilinear? The latter is how I used to do it on KEGA Fusion, and to my naive sensibilities seems like the better option. Is there a technical reason for your approach?

    Historically, my preference is Pixellate, but for some unknown reason it has never even once worked on my HTC One M8. I have no idea why that would be, but several GLSL shaders show weird results on my phone, like the NTSC shaders.

    As for CRT shaders, there's a lot of weird ambiguity on various emulators regarding aspect ratio and overscan. You mentioned somewhere on the forums some people create the smallest integer scaling that won't fit in the screen and just treat that as their overscan, allowing CRT shaders to work on an integer-scaled image while still filling the full screen. Right now, there's no easy way to set that up, so I'd really enjoy it as an option in the future.

    Oh, and I'm Queen Fiona on the forums, so you've probably heard some of this before. Love your work, and the rest of libretro's, as always!

    ReplyDelete
  3. @FM
    Hey :)

    You can do that manually with sharp-bilinear and it looks the same as targeting the one below AFAICT. If your fractional scale factor is something other than 0.5, you probably want to go with the *closest* integer rather than always going one above or below. Also, the further you go beyond the next integer, the closer it becomes to regular NN scaling (i.e., the pixels get warped again).

    Yeah, there are some issues with the machine-converted GLSL shaders. I'm going to start hand-converting them soon, so hopefully some of those issues will be resolved. Likely some issues will remain on mobile devices, though, due to precision issues and differences between desktop GL and GLES.

    It's pretty easy to setup the greater-than-fullscreen thing: just go to settings > video and set aspect ratio to 'custom', integer scale to ON and then set the X scale to 6x and the Y scale to 5x. Save that to a core override and you're all set.

    ReplyDelete
  4. If I use a non-integrer scale,should I use Quilez?

    ReplyDelete
  5. @Anon
    Sure, you can. All of the shaders in this post are for use with non-integer scales and that's their only function, in fact. At integer scales, you can just use Nearest Neighbor scaling.

    ReplyDelete
  6. For best results should pixellate be before or after a CRT shader?

    ReplyDelete
  7. It would need to go before, but you don't usually need to do such a thing because CRT shaders typically want the original-res output, and they have their own scaling kernels that will even out the scaling.

    ReplyDelete