Designing a Large-scale Phosphor Filter/Shader

Update (5/9/2013): I ported my shader to the Cg shader language and made some improvements in the process (click here to download). It now looks decent at sane, non-4k scale factors and has some easily-modified variables to allow users to make their own adjustments more easily. See the bottom of the post for new screenshots.

Update (3/29/2013): Some other versions of the shader and more screenies at the bottom of the post.
Update (3/18/2013): I think the shader is pretty much done! You can download it here. Skip to the bottom of the post for images and some details.

With 4k TVs on the horizon and high-dpi computer monitors already here, I figured it was time to start thinking about using these screens to better recreate the look of CRT TVs.

cgwg's CRT shader is already fantastic, but it was designed with the assumption of relatively low resolution (i.e., the current HD/1080p paradigm at the upper end and 720p screens at the lower end, resulting in ~3-4x scale factors) and single-pass shader implementations, and it makes certain concessions with those limitations in mind.

At 10x scale (2240 vertical pixels for NTSC) and assuming multipass support, we can forego some of those concessions, though. For example, you can draw the shadow mask right on the screen at 1 pixel width and each phosphor lens at 3 pixels tall by 9 pixels wide, which gives you a pattern like this (blurriness comes from the bilinear upscaling; click for normal size):

In this figure, the phosphors have a horizontal orientation rather than the vertical orientation seen in all of the phosphor images I've found online (like this one). I tried both ways and the vertical orientation didn't look right to me, insofar as it had vertical scanlines rather than horizontal ones. I'll show results both ways later on and you can judge for yourself.

Update (3/18/2013): Turns out the vertical orientation is correct. I got a macro lens and took some pictures of a CRT up close (the letter 'e' followed by a heart icon from Zelda OoT):

I still think it looks better horizontally at small scales (i.e., 1080p and lower) but vertically at larger scales. Anyway, back to the original post.

With that LUT tiled up to cover the entire SNES image (you can download it here), we're ready to add in our image, scaled to 10x using nearest-neighbor, which looks like this (click thumbnails for full resolution):

Next, we need to invert the colors in the image, which looks like this:

We then need to subtract these pixel color values from our LUT matrix, which results in this:

We subtract because the phosphor LUT represents a full 100% brightness in all phosphors, which is blended by our eyes into flat white. In our image, the white pixels are inverted to black, which subtracts nothing from our LUT and leaves those parts white, while magenta blanks out the red and blue pixels, leaving just green, and so on. Likewise, brighter pixels are inverted to dim, and they subtract less brightness from our LUT phosphors, leaving them bright. At this point, we have our basic phosphor coloration and luminance values represented.

Next, we'll make a new pass using the result of the previous pass as input (or in Photoshop, create a new layer), to which we'll apply a gaussian blur that will approximate the luminance-based color bleed into neighboring phosphors (no picture here, since it looks the same, just blurry; not very exciting). According to cgwg's comments in his existing CRT shader, this blur portion may be better suited to a 2-part process of blurring, once horizontally and once vertically, but I just did one all-over blur. You can use a blur intensity of 3 pixels to keep some sharpness or increase the value to 5 pixels to get a more bloomy, halated look. I'll show examples of both in a bit.

Obviously, this is image is still waaaay too dark, so we'll also need to crank up the brightness of our blurred layer to around 150% and combine it with the underlying raw image using the 'screen' method, which lightens everything a bit more. The math for a screen combine is f(a , b) = 1- (1 - a)(1 - b), where 'a' is the base layer value and 'b' is the blend value.

That gives us this result:

At this point, it looks pretty good and we could probably call it quits, but the darkening from the shadow mask is still monkeying with our perceived colors a bit, so I played around with the levels to try and get something that looked closer to the original image. I found that moving the black point to around 23 and the white point to around 179 while keeping the gray point at 1.0 gives a good, bright result:

This adjustment also ensures that blacks are actually black and not brown and whites are white instead of gray. Here's a comparison shot:

While we're looking at this Super Metroid shot anyway, lets zoom in and take a better look at the luminance-based glow created by our blur:

You can see where the brighter pixels bleed into the adjacent phosphors, particularly on the glowing suit highlights.

Using a 5 px gaussian blur bleeds the colors even more but ends up making the entire picture more hazy (3 px blur followed by 5 px blur):

You can see from the thumbnails that the 5 px picture looks nice from a distance, while the 3 px looks a lot better up close. So, you could make a valid argument for either one depending on your preferences and intended viewing distance.

The above process could work alright at a 5x scale, using a bilinear reduction on the LUT and a 1.5 px blur, and not look too bad:

But dropping it to 4x starts to really cause some problems with the colors, giving the appearance of too much gamma: 

At these scales and lower, cgwg's CRT shader looks much, much better, due to his compensation for LCD subpixel aliasing (among other things, such as gamma interpolation, screen curvature, etc.).


The last thing I want to cover is the phosphor orientation. I oriented my LUT with red at the bottom and blue at the top because doing it the opposite way looks weird up close (notice how the red pixels on the shell create a sort of halo instead of having the intended cartoony black outline, as compared with the pictures above):

Likewise, using vertical phosphors--as shown in all of the phosphor images I could find online--looks like crap close up *and* far off (eschew the cutoff edge; click to embiggen):

At the moment, I've just been playing around with Photoshop, but I hope to do some work on a legit shader that performs this process in real time. Unfortunately, I suck ass at writing GLSL, so it may never get anywhere. If anyone else would like to take a stab at it, I'm happy to provide any information I have.

Update: For the Photoshop steps, Romain Dura has helpfully converted the math functions to GLSL, and the gaussian blur steps from cgwg's halated CRT shader can drop right in.

Update 2: I cobbled together some code that works at least partially and I'm pleased with the outcome so far. I don't have a high dpi screen, so I'm stuck working at 4x, but I think the code should work fine with a larger resolution.

Currently, I can get it to the point where it has the shadow mask and phosphor lenses, as well as gaussian blur (though the blur is happening underneath the phosphors, which isn't exactly what I wanted):

This is obviously much too dark, but when I tried to lighten it up, I couldn't get things looking quite right. It would either blast out the colors underneath or make the phosphors visible on a flat black screen, which is unacceptable. When I tried adding a bloom pass, the phosphor bleed looked good (if a bit exaggerated) and the brightness was pretty close but the colors got totally wrecked >.<

Oh well... I'll keep at it. If anyone wants to play around with the code I've got so far and/or play around with my phosphor LUT, you can get it here.

Update 3: As cgwg pointed out in the comments, mapping the phosphors 1:1 on the SNES pixels, like I did above, isn't really how TVs worked. They were designed to receive a 480i signal (i.e., 2 sets of 240 lines of resolution, alternating which lines get displayed for an effective 30 fps), as per the NTSC broadcast spec, which Nintendo and others tweaked to show just 240(ish) lines, non-alternating, at 60 fps. This format goes by many names, including non-interlaced mode, 240p (not a real spec, btw) and "double-strike."

I did some more mockups using the same procedure as above, but using a phosphor array that's ~480 pixels tall (I actually went with 448; i.e., double the SNES res) and started with an image that should match the "double-strike" idea at 10x, namely, a 20x image with half of the pixels blanked out, like this (note, it looks exactly like if you added a 100% black scanline filter; if this is somehow incorrect, I'd love to hear about it in the comments):

Since the vertical resolution doubled, I also doubled the pixel radius of the gaussian blur, up to 6 px and, instead of adjusting the levels, I increased the brightness by 300% and the contrast by 25%, which resulted in this (crap: blogspot won't display that large of a file and insists on shrinking it and converting to jpg; here's a link to download the full-res if anyone wants it). And here it is shrunken bilinearly back down to 10x: 

 It still looks pretty nice. And here's a detail shot:

Interestingly, at this scale, the vertically oriented phosphors look a lot better and produce an image that's much less scanliney, so perhaps it is the better option:

Here's a link to download that vertically oriented shadow mask / phosphor LUT, if anyone's interested in playing around with it.

UPDATE (3/18/2013): After digging around in an old thread on byuu's forum, I came across a post from Themaister where he explained how to go about using and accessing various framebuffers in shaders, which allowed me to finish this one up, for the most part (at least to the point where I don't have to keep poking away at it). It's a real beast of a shader and doesn't run fullspeed on the Intel HD4000 I've been testing with, but it looks pretty good in slow motion :P

You can download the shader and a couple of LUTs here. (same as the link at the top of the post)

I included 240p and 480p LUTs with both horizontal and vertical phosphor orientations. The 240phoriz LUT will give you essentially the image I was going for at the start of this post:

Notice, these shots look a bit dark, due to the way the LUT gets shrunken on my 1080p/4x scale screen. In the shader, there's a line you can uncomment when you're on a low-resolution screen like this to make the colors a bit brighter, which gives you something like this:

For anyone on a larger screen (or if you'd like to play around with the higher-resolution LUTs), I also included 480p variants, the vertical version of which looks like this:

Again, these shots would be brighter with the low-res mode enabled.

Last, just for fun, I made some screenshots of Super Metroid using the low-res option with the 240phoriz LUT, just like the Photoshop renders I made above, and here's how it looks:

Not bad! :D

One thing that still needs work is that the gaussian blur is only happening vertically, I think, but I'm not really sure why. I'm assuming I didn't put the horizontal pass into the framebuffer properly, so I'll keep fiddling with that. Also, certain emulation cores (such as snes9x-next) don't really play nicely with it, presumably due to horizontal resolution issues. Genesis Plus GX seems to work fine, at least:

 

Update (3/29/2013): I made a couple of small changes to the shader and a few different variations. I added in a new variable called 'brightness' (toward the end of the file) that you can raise/lower to increase/decrease the brightness of the image if it's too dark (within a range). Here's a pastebin of the updated default shader (shown with 240pvert LUT):

And here's one with scanlines added (sry, no screenshot; I'll try to add one soon):

And here's a super-mega version with Themaister's multipass NTSC shader+scanlines (another missing screenshot).

GPDP made some new LUTs, too, which is awesome. The first one mimics an aperture grill (click to download the LUT; shown here with scanlines added):

This one will look best at 9x scale instead of 10x.

The second LUT is actually pretty good at smaller scales, such as 4x, and takes the same idea as cgwg's CRT shader in that it tints alternating pixels green or magenta (click to download the LUT;

 I also modified cgwg's CRT shader to use the phosphor LUTs instead of the built-in pink/green pixel tint (note: screen transformation doesn't work right in this version) . It looks like this (click to download; shown with 240phoriz LUT):

Just as a warning: there's no reason to use this one instead of the regular CRT shader at less-than-huge scales. It looks significantly worse than the normal version. TBH, that's true for all of the shaders in this post...

And here's one with aliaspider's GTU shader replacing the gaussian passes (click to download; shown with 480pvert LUT):

Update (5/9/2013): Here's how the updated Cg port looks. I now consider this the best version and recommend users upgrade to it if they've been using the older GLSL version:

These shots were all taken at 4x, which would never have been possible with the GLSL version (without it looking like total garbage). It also handles pseudo-hires transparency decently well (see: Kirby shot), though you'll notice there's quite a bit of color mangling still...

You can download this shader here.