Some Shaders I've Been Working On

“Whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided. It’s the sound of failure: so much modern art is the sound of things going out of control, of a medium pushing to its limits and breaking apart. ... The excitement of grainy film, of bleached-out black and white, is the excitement of witnessing events too momentous for the medium assigned to record them.” -Brian Eno 

I've been working on some shaders lately. 

The first one I want to talk about was inspired by some posts on the RetroArch/libretro subreddit, where a guy was using overlays (I think? might have just been photoshops) to make some really dark images of Game Boy Color games that mimicked the experience of using their technically awful front-lit LCD screens.

I never owned a GBC, and only played on a GBA very briefly in its day, but the enthusiastic response to those shots was really something, and it highlighted a significant unmet need that I had never really considered before. So, I borrowed a friend's GBA 001 and set to work on it, and here's what I came up with (as always, click to embiggen):

Rather than reinvent several wheels, I'm leaning on fishku's "authentic-gbc" LCD shader, which includes the characteristic little dog-ear on each subpixel, as well as Matt Akins lovely "pixel transparency" shader, which makes the white pixels appear transparent and adds a little drop shadow to them.

My addition darkens the image (adjustable how dark you want the ambient light), desaturates it, and then adds a point light source, the focus of which you can adjust (diffuse, like an overhead light, or more focused like a lamp or even a "worm" light attachment) and move around (by default, it's in the upper-right corner, like an overhead light). Direct light washes out whatever is under it, but indirect light fades off quickly, and some of the colors get inverted in the semi-lit area.

I also added in a "shmutz" effect using a dirty glass texture, which only really shows up when it's directly lit, along with a slight green tint that those old LCDs had under direct illumination.

There's also a "Nighttime Car Mode," which makes a bright light sweep across the screen every few seconds to simulate a car passing under streetlights in the night, which is a situation people seem to have a perverse nostalgia for.

While I was working on my shader, Matt Akins was exploring another idea that had been teased by that same redditor, of showing a rainbow pattern at certain angles due to a quirk of the GBC LCD's internal structure. He added this functionality to his pixel transparency shader, and he also submitted a patch to RetroArch to enable gyro and accelerometer control hooks to the shader backend. I was planning to work on this, too, but I'm thankful that Mr. Akins got to it first because he did a much better job than I would have.

With that in place, I added an option to control the light source using gyro control, which makes for a very uncanny gaming experience, tilting a controller to shine a virtual light source on a virtual crappy screen, but then seeing hints of rainbowing as you tilt it. Fun stuff.

I also made a modification of the 'gbc' console-border preset, which looks pretty nice, but the light source only affects the screen and would have required a lot of invasive changes to work on the border, too, so I didn't think it was worth the trouble. Ultimately, it's not my favorite, but it still works well for people who really want a border.

The other shader I've been working on is a CRT shader. Yes, another one, because there aren't enough already. This one focuses on monochrome displays, which are usually just an afterthought in CRT shaders, and monochrome modes don't usually exhibit any special characteristics vs the color-full settings. This is unfortunate, I think, because monochrome CRTs are cool devices that ruled the land for decades, even after color TVs were commonplace.

First off, monochrome CRTs don't have subpixels or masks. The entire screen surface is coated with a layer of powdered phosphor material that glows when the electron gun scans over it. Having this direct view of the beam makes the dynamic width very visible, and the beam itself has a bit of ... life to it.

I tried to capture that, and, as a result, static screenshots aren't going to look very good or capture the "magic," so I recommend trying the presets live / in person, if you have the inclination. Nevertheless, here's what it looks like:

That's the 'black and white tv' preset, which is intentionally pretty blurry, horizontally. Monochrome computer monitors could be quite sharp, though, and there's an "apple-monitor-ii" preset that handles that better:

In addition to the beam perturbations, this preset also has the scanline brightness cranked up really high, which brings in some general glow visible around the numbers, and it also makes some nice trails anywhere there's movement (courtesy of the 'glow-trails' shaders I made several years ago): 

Both of these presets include my analog service menu shader, which lets you adjust lots of physical CRT parameters, like h- and v-size, pincushioning, etc., as well as a version of crt-royale's geometry control that I isolated just for this. It allows you to tilt the monitor up/down/left/right and control the radius of its curvature, among other tweaks. I picked a really exaggerated, low-radius curvature, since these displays were often very small (sub-15 inches), but I used the pincushion control to flatten the middle out some, just as you would on such a small display.

These presets also leverage blurbusters' and Timothy Lottes' crt-beam-simulator shader whenever shader subframes are enabled, and, in addition to the motion clarity improvements, I think the flicker it introduces provides another level of verisimilitude that I wasn't able to capture properly in still shots.

The monoCRT shader also includes several different monochrome phosphor colors:

Just be sure to use integer scaling with this shader, since the scanlines look like crap otherwise. As usual, it looks better when it has more pixels to work with, and I think 5x scale is a minimum, making integer overscale on the Y axis a great way to experience these on 1080p displays. 

So that's what I've got. The 'frontlit-reflective' shader is available in handhelds/shaders, and monoCRT is available in crt/shaders, if you want to use them in your own presets. Most of the presets I've shown here (authentic-gbc-frontlit, black-and-white-tv, and apple-monitor-ii) are available in the 'presets' directory. Aside from the beam sim effect and the gyro controls, which aren't widely supported outside of RetroArch, these shaders should work fine anywhere you can load slang shaders (e.g., ares, snes9x, OpenEmu, etc.).

Kasser DAFM Mega Drive Instruments

I've been meaning to write this post for over a year, so I'm a little fuzzy on the details, but I figured it best to just get something down instead of putting it off any longer.

About a year and a half ago, I picked up a DAFM kit from Kasser Synths. It's a really great kit and an awesome FM synth based on the legendary YM2612 chip, and it supports the ability to load up to 6 custom OPL instruments in DMP format (used by the DefleMask Tracker software).

I wanted to use the instruments from Mega Drive / Genesis games, which required getting the games' music in VGM format and then using the vgm2pre software to convert them to DMP format. There are no pre-compiled linux binaries of that software, and it doesn't like to compile on modern systems without a few tweaks to the Makefile (someone else apparently ran into the same problem and put in a PR to fix it, so be sure to check that out if you run into it, as well), but once that was taken care of it, the program was easily scriptable to hit the entire set all at once. Of course, if you don't want to fool with it, you can just download my full set of instruments here. 

Not all songs have instruments embedded, apparently (hence, some folders are empty; maybe those are using the PSG chip instead? Sonic CD doesn't have any because it's using CD audio, of course). Furthermore, many of the resulting instruments are very similar/redundant, possibly a result of widespread use of the GEMS software for development.

6-button Pads in RetroArch

For the next post in my series aiming to de-mystify using abnormal input devices in RetroArch, I'd like to take a look at gamepads with six face buttons. I touched on these briefly in my previous post about N64 mapping, but we get a lot of confused users wondering why they can't find C and Z buttons in the input menu or outright claiming RetroArch doesn't work with 6-button pads.

Ultimately, this confusion typically stems from an incomplete (or altogether nonexistent) understanding of how RetroArch handles input--specifically, through the retropad abstraction that cores and frontends use to communicate--but the most basic aspect of this misunderstanding may admittedly come from our use of Nintendo-style button names and 4-face-button images in our UI:

Now, the retropad concept is core to libretro and RetroArch, so we can't make any major changes there, like adding additional buttons or changing the names of existing buttons, without modifying every single core ever made, which would be a lot of work and just generally not worth the trouble for what is a niche input modality (no offense to 6-button peeps).

Thankfully, buttons is buttons, and changing icons for those buttons is very easy. Just download this pack of modified icons, unzip the archive and drop the 'assets' folder into your RetroArch folder, overwriting any conflicts. Again, this will not change the names of the buttons from the Nintendo-style names, but it will update all of the helper icons, like this:

Hopefully, this will help 6-button users have an easier time mapping their devices and allow them to navigate the user interface more intuitively.

These icons will get wiped out if you update your assets, but you can always just go through this process again to get them back. I've applied the icons, wiped them out by updating and then reapplied them a dozen times or so while writing this post.
 

Mapping controllers for N64 in RetroArch

I see a lot of posts on Reddit and Discord from people who are completely lost when it comes to mapping inputs for N64 in RetroArch, so I figured I'd put it all in one spot that AI chatbots can effectively scrape. There are a couple of variables involved in the topic, so we'll just work our way through them one by one: 

The first complication is the "retropad" virtual gamepad abstraction that libretro uses to communicate inputs between frontends (e.g., RetroArch) and cores (e.g., Mupen64plus-next). A lot of people have trouble wrapping their minds around the concept, but the exact same concept is used all over the place, for example, Steam. Steam's input layer runs everything through a virtual XBox pad. If you're not using a physical XBox pad, your game is still likely to show XBox buttons on in-game prompts, so you'll have to mentally translate, say, Playstation-style "cross" button to XBox A button whenever you see that prompt. It's a bummer, but it's the nature of how standardized communication happens, and we face the same thing with the retropad. But the thing to remember is:

buttons is buttons.

You can map and move stuff all over the place, and it doesn't matter. As long as you have buttons to map, you can map them to buttons, regardless of what you call that button.

Okay, with that core concept out there and in our minds, let's move on to the first real scenario:

Modern XBox/PS-style Controller <-> N64 Pad

If you're using a modern controller, it's probably covered by our autoconfiguration profiles and doesn't need to be mapped against the virtual retropad. If that's indeed the case, DO NOT go into settings > input > retropad binds and start mucking around with stuff to move around your N64 inputs. This WILL cause you a bunch of grief. 

By default, the N64-focused libretro cores assign their inputs to mimic Nintendo's own mapping for N64 games on modern controllers, as exemplified by the Wii Classic Controller mapping for the Wii Virtual Console games. That is, the C-buttons are mapped to the cardinal directions of the right-analog stick, while the R2 button (right trigger) is used to trigger the "C-button Mode," whereby the C-button functions move up to the corresponding four face buttons (ABXY) as long as the button is held down.

Most games only used the C-buttons for things that don't require precision, such as camera control in Super Mario 64, and many others used them for precise inputs, but only very briefly, such as hammering out ocarina tunes in The Legend of Zelda: Ocarina of Time.

Both of these cases are covered very well by the default mapping, and I would strongly recommend everyone with a modern controller give this a shot before going crazy in the remap menu. It takes a few minutes at most to get the hang of it, and it's very comfortable and effective once you do.

There is a small handful of games that don't work well like this, such as Killer Instinct Gold, which treats the C-buttons as first-class citizens to have the traditional 6-button fighting game layout. In this case, having anything on the right-analog is awkward/imprecise, and it's simply not feasible to require tapping R2 to trigger "C-button Mode" on/off throughout a complex combo. For these games, we have the "Independent C-button Controls" core option, which takes the right-analog stick out of the picture entirely and removes the R2/C-button Mode function. Instead, all of the face buttons are moved over to digital inputs on the retropad, so you can move them around as you see fit in quick menu > controls > port 1/2/3/4 controls.

Modern 6-button Controller <-> N64 Pad

Recently, there have been a number of gamepads to hit the scene with 6 face buttons, usually in the form of "fightpads" for fighting games and/or based on Saturn/Mega Drive 6-button pads. I've seen a lot of people claim incorrectly that these pads are "incompatible with the retropad because of the 6 face buttons vs the retropad's 4." Again, buttons is buttons. As long as you have all of the physical gamepad's buttons mapped to something on the retropad, you can move the core's functions around on them however you like, and the "Independent C-button Controls" option makes everything nice and straightforward (i.e., none of the confusing "B / C-Left" nomenclature, which, for the record, is describing the regular mapping alongside the R2/"C-button Mode" mapping).

Original N64 Controller <-> N64 Pad

This one should be the most intuitive one to map, but it's actually the least. There are two main ways to handle it, and you'll have to decide which way you want to go at the outset: stick with the default core mapping and manage our retropad <-> N64 controller mapping accordingly, or use "Independent C-button Controls" core option, and the retropad <-> N64 controller mapping is nebulous (you just have to keep straight whatever you choose).

Default Core Mapping 

If you go with the former (i.e., no "Independent C-button Controls" option), we can crib from one of our existing autoconfig profiles to save ourselves the trouble of looking up the retropad <-> core function mappings manually:

input_b_btn_label = "A"
input_y_btn_label = "B"
input_start_btn_label = "Start"
input_up_btn_label = "D-Pad Up"
input_down_btn_label = "D-Pad Down"
input_left_btn_label = "D-Pad Left"
input_right_btn_label = "D-Pad Right"
input_l_btn_label = "L"
input_r_btn_label = "R"
input_l2_btn_label = "Z"
input_l_x_plus_axis_label = "Joystick Right"
input_l_x_minus_axis_label = "Joystick Left"
input_l_y_plus_axis_label = "Joystick Down"
input_l_y_minus_axis_label = "Joystick Up"
input_r_x_plus_axis_label = "C Right"
input_r_x_minus_axis_label = "C Left"
input_r_y_minus_axis_label = "C Up"
input_r_y_plus_axis_label = "C Down"

So, head over to settings > input > retropad binds > port 1 controls, hit "Reset to Default Controls," then hit "Set All Controls." This will guide us through the mapping process, offering up a retropad button, and then listening for which physical button we want to map it to. Use the button label list above as a guide for what to press when. The only trick here is that I recommend also mapping retropad-A to N64-B (i.e., you're mapping both retropad-A *and* retropad-Y to the same N64-B button), which will be useful for navigating RetroArch's menus. It's not vital, but it's a nice convenience, especially if you won't have a keyboard handy when you're playing.

Once you're finished mapping, hit "Save Controller Profile," and then hit "Reset to Default Controls" again. 

Just for the record: if you don't 'Reset to Default Controls' before mapping, any manual mappings you've made previously will interfere with your profile, and if you don't do it afterward, your crazy N64 mappings will screw up any subsequently loaded autoconfig profiles, since manual mapping supersedes auto-mapping in RetroArch

Independent C-button Controls Enabled

This method is pretty straightforward, but it requires you to be on top of translating your N64 buttons to retropad inputs, so I recommend taking notes as you map things, since you're going to need them when you're moving things around in quick menu > controls > port 1 controls.

Just like above, head over to settings > input > retropad binds > port 1 controls, hit "Reset to Default Controls," then hit "Set All Controls." As it goes through the available retropad buttons, map them wherever you like, just make sure you keep track of what is going where. When you're all finished, hit "Save Controller Profile," and then hit "Reset to Default Controls" again.

Next, load up your core plus content, hop into the "Quick Menu" and scroll down to "Controls," then "Port 1 Controls." This is where you can move your core functions (the column on the right) around on the gamepad/retropad buttons (the column on the left) without messing up your global/retropad inputs that affect everything else (other cores, menu controls, etc.). The only trick here is that, unless you've added button labels to your autoconfig profile, you're going to see the retropad labels in the column on the left instead of your physical N64 pad's button labels, so you'll want/need to consult your notes to see what is mapped where--just like the Steam/button label example at the start of this piece.

Korg NTS-1 Input Volume Fix

I bought a Korg NTS-1 kit about a year ago and have really liked it as a small and powerful stereo multi-effects unit. The only problem I've run into is that the input jack's volume is set really low out-of-the-box, so you have to really crank its output, and if you then trigger any of its own internal synth sounds, they're super-loud vs the stuff you're running through it.

To fix it, you just have to boot into the "Global Parameters" menu by holding the 'Reverb' button while you power it on, then go to the second item--input trim (which shows as 'trn' on the 7-segment display; very helpful)--using the 'type' knob, and change its value from the default -6 dB to 0 dB (as high as it goes).

To the best of my knowledge, this is never explained directly anywhere in the manual, though to be fair, the manual doesn't really explain much of anything. The section on the Global Parameters menu lists the available values for this setting, FWIW. Seems like a weird value to have as a default, even setting aside how obscure the setting is, but whatever. I'm not calling the shots at Korg.

Anyway, I just got another NTS-1 and spent a half hour trying to find this setting again (which is indicative of how well-hidden this info seems to be, since I already knew what I was looking for this time), so I figured I'd better put it on the ol' blog in case I need it again in the future.

Audio Pop/Click on Roku Express 4K+ Plex AAC 5.1/7.1 Surround

This is just a quick post about a dumb problem that I had a lot of trouble pinning down: audio popping on my Roku. The tl;dr is: set "direct play" to "auto" instead of "forced." If you'd like to understand why, keep reading.

Background

I got a new TV recently--a Hisense U7H--to replace my crappy, ultra-budget TCL 4-series Roku TV. I was planning to use the built-in Android OS to run the Plex app to stream media from my local server but the app apparently sucks in a number of strange ways, especially when it comes to music and playlists, so I bought a cheap Roku Express 4K+.

The Problem

Some of my videos started having an annoying popping/clicking in the audio. I tried the same videos with the native Android app and that resolved the clicking, so I was pretty sure it was Roku-specific. From there, I did some googling and came up with some possibilities: apparently Roku OS 10 has some sort of issue with AAC and AC3 streams, and there are some issues with Roku's "audio leveling" feature. I tried reencoding the audio on a persistently popping video from AAC to AC3 and the popping went away, so I don't think it was the OS 10 issue, and I didn't have the audio leveling enabled in the first place, so it didn't seem to be that one either.

Additionally, I tried playing the offending video on Jellyfin to rule out any Plex-specific issue and popped/crackled there, too.

One of the posts in the audio leveling thread caught my eye, though:

With a little more searching, I learned that Roku devices apparently cannot playback AAC via hardware at all(?) and will, at best, convert them to something else, but they apparently don't have the grunt to decode multi-stream audio--like 5.1 or 7.1 surround--at full speed, which results in the popping.

The Solution

The solution appears to be changing the "direct play" setting in the player/client from "force," which I had selected to minimize the CPU load on my server, to "auto." Your server will have to reencode the audio more often, but it should get rid of the pops.