N64 3-Point Texture Filtering in mupen64plus-libretro

The Nintendo 64 console used a lot of weird hardware that contributed to its distinct look. For example, N64s used bilinear filtering when scaling some textures, similar to how modern GPUs handle texture resizing, but instead of using 4 sampling points like modern hardware, the N64 only used 3 (current texel, upper-left and bottom-right).

Update (12/29/18): I thought it would be helpful to post some text from Nintendo's patent filing regarding the N64's texture filtering:

Texture filter 532 in this example can perform three types of filter operations: point sampling, box filtering, and bilinear interpolation. Point sampling selects the nearest texel to the screen pixel. In the special case where the screen pixel is always the center of four texels, the box filter can be used. In the case of the typical 3-D, arbitrarily rotated polygon, bilinear filtering is generally the best choice available. For hardware cost reduction, display processor texture filter unit 532 does not implement a true bilinear filter. Instead, it linearly interpolates the three nearest texels to produce the result pixels. This has a natural triangulation bias which is not noticeable in normal texture images but may be noticed in regular pattern images. This artifact can be eliminated by prefiltering the texture image with a wider filter.

This caused textures to have a distinctive, hexagonal "rupee" shape:

Image taken by TrekkiesUnite118 on the Sega-16 forums; see the texture pattern on the wall.

When emulating an N64, if you use modern 4-sample bilinear filtering, those textures don't render properly, which can lead to ugly artifacts, like the jagged texture under these stairs in Kakariko Village:

Way back in 2010, there was a discussion on the devmaster forums about reproducing this 3-point sampling in software with some great screenshots and code samples. Then, a couple of years later, ArthurCarvalho posted an HLSL shader that performed the same function on the Emutalk forums.

Skip to 2014 and my friend aliaspider, author of the awesome GTU shader and the guy responsible for porting RetroArch to the PSP (among many other things), ported this HLSL code to GLSL and plopped it into the rendering code for the libretro fork of mupen64plus, where it is applied on a per-texture basis. This clears up many of the texture artifacts typical of N64 emulation and provides that familiar, pointy-textured look:

To my knowledge, no other N64 emulators have implemented this texture filtering option at the time of this writing. Update (9/15/2015): looks like gonetz has added it to his crowdfunded gliden64 plugin.