My student worker, Alex, borrowed a digital oscilloscope and photoresistor from a coworker of mine and we sat down at my workstation to collect some data in an area that's often discussed (vociferously!) but rarely actually tested: latency. Most latency testing is unscientific voodoo ("I can *feel* it") that also suffers from confused terminology (see: the fighting game community's complaints about "lag" and how it makes them drop their combos). In this case, we're specifically examining input latency; that is, the difference in time between pressing a button on the controller and the action taking effect on the screen.
Here's a picture of our test bench, which consisted of a button from my trusty Happ-modded Mad Catz SE wired into the aforementioned oscilloscope:
Note: my full system info is Intel i7 Sandy Bridge with AMD R7 200 series with all of the GPU control panel crap turned off except for Eyefinity.
Anyway, here are the graphs that illustrate some of the more interesting comparisons:
That all said, these data indicate that RetroArch is approximately 1.5 frames slower than GroovyMAME, while the difference between mainline MAME and GroovyMAME is within the variance of USB polling rates. However, in light of the counfounders, I think the strongest conclusion we can draw reliably from the arcade comparison is that RetroArch isn't any *better* in Win64 (i.e., a null finding), so users should go with whichever platform has the features that best suit their needs rather than worrying about slim-to-nonexistent latency differences.
While the testing was not 100% reliable due to multiple confounders in several areas, we can see some trends emerge that can inform our discussions about latency in emulation. Windowed is definitely worse than fullscreen, and enabling Aero compositing is worse than without while also increasing variance and unpredictability. Shaders can actually cause excess latency, sometimes severely so. ZSNES, which has become a bit of a punching bag among SNES emulation scenesters, has outrageously low latency in fullscreen, so if you can stomach the terrible accuracy, there's actually some justification for using it now other than OMGSNOW!1! Alcaro's ZMZ also performed very well and can utilize more accurate emulation cores, so it can be a means to leverage some of ZSNES' latency benefits without being stuck with its poor accuracy.
In the future, I would like repeat these tests with a CRT monitor, which would have a predictable baseline of near 0 ms. I would also like to test latency in other environments, namely Linux+KMS. Finally, it would be very useful to have some comparative figures for original SNES hardware (both via CRT and upscaled via XRGB-Mini) and for RetroArch running via console.
Here is a link to download the raw data in Excel format, in case anyone would like to look at the numbers in more detail and/or perform other comparisons that I didn't think of.
EDIT: I think some people are drawing more conclusions from these data than is really appropriate; specifically, some folks are trying to draw direct comparison between the emulators/frontends tested. These data are simply not extensive enough for that. Furthermore, it's important to keep in mind that I didn't test the quality of sync, which could heavily affect the results. Namely, ZSNES and ZMZ both suffer from frequent audio crackling and frame stutters, which indicate issues with vsync, while RetroArch has none of either. I didn't test RA with vsync disabled (i.e., blocking on audio with video tearing), which could have an effect, and in general gameplay, users need to decide whether improvements in sync are worth minor (potential) increases in relative latency.