Tuesday, December 28, 2010

CRT Pixel Shader Filter for SNES Emulation

Update 05/18/2011: More screenshots for more new filters in my new post. :-)

Update 05/02/11
: After many changes, it looks like the CRT shader development has settled down, so there's less need for me to maintain the older versions of the shaders. From here on out, I recommend visiting Screwtape's git repo for all of your XML shader needs (my mediafire account will still be there, but Screwtape has all of mine and more).

CRT.OpenGL.shader and CRT-flat.OpenGL.shader are similar to the shader covered in this post, though they run slightly faster and have no visible artifacts. CRT-simple.OpenGL.shader is a simplified rewrite that should be usable on much older, slower machines.

Here is the original post for informational purposes:
This post covers the use of filters to upscale pixel art--specifically as it applies to SNES emulation--with special attention to CRT reproduction. If you just want the pictures and to download the filters, skip to the bottom of the post.

Background:

As everyone who dabbles in old-school emulation knows, artwork that was intended for a 480i CRT television that has been upscaled to an HD resolution looks like absolute garbage on an LCD monitor. The chunky sprites with their often thick, cartoony outlines just weren't designed to be reproduced with sharp edges resulting from nearest-neighbor upscaling.

To get around this ugly upscaling effect, many emulators now include upscaling interpolation filters, which apply complex mathematical algorithms to the original picture to fill in the gaps between things that are impossible to represent in chunky low-res, such as curves and smooth diagonal lines. You're probably familiar with some of the more common and popular interpolating filters, such as SuperEagle, SuperSaI and HQ2x. Unfortunately, none of these filters gets everything quite right, especially numbers and letters, which can look bubbly or overly smoothed (you can learn more about pixel art scaling algorithms here).

Purists have long been turned off by the inaccuracies of interpolating filters and have instead used scanline masks to try and capture the effect of an interlaced display, relying on the human brain's natural ability to recognize patterns and fill in the gaps between lines (you can learn everything you ever wanted to know about scanlines here). However, this too falls short from a true representation of a CRT display, as it ignores the existence of phosphors--the tiny red-, green- and blue-colored lenses that the electron gun in the back of a CRT tube shoots with a beam of electrons to recreate a colored pixel--and the color bleed that naturally occurs in these displays.

Recently, a number of determined individuals have set out to try and capture all of the different effects of a CRT display, warts and all, to truly reproduce classic pixel art the way it was meant to be viewed.

The Comparisons:
(Each of these images is presented as it would be displayed onscreen, at a resolution of approximately 800x600, then again at 400% scale without any interpolation used when scaling; as always, click the thumbnail to embiggen)

First, we should look at the baseline. This was scaled up to size using nearest-neighbor and is otherwise untouched:
Next, we'll add blargg's NTSC filter, which emulates the noise and color bleed of an NTSC video signal (this filter has several presets; I will only be showing the RGB preset, which reproduces the look of an SNES hooked up via RGB connection [not available in the U.S.], and the RF preset, which reproduces the look of the SNES RF modulator attachment, respectively):
(<- Look at that noisy RF signal!)
As a note, blargg's NTSC filter is so accurate that byuu, the author of bsnes, recommends its use along with bsnes' accuracy profile to achieve proper blending on games that use halftones to simulate transparency (Jurassic Park and Kirby's Dreamland, for example).

Next up, we'll look at cgwg's CRT shader, which includes a phosphor mask and barrel distortion to simulate the screen curvature of a CRT television (just look at those RGB phophors!):
Similarly, there is a version of cgwg's CRT shader, which doesn't include the barrel distortion and represents an idealized flat CRT (actual flat CRTs tended to have slight blurring at the edges where the tube curvature would normally be). Incidentally, this version also has no visible garbage pixels (the occasional black specs that are visible in the curved version):

Pixel Shaders vs. Software Filters

cgwg's CRT shader is a special kind of filter known as a pixel shader. Unlike regular filters, which rely on the CPU to do all of the complex upscaling calculations, pixel shaders draw on the awesome computing power of the video card to do the calculations, thereby leaving the CPU to focus on emulating the SNES. Additionally, since the pixel shader is calculated separately from the filter in bsnes, you can stack blargg's NTSC filter with cgwg's CRT shader:
Finally, for non-purists, we'll look at the combination of cgwg's CRT shader with the popular SuperSaI filter, which creates a pleasing--though not quite as accurate--output:
As amazing as cgwg's CRT shader is already, there is still some room for improvement. For example, the current implementation misses the intensity-based bloom effect on individual phosphors that can be seen in a true CRT. DOLLS (J) [!], one of the contributors to the CRT reproduction effort, intends to write a more complete CRT emulation shader in the future that will incorporate these and other idiosyncracies.

Click here to download cgwg's CRT shader (Also includes the 'flat' version for those who don't like the tube-style curvature; UPDATE: fixed dead link), which is compatible with bsnes and the newest release of snes9x. For more information on CRT emulation, you can check out this highly informative thread on the bsnes forum. For other bsnes-compatible shaders that are not included with the official download, check out my mediafire account.

UPDATE (3/4/2011): Themaister did a rewrite of the flat version of cgwg's CRT shader, moving many of the calculations from fragment to vertex, which provides a substantial ~20% increase in speed (making it usable on many older and less powerful video cards). This rewrite also appears to conform more rigidly to the GLSL shader spec, making it compatible with more cards from different vendors. I have labeled it v4 of cgwg's CRT Flat, and it is available in the aforemented mediafire account.

Friday, December 24, 2010

bsnes Phoenix GUI Filter Changes

Byuu has added support for binary filters in his Phoenix GUI as of v073, so we can finally use cgwg's CRT Curved shader in conjunction with blargg's NTSC filter! These binary filters are loaded just like shaders, i.e., via a dialog box in the video options.

You can download the available filters, compiled for 64-bit Linux, here or compiled for 32-bit here. These filters do not appear to work on Windows systems.

Saturday, December 18, 2010

Bsnes Special Chips Dumped

Update (4/08/12): All of the special chips have been dumped and emulated. You can download all of the images here. The archive contains images for use with bsnes versions v086 and earlier, as well as images for bsnes v087 and later.


Update (6/28/11): Another chip dumped and emulated! This time, the Cx4 chip (cx4.bin), which is used in Mega Man X2 and X3. The contents of this chip were really just math tables--naturally occurring relationships between numbers--and hence not protected by copyright, so byuu felt comfortable hosting the image on his own site. You can download it here (I'll also mirror it in my mediafire in case byuu's copy disappears for whatever reason).

Update (1/13/11): Dr. Decapitator has dumped and byuu has added low-level emulation for the NEC uPD96050 coprocessor, which is used by the ST-0010 and ST-0011 chips. This allows proper emulation of F1 Race of Champions II and playability for Hayazashi Nidan Morita Shougi. Click here to download the pair. Click here to download all DSP and ST-00* dumps.

Update (12/23/10): All DSP chips have been decapped and dumped. Click here to download the complete, unpadded (as per byuu's preference) set. Furthermore, thanks to the donations of several generous enthusiasts (and a particularly generous contribution from Mr. Krawczyk), all special chips have been paid for. Now, we wait for Dr. Decapitator and byuu to do their thing. ;)

Update (12/21/10): W00t! The DSP chips are all paid for and the decapping and emulation process has already begun. Click here to download a copy of the dsp3.bin file, which enables perfect emulation of the previously unplayable SD Gundam GX (J)!

While many individuals may think that emulating the SNES is long-since perfected, the truth of the matter is that a number of special auxiliary chips were never properly emulated and instead were approximated using what is known as High Level Emulation, or HLE.

To implement HLE, someone monitors what goes into a chip and then monitors what comes out in an attempt to infer the way the chip works. Using this strategy over the course of several years, 4 emulation heavy-hitters were able to deduce much of how these special chips work. However, they were never able to figure out any timing information, so to paraphrase bsnes author byuu, the chips were treated as magical black boxes that instantly convert input to output.

This is just beginning to change, though. Thanks to donations from emulation enthusiasts, renowned chip decapper Dr. Decapitator was able to shave down to the guts of the DSP-1B chip and see exactly what's going on inside. This breakthrough will eventually lead to the squashing of several bugs and will enable near-perfect emulation of both Super Mario Kart and Pilotwings.

However, to run any games that require the DSP-1B chip, you will now need to put a copy of the dumped image into the same directory as the game ROM image. Click here to download a copy of the dsp1b.bin file.

As awesome as this is, there are still a number of chips that need to be decapped (you can read all about it here) and byuu is running a collection to get things started in this thread. If you would like to contribute, visit byuu's official donations page. It's your chance to be a part of SNES emulation history.

Wednesday, December 1, 2010

First Impressions of Ubuntu 11.04 Natty Narwhal

I decided to try out the latest build of Ubuntu's next future release, version 11.04 Natty Narwhal, in a VirtualBox VM. I downloaded a pre-alpha daily build and so far have been impressed with the stability and ease of installation.

As with Maverick, the installer is clean, polished and easy, and continues the trend of asking most of the configuration questions once the actual installation process has started to save time. You also have the opportunity to install third party programs, such as Fluendo's MP3 codec, at the time of installation, which is nice.
At this point, I don't think they've switched over to the proposed Unity interface by default, so I can't say whether I'll have a better experience with it now than I did on my netbook a few months back, but I'll update this post with more information as soon as I have it.

As for its performance in a VM environment, everything is stable and the VBox Guest Additions installed just fine, imparting mouse pointer integration and support for arbitrary resolutions. The only thing lacking is 3D acceleration within the VM, but I assume this will be supported at a later date.

Overall, the experience is not significantly different from that of 10.10 Maverick so far, but I expect things to change rapidly in the near future. I'll keep this post updated with any changes I find.

Friday, October 22, 2010

List of SF4 Materials and Attributes

This is just the start of the list, so hopefully I'll be able to add a lot more as time goes on. Everything will be formatted as:
Name/Source:
Material: Plain English
[Hex code]
Attribute: Plain English
[Hex code].

Akuma's eyes:
Material: Emission_W
[45 6D 69 73 73 69 6F 6E 5F 57 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 09 00 00 00]
Attribute: Glare
[47 6C 61 72 65 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 01 00 00 00]

Highly Reflective Metal Effect:
Material: SpcEnv_W or SpcEnvAlpha_W (for use with an alpha channel)
Attribute: ReflectCoeff
[52 65 66 6C 65 63 74 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 CD CC CC 3E]
ReflectFresnelBias
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 42 69 61 73 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 33 33 33 3F]
ReflectFresnelCoeff
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3F]

Medium Reflective Metal Effect:
Material: SpcEnv_W or SpcEnvAlpha_W (for use with an alpha channel)
Attribute: ReflectCoeff
[52 65 66 6C 65 63 74 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80 3F]
ReflectFresnelBias
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 42 69 61 73 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 CD CC CC 3D]
ReflectFresnelCoeff
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80 40]

Minimal Reflective Metal Effect:
Material: SpcEnv_W or SpcEnvAlpha_W (for use with an alpha channel)
Attribute: ReflectCoeff
[52 65 66 6C 65 63 74 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 CD CC 4C 3D]
ReflectFresnelBias
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 42 69 61 73 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0A D7 23 3C]
ReflectFresnelCoeff
[52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 43 6F 65 66 66 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 CD CC CC 3D]

Works on any non-metal material:

Full Transparency (compatible with DXT5 or DXT1 w/ 1-bit alpha DDS)
Attribute: AlphaTest
[41 6C 70 68 61 74 65 73 74 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 96 00 00 00]

Semi Transparency (only compatible with DXT5 DDS)
Attribute: AlphaBlend
[41 6C 70 68 61 62 6C 65 6E 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 01 00 00 00]

Attribute: AnimationChannel
[41 6E 69 6D 61 74 69 6F 6E 43 68 61 6E 6E 65 6C 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 03 00 01 00]

Attribute: ZWriteMask
[5A 57 72 69 74 65 4D 61 73 6B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 01 00 00 00]

Attribute: GlareCol(RGB)
[47 6C 61 72 65 43 6F 6C 52 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EC 51 78 3F 47 6C 61 72 65 43 6F 6C 47 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EC 51 78 3F 47 6C 61 72 65 43 6F 6C 42 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 EC 51 78 3F 47 6C 61 72 65 43 6F 6C 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80 3F]

Attribute: LowRes
[4C 6F 77 52 65 7A 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 01 00 01 00 00 00]

New Arcade Edition-only Materials (vfx-related)

HAKAN's Oiled body: SpcBumpBrushOilSSS_Shadow_UV1_W
HAKAN's Oiled clothes: SpcBumpBrushOilSSS_UV1_W
HAKAN's Oil: T1_C_Scroll_Emission_W

MAKOTO's body turning red: SpcBumpBrushSSS_Col_S_UV1_W
MAKOTO's face turning red: SpcBumpBrushSSS2_Col_UV1_W

EVIL RYU's body emission: SpcBumpBrushEmiMask0_UV1_W

ONI's body emission: SpcBumpBrushEmi_UV1_W
ONI's Right Arm emission: SpcBumpBrushEmi0_UV1_W
ONI's Left Arm emission: SpcBumpBrushEmi1_UV1_W
ONI's Right Leg emission: SpcBumpBrushEmi2_UV1_W
ONI's Left Leg emission: SpcBumpBrushEmi3_UV1_W
ONI's hair emission: SpcBumpBrushEmiAlphaConst_UV1_W
ONI's UC2: SpcBumpBrushEmiMaskConst_UV1_W

Thursday, October 21, 2010

Beginner's Guide to SF4 Skinning

There are several tutorials floating around for skinning in Street Fighter 4, namely those by Providenceangle and hornyyoshi, but they were both written fairly early on and have not been updated to reflect new, easier methods. So, here is a definitive guide to creating new skins in SF4:

1. Collecting the Tools

First and foremost, you'll need a painting program. Photoshop is obviously quite common but also very expensive. GIMP is a free alternative that works quite well (it's what I usually use) and you can download it free from here. This tutorial will mainly focus on Photoshop, but I'll include some info for GIMP when possible.

Next, you'll need a plugin for your paint program to allow you to open/modify DDS textures, which are used by SF4. For Photoshop, you can use Nvidia's free Photoshop plugin, available here. If you would rather use free software, GIMP has a DDS plugin available here. I'm not going to cover installing the plugins, but you should be able to find documentation at the respective sites.

You'll also want to download Nvidia's free Windows Texture Viewer tool, which will give you some valuable information about the DDS textures you will extract. You can download it here.

Finally, you'll need to download the latest version of piecemontee's Asset Explorer, which greatly simplifies the whole modding process, from decompressing files to identifying, viewing, extracting and injecting textures. It does it all and you can download it from here.

2. Some basic information

Vanilla SF4 uses two main files to control the appearance of a character:

A. The cos file (stands for costume), which is designated by the character's three-letter name abbreviation (ZGF for Zangief, RYU for Ryu, etc.), a number delineating which costume (01 for original, 02 for Capcom's alternate), the file extension (cos) and the compression container (emz). So, Zangief's original costume will be named ZGF_01.cos.emz.

B. The col file (stands for color), which is similarly designated by the character's three-letter name abbreviation, a number designating which cos file the color goes with (i.e., original costume or alternate), a number delineating which color slot the file occupies (one through ten), the file extension (col) and the compression container (again, emz). So, Zangief's first color for his original costume will be named ZGF_01_01.col.emz.

The cos files primarily contain the 3D model and other embedded files used by the engine to display the body that you see in-game. I won't cover editing any of these files here, but I wanted to mention it anyway.

For simple skinning, you will be entirely focused on the col files, which contain embedded texture files stored in DDS format (hence the need for a DDS plugin).

In AE, the *.emz containers are no longer used, so all of the component files are just sitting inside the character's directory. 3D models are stored in the ***.obj.emo file, while textures are stored in the ***.col.emb file.

3. Using piecemontee's Asset Explorer

The Asset Explorer is the cornerstone of the modding process and it takes the place of offzip, Dragon Unpacker and Infuser, all of which are difficult to use and can be scary for would-be modders.

So, just open up the Asset Explorer and drag a cos file and a corresponding col file which you wish to edit into its left-hand pane. In AE, you would drag over an ***.obj.emo model and a ***.col.emb texture file.

Now, expand the entries for the cos file until you see and entry that says #EMO (HND_01.obj.emo) (HND will be replaced with your character's three-letter abbreviation) and highlight it. As you can see, the program will then load the costume model, which will appear spooky and gray (this is the ambiant occlusion map; nothing to concern yourself with, but I thought I'd mention it for completeness):
To see the col textures applied to the model, use the pulldown menu in the middle of the window to select an option that displays 'texture' (the one at the bottom, 'Shadowed + Ink Shader + Texture + Normal Map' looks the closest to what you would see in-game):
Throughout the skinning process, you can use this to preview your work without loading up the entire SF4 program just to check a small detail.

4. Extracting Textures

Back in the left-hand pane of the Asset Explorer, expand the col file structure (and the entry labeled #EMB (HND_01_01.col.emb)) until you see one or more entries named 'DDS.' If you click on it, the preview window will change to display the texture file:
Once the desired DDS is selected, if you right-click on the DDS entry, you'll see a contextual menu with an option to 'Extract...' Choose it and you will be offered a chance to save the extracted texture as a standalone DDS file. This is what we'll be editing in our paint program and then re-injecting later, so I like to name it something that will tell me its position in the DDS order later, such as "ryu-bag-1.dds" or whatever. At this point, I also like to make a backup of the original DDS texture for future reference.

5. Painting Your Skin

When you open your extracted DDS texture(s) in your paint program, be it Photoshop or GIMP, it will bring up a dialog box asking you some questions. It's fine to skip loading MIP maps and to 'Load Using Default Sizes.'
I won't go over how to use the paint programs here (that's up to you to figure out), but some things to keep in mind:

A. Straight-up 'painted' textures usually look like crap. You're often better off using the hue/saturation tools to change the existing textures (there are obviously plenty of exceptions to this, so try things out and see what you like).

B. Some, though not all, textures include transparency, via what is known as an 'Alpha Channel.' I won't spend much time on it in this tutorial, but it is the cornerstone of another modding trick, known as 'col-based transparency.' I just wanted to mention it here so you're aware of it.

C. Since you'll be applying these textures to curved, irregular models, what looks like a straight line in Photoshop does not always look straight in-game. Frequently check your work in the Asset Explorer to ensure everything looks right and save yourself time and effort later.

6. Saving

Saving is a little more complex than hitting ctrl+S. After you name your file and choose its destination, a new dialog will pop up with a bunch of esoteric options:
We need our new texture to have exactly the same filesize as the original or the game will choke on it and crash. So, we'll want to 'Generate MIP maps' and make sure we select the correct DXT compression from the pulldown menu. Remember that backup I suggested at the end of Step 4? To find out which DXT compression to use, take your backup and open it with Nvidia's Windows Texture Viewer tool:
Along the bottom of the window, we can see the format, in this case, DXT5, so that's what we'll save our new texture as.

7. Injecting Your Texture Into the Col File

Now, back to the Asset Explorer. Navigate to the DDS textures and select the one you wish to replace. Right-click to bring up the contextual menu and, this time, choose 'Inject...' and browse to your new texture. If it complains about the filesize being incorrect, you have messed up something, probably in either resolution or one of the saving options, so go back and try to find your error. If not, you can click back on your model and see how it looks:
That's it. You've made your first skin. Congrats!

Once you feel comfortable with this process, you can move on to hex-editing the col file to create transparency effects or material swapping, making custom Normal Maps and directly editing the 3D mesh model.

Wednesday, October 20, 2010

How to Extract SSF4 Data From an Xbox 360 ISO

For people who want to get into using files from Super Street Fighter 4 on Xbox 360 with their PC version of vanilla Street Fighter 4, you have to go through a number of steps that have never been appropriately documented, so here goes:

1. Collecting the right tools for the job

First off, you need a copy of SSF4 in ISO format. I don't want to know how you get it, but I recommend purchasing it and then ripping it to your PC using the 'hot swap' method.

Next, we need a copy of isoextract, which extacts the data from the 360 disc image. You can download it here or here.

We also need QuickBMS, a script-based file extractor written by Luigi Auriemma, as well as a script to use with it. You can get QuickBMS here and the extraction script here or here.

2. Extracting the ISO

Open up isoextract and you should see this screen:
Using the buttons on the upper-right, click on 'choose ISO-folder' and navigate to the directory where your SSF4 ISO is stored. Next, click on 'choose destination' and select a directory where you want your extracted files to reside.

Then, just click the big button marked 'go' down in the bottom right.

Now, once it's all finished, if you navigate to your extraction directory, you'll find it has created a directory structure from the disc. Go to archive > battle and you should see a file named 'battle.eaf.' This guy has all of the goodies, so we gotta get them out using QuickBMS.

3. Extracting the files

Open QuickBMS and it should pop up a console window like this:
It should also automatically pop up an explorer window, which you can use to navigate to the QuickBMS script we got back in Step 1.

Next, it will pop up another explorer window, which we will use to navigate to our battle.eaf file.

Then, it will pop up a final explorer window, which we will use to select our output directory. I just chose that same 'battle' directory.

Now, back in the console window, you should see a whole bunch of output stream by, listing the files that are being extracted.

That's it. You're done. You should have a complete file structure available now, with access to all of the proper files, including character and stage data. Most of this can be opened with the latest version of piecemontee's Asset Explorer, using the new support for big-endian files.

Wednesday, October 6, 2010

SF4 Beginners Guide to Material Attribute Swapping

Credit for this tutorial goes to alekqz, I just cleaned it up a bit.

This tutorial will show you how to give a body part a reflective, metallic look, but a similar process can be used to grab other materials, such as Akuma's glowing eyes.

1. First, open a *.cos.emz file using piecemontee's Asset Explorer and look for the body part you wish to modify, in this case, Zangief's upper body:
Take note of the name used to reference the body part, in this case, 'body.'

2. Next, open a decompressed *.col.emz file in a hex editor (I use the free Frhed, but any hex editor will work) and search for that body part name:
Below the name, you will find the material type attribute, usually something like SpcBumpBrush with some more letters and stuff after it, like this:
3. Now, we want to overwrite that material type with 'SpcEnv_W.' In hex, we want to paste in (53 70 63 45 6E 76 5F 57 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1E 00 00 00).

As with all hex-editing in SF4, it's very important that you overwrite because the resulting files must be *exactly* the same size as the original or the game will crash. It should look like this when we're finished:
4. Next, we need to scroll down to the bottom of that section, just before the next section starts, and look for BrushA, BrushB and BrushC, like this:
We want to replaces those Brush entries with ReflectCoeff (52 65 66 6C 65 63 74 43 6F 65 66 66), ReflectFresnelBias (52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 42 69 61 73) and ReflectFresnelCoeff (3C 52 65 66 6C 65 63 74 46 72 65 73 6E 65 6C 43 6F 65 66 66), respectively (forgive the typo in the pictures; it's Fresnel, not Fesnel):
5. Finally, in the case of metal effects, we have to choose how reflective we want to make the material (also known as 'specularity'). This is controlled by modifying the spaces after the Reflect* entries. For a chrome-type effect, change the last 2 numbers in each section to @ symbols (40 40), like this:
For an enamel-type effect, the last 2 numbers after ReflectCoeff and ReflectFresnelCoeff need to be a Euro symbol and a question mark, like this (the last 2 after ReflectFresnelBias should be blank--or 00 00 in hex):
For a polished effect, the last 4 numbers of the sections need to be changed to the hex code of (CD CC CC 3D), (CD CC CC 3D) and (66 66 66 3F), respectively, like this:
Save your file and you're done!

These are the only 3 metal effects known so far, but there may be more, so be adventurous! Experiment!

Update 10/18/2010: sensibeat tracked down 3 more metal effects:
metal_gold (less reflective):
ReflectCoeff ÍÌL= (CD CC 4C 3D)
ReflectFresnelBias .×#< (0A D7 23 3C)
ReflectFresnelCoeff ÍÌÌ= (CD CC CC 3D)

metal_silver (more reflective):
ReflectCoeff €? (80 3F)
ReflectFresnelBias ÍÌÌ= (CD CC CC 3D)
ReflectFresnelCoeff €@ (80 40)

crystal (highly reflective, mirror-like):
ReflectCoeff ÍÌÌ> (CD CC CC 3E)
ReflectFresnelBias 333? (33 33 33 3F)
ReflectFresnelCoeff ? (3F)

One word of caution: you'll want to modify the DDS textures to look like whatever material you're trying to mimic but don't make it too dark, as the reflection can make it hard to see.

Monday, September 13, 2010

Refutation of G4TV's Metroid: Other M Review

I just got done reading a review of Metroid: Other M by g4tv.com and had a number of disagreements with the author, so I thought I would address them here.

The author takes the stance that, not only is the game technically bad and unfun, but the story presents Samus as a childish, vulnerable brat pining for male validation, thereby shattering her image as a tough, feminist role model, unique amid the sea of helpless-princess-bimbos in video games.

From a technical standpoint, I thought the game was fun and felt like a real sequel to Super Metroid, which the Prime series was never able to do. I agree that the IR-driven scanning/missile dynamic is a little frustrating at times, but I thought it was a decent way to sort of bridge the first-person Prime series and the traditional sidescrolling entries.

The missile/health regeneration dynamic is also a little strange in the context of the series, but it's just a fact of modern combat games, in my opinion. I don't like it in Halo/Gears of War/Modern Warfare/Generic SHOOTAN GAEMS either, but I can live with it.

As for the storyline, this game was much more story-driven than previous games, which often featured no dialog (or really even characters) at all. Now, this is an important point: in previous games, Samus was a (nearly) silent protagonist and it was largely up to the player to decide whether her silence was stoicism or something else entirely.

First off, the g4 reviewer takes issue with the fact that Samus doesn't discover her weapons throughout the environment as she did in previous games, and instead is holding them back on order of her former commanding officer. While the reviewer described this dynamic as "painfully stupid," I find it less so than it would have been to find a bunch of suit-compatible Chozo artifacts scattered around the game's setting, which is just a Federation ship and not an ancient planet. I guess a more satisfying (to me) option would have been for her to decide on her own when she needed to use the items, but that wouldn't make much sense either (why would she *not* use a super missile to open a stubborn door and thus ruin the whole 'metroidvania' mechanic?).

As for the story, I have some significant disagreements with the reviewer's interpretation. She seems to want Samus to be a butch, emotionally void ass-whupper--of which there is no shortage in video games--with her sex being the only discernible difference from Halo's Master Chief. I think this view is belied by previous games, though, where she shows compassion for the baby metroid and allows it to cling to her rather than just blasting it when she has the chance.

Likewise, the reviewer interpreted her deference to Adam Malkovich as weakness, where I saw it as an effort to show respect when she previously had not. She left his command to become a bounty hunter in the first place, so--when she encounters him and her former squadmates at the start of this game--she follows his commands by her own choice. In the cutscenes, we see that she didn't follow his orders very well when she was a cadet, so to follow his orders of her own volition shows maturity, not childishness. Plus, part of the story is that she is reliving painful events from her past in a sort of brutal alien Groundhog Day, which causes her to reexamine her youthful indiscretions.

The reviewer states, "Confronted by her longstanding nemesis, Ridley, [Samus] is spliced into flashes of a little girl, crying and afraid, despite the fact [that] she has already defeated Ridley at least FOUR times already, once when he was a powerful robot." I took issue with this statement because, in an earlier cutscene, Samus mentions her relief that her nemesis was killed in the blast that destroyed Zebes. And then, when she finds Ridley resurrected, she is naturally taken aback and experiences a moment of paralysis as she experiences flashes of her childhood encounter with him. This does not seem out of place to me. Perhaps out of character, but as I said before: she never really had any character to begin with, except what we projected on her from our own imaginations.

In Metroid: Other M, a partial reboot of the series, we get a view into the backstory and motivations of gaming's baddest bitch, but it turns out she's not as badassed as some people thought or would have liked. Perhaps this insight into her mind/feelings was a bad choice on the part of her developers, akin to giving Gordon Freeman a bland voiceover and cornball catch-phrases, but I didn't think it was that bad. Instead, I thought it provided a modicum of motivation to see the story progress rather than the same ol' mindless backtracking to pick up that just-out-of-reach missile pack.

I will concede that I thought the "military obfuscation" aspect of the story was just kinda dropped for the most part toward the end of the game, and I would have liked to see it explored more fully, rather than focusing so much on Samus' feelings. Nevertheless, I found the entire experience to be satisfying and a good, faithful addition to the series.

It's interesting that Nintendo catches a lot of flack for not pushing their games into new directions, yet every time they try, they get slammed for destroying people's preconceived notions of the characters. However, these characters were created at a time when any personalities had to be inferred due to technical limitations, and--as we've all seen with book-to-movie conversions--the "Hollywood" version never stands up to our own imaginations.

Tuesday, August 31, 2010

Ubuntu Maverick on VirtualBox

The third alpha release of Ubuntu 10.10 Maverick Meerkat was just released, so I figured I'd give it a shot in the ol' VirtualBox VM.

The Pros: Mouse pointer integration is built-in, so you don't have to deal with reinstalling Guest Additions every time you do a kernel update.

The Cons: Since Maverick started using a new X-Server (1.9), Guest Additions no longer do their acceleration and resolution magic. This includes manually adding resolutions to the xorg.conf, so until something changes, it's 800x600 all the way...

UPDATE (9/30/2010): Looks like the open source edition of VirtualBox has us covered now. Just open up a terminal and type (all one line):
sudo apt-get update ; sudo apt-get install build-essential linux-headers-$(uname -r) ; sudo apt-get install virtualbox-ose-guest-x11
Thanks for the tip, Anonymous!

I'm putting this here for my own reference, but if you want to automatically mount your host's shared directory in your Linux guest OS, you can edit /etc/fstab thusly:
[host's name of directory] [mount point] vboxsf defaults 0 0
which, for me, translates to:
vbox_share /media/vbox vboxsf defaults 0 0
No more tedious mounting and no mucking around with startup scripts running as root...

Something else I've encountered, not specific to Maverick, but I figured I'd share it here anyway:

I decided to give BTRFS a shot, so I formatted my partitions with it (it still won't support booting, so I had to make a small boot partition with ext4, but / and /home are BTRFS). Unfortunately, there is a hellacious bug somewhere that causes apt-get update/upgrade to be godawful slow (like, 4 hrs. for a big-but-still-reasonable upgrade) with this FS. However, I came across this blog post that suggests using a patched copy of dpkg, which is available from this ppa, which specializes in BTRFS-optimized packages. To add the PPA, add it to your sources in Synaptic or open a terminal and type:
sudo apt-add-repository ppa:brian-rogers/btrfs
Then just update/upgrade as usual. The package comes with a pretty scary warning about how it can hose your system, so you should take frequent snapshots (that's why you're using BTRFS anyway, right? Right??). Unfortunately, this didn't really fix my problem, so it's back to ext4 for me. :(

Once perk of reinstalling using the latest Beta release package is that I got to check out the new installer. It looks really nice and they've done some really smart things with concurrency whereby it starts installing your system before it asks you all of the time-consuming personalization questions. All in all, installation was easy and fast, fast, fast!

Everything else appears unchanged since Lucid.

I'll update this post as I find anything else out. If anyone has anything to add, feel free to leave a comment.

Saturday, August 21, 2010

Ubuntu 10.10 Maverick Meerkat on Acer AspireOne

One caveat before you read any further: my AspireOne is old as hell (9" w/ 8GB SSD) and these problems/fixes may not work on newer models.

Ok, now with that out of the way, I've got some good news: Maverick works great on my AO. Installation was fast and painless (it only took around 20 min., I think), and booting is super-quick (something like 15 seconds to a working desktop in my experience). Wireless works out-of-the-box, as does the wifi activity light :O. Unfortunately, the SD card slots are still b0rked. To get them working, we can use the same fix from my previous AO/Ubuntu post:

Open up a terminal and type:
sudo gedit /etc/modprobe.d/options
Then paste in:
options sdhci debug_quirks=1
Reboot and you should be all set.

I also ran into a problem trying to create shared directories using samba. The system automatically installs the samba packages just fine, but when I try to actually make a share, it fails with the error "Failed to execute child process "testparm" (No such file or directory)." If this problem affects you, too, hop back into your terminal and type:
sudo ln -s /usr/bin/testparm.samba3 /usr/bin/testparm ; sudo ln -s /usr/bin/net.samba3 /usr/bin/net
This should get you all fixed up. You don't even need to reboot again.

Some other things to note about this new release:
1. Virtualbox no longer needs Guest Additions installed to utilize mouse pointer integration.
2. The new unity netbook interface sucks. I think the idea of putting the launcher bar on the left is great, and I like the look of it, but I could never track down where they had the gnome-terminal hidden, and it wouldn't show up when I searched for it in their application search applet. Also, hitting Alt+F2 wouldn't bring up the execution prompt, so after a few minutes of frustration, I uninstalled it and went back to the standard interface.

I'm happy to see the improvements they've made in this new release and I look forward to seeing what else comes in before the upcoming feature freeze. If you run into any other problems with this release, drop me a comment and I'll help however I can. Also, keep an eye on my PPA for packages of bsnes v1.0, which should be optimized enough for use on your AspireOne netbook.

Wednesday, August 18, 2010

The Runup to bsnes v1.0

As a kid, I was lucky enough to have grown up during the Golden Age of console gaming. I am old enough to have played some Ataris and my older brother had a regular Nintendo Entertainment System when I was really young, but the first console I personally owned was the Super NES (or SNES).

The SNES was a pivotal piece of hardware in console gaming. Prior to the 16-bit era, hardware limitations in gaming consoles necessitated that graphics be highly representational and storage constraints were a burden on many developers. This all changed with the SNES, which could produce graphics that were good enough to not impede storytelling and which used cartridges that had enough storage to encapsulate some truly lengthy and epic games.

Early SNES Emulation

Just as the SNES heralded a new period in console gaming, the emergence of popular SNES emulators in the late 1990s changed the landscape of the PC console emulation scene. ZSNES and SNES9x pushed to the forefront of the scene with good compatibility for most popular, commercially available games through a variety of game-specific hacks and the ability to run full-speed on the prevalent hardware of the time (90 mhz Pentium IIs).

However, despite the compatibility hacks, these early emulators experienced a number of emulation bugs that range from minor to rather serious. For example, ZSNES runs games at approximately 140% speed and some games experience graphical glitches. Many games that use special hardware, such as SuperFX chips, run very slowly or not at all. Some more obscure games, such as Speedy Gonzales, experience show-stopping bugs that completely halt progress in the game. Nevertheless, gamers accepted these foibles and adoption of these early emulators was widespread. As time went on, though, the authors' enthusiasm for squashing bugs waned and development on these early emulators slowed to a crawl. Furthermore, the code behind these early emulators was esoteric and poorly documented, making a significant barrier to entry for potential newcomers to contribute to their development.

A New Challenger

Fast forward to 2004.

A ROM-hacker that goes by the name 'byuu' became interested in writing his own SNES emulator from scratch in his spare time, so he could better understand how the SNES works. Instead of focusing on speed, byuu focused on accuracy, debugging functionality and clean code. This shift in focus was revolutionary for several reasons:
1.) Compatibility would no longer require hacks. If the emulation of SNES hardware is perfect, it stands to reason that all games that work on real hardware should work on the emulator.
2.) The code would be self-documenting and would also document the hardware of the SNES. Someone reading the code could see not only that 'this is how the emulator does it' but that 'this is how SNES hardware does it,' which is of utmost importance to digital archivists and preservationists.
3.) System requirements would be extremely high, putting such an emulator far out of reach of older systems, even those that could run previous emulators at full speed.

The first public release of this emulator, dubbed bsnes, was in May of 2005. At the time, the code was completely unoptimized and bsnes still experienced a variety of issues that were not present in previous emulators. However, byuu made rapid progress and a number of other talented programmers, including GiGO--the author SnesGT, another SNES emulator--and blargg--the author of QuickNES--supplied components for cycle-level emulation of different pieces of SNES hardware.

Unfortunately, these early releases had very steep requirements to run (Richard Bannister's port of bsnes to Apple's OS X initially required a then-top-of-the-line G5 CPU to even approach playable framerates), and many early users were turned off by the project, assuming it was simply a proof-of-concept. Eventually, though, PC hardware advances and byuu's code optimizations met somewhere in the middle, with bsnes emerging as a playable, extremely accurate SNES emulator.

Soon, bsnes had become the first and only SNES emulator capable of full SPC7110 emulation, SPC700 cycle-level emulation and full Super Gameboy hardware emulation. Even cheat codes were implemented faithfully by emulating the way a Game Genie would interact with actual SNES hardware. Additionally, byuu's clean code and commitment to openness allowed bsnes to be ported to all major, modern operating systems, including MS Windows, OS X and Linux.

Modern Innovations

After reaching maturity, bsnes gained many of the features users enjoyed in ZSNES and SNES9x, such as software filters that would improve the look of pixelated SNES sprites on high-definition, flat-panel monitors, as well as some modern features that were not possible in earlier emulators, such as hardware-based shader effects.

By June 2010, bsnes had achieved 100% compatibility with all commercially released games while still being playable on standard, albeit modern, hardware, so byuu set out to tackle one of the final frontiers of SNES emulation: a dot-based S-PPU rendering core.

Up to this point, bsnes, like all other SNES emulators before it, had used a scanline-based rendering core, which would fetch and display data entire scanlines at a time. In contrast, actual SNES hardware would only fetch and display data one dot at a time. While this did not make much difference in a majority of games, some games, such as Air Strike Patrol, used dot-level calculations to show certain effects; in ASP's case, the shadow of the plane (using scanline-based rendering, ASP shows no shadow at all, which makes it really tough to accurately drop bombs on targets). Likewise, some games used dot-based effects to create a faux transparency, similar to the artistic technique of pointillism.

Unfortunately, using this new hyper-accurate rendering engine incurred a nearly 40% reduction in emulation speed, once again making full-speed gameplay difficult-to-impossible, even on state-of-the-art hardware (at the time of this writing, including Intel i7 CPUs). This left byuu with a dilemma: continue using the most faithful, accurate emulation code available at the risk of again alienating users and creating what amounts to an unplayable benchmark or sacrifice accuracy to maintain playability.

The Future of bsnes

This crossroads led to much discussion on byuu's message board, with some users suggesting he go with the most accurate code and just wait for the hardware to catch up, while others suggested he fork the code into two separate projects, an accurate emulator and a fast one.

Rather than compromise on any of his core goals for bsnes, byuu decided on a third option in which he would enable users to choose at runtime among three distinct processing cores for bsnes: an 'accuracy' core that focuses entirely on accuracy and is only intended for research purposes, a 'compatibility' core that maintains modest requirements with high compatibility (this core is similar to that of bsnes v0.067), and a 'performance' core that makes small sacrifices in accuracy and code readability to make bsnes faster than it has ever been (so fast, in fact, that it can run most games at full speed on an Intel Atom 240).
These three cores will be included in the upcoming 1.0 release of bsnes, according to byuu. UPDATE: The three cores were included in v0.068! Windows users can download it at byuu.org/bsnes and Ubuntu users can get it from my PPA.

While he has not given any estimated release date for version 1.0, he has already begun beta testing the cores against each other and against actual SNES hardware to spot any remaining bugs. Additionally, leading up to v1.0, byuu has solidified his API for libsnes, a portable core library at the heart of bsnes, which can be used to drive lightweight, platform-specific graphical interfaces created by the community.

If any Debian/Ubuntu users would like to try bsnes, I have builds of the latest publicly available code published in my PPA.

UPDATE (3/11/2015): heh, the 1.0 numbering convention never came to pass. At the time of this writing, we're at v095... libsnes has also been abandoned and then forked into libretro.

Wednesday, August 11, 2010

piecemontee's SF4 Viewer Now Open Source!

Piecemontee is a swell guy who created one of the more vital tools to Street Fighter 4 modding, known variously as the SF4 Viewer or SF4 Asset Explorer. This tool allows modders to decompress SF4 files (no more offzip! yay!), extract textures (no more dragon unpacker! yay!) and obj models (unique to the Asset Explorer AFAIK...yay!), and then reinject modified versions of these assets back into the game (no more infuser! yay!). It also enables modders to view their work within the program instead of having to load the entire game up each time.

The version of the SF4 Viewer that is most widely used is 0.33, but piecemontee made some significant strides since that release and has since released another stable version (0.35) and an unstable beta (0.36ik; you can get these and other versions here). From piecemontee (regarding the unstable version):
[I]t includes preliminary animation display (open up .cmn.emz / .cos.emz and .col.emz then go to the EMO model to see a new combo to select the stance)
I tried to display BCM input commands as well but I don't remember how far I got I planned to allo input commands edition
Warning this version is buggy as hell and if I remember well you cannot resize the window after loading a model

Here's a screenshot of the animation viewer in action:
Yeah, it's a little fucked up, but it's a start!

In addition to these newer versions, piecemontee gave the modding community the best gift anyone could ask for: he opened up the source code for anyone to work on or add new features! The program uses DirectX 9 and pure Win32, so anyone who has experience with these APIs can head on over to SourceForge and check out the code, via mercurial.

Piecemontee told me that he doesn't have the time nowadays to continue hacking away at the SF4 Viewer. However, now that the source code is available, there is the possibility that someone will modify it to work with assets from Super Street Fighter 4 on Xbox360. According to a post over at the XeNTaX forums (from this thread), all it probably needs is for someone to switch the "endianness" (from big to small or vice versa, I guess) and it will be able to view/import things just fine. At that point, we should be pretty close to importing assets from SSF4 into vanilla. :D

If anyone wants more information on this or is interested in adding new features, feel free to leave a comment here, PM me on the Shoryuken forums (my handle there is hunterk), or give me a buzz via sf4mods.pbworks.com.

Sunday, July 18, 2010

Calibre E-book Library on Ubuntu Lucid with Amazon Kindle

Amazon just had a big sale on 2nd-gen Kindle E-readers, so I picked one up. I am completely happy with the reader and I love the screen, but I am not so happy with the relatively small number of file formats the Kindle supports. I have picked up quite a few ebooks over the years, mostly in Epub and .lit formats, but with sprinklings of a number of other formats, as well, none of which are natively recognized by the Kindle.

Luckily, there's a wonderful piece of software available, known as Calibre, which is capable of converting the vast majority of the myriad ebook formats into the digitally signed .mobi format that the Kindle loves so much.

In addition to the conversion capability, Calibre is essentially an ebook library management tool similar to what is available for music files, such as iTunes, winamp, etc., with support for metadata, covers and so forth. It also has syncing support for a variety of e-readers, including the Kindle, the iPad, Sony's E-Reader and more, though some require additional software support.

As far as I know, Calibre is the only software of its kind and it is, thankfully, free, open-source and multiplatform. You can download it here, or from Ubuntu's official repositories. If you get it from the repositories, though, be aware that it does not come it's own copy of libunrar.so, which will cause it to choke on compressed archives, such as .cbr comic book archives and any otherwise compatible ebook files contained within .rar archives, usually with an error something like this:
OSError: libunrar.so: cannot open shared object file: No such file or directory
This error will occur even if you the unrar and unrar-nonfree packages installed, and I am not aware of any packages that will provide libunrar.so. However, you can download a precompiled (32-bit) version here.

To install it, decompress the .tar.gz archive by right-clicking on it and selecting 'Extract Here.' This will decompress the archive into a directory that is misleadingly named libunrar.so (the real library binary is inside *that* directory). So, open a terminal, navigate to the newly decompressed directory and type:
sudo cp libunrar.so /usr/lib/
After that, you should be ready to run by either choosing it from the 'Office' section of the Ubuntu menu or by typing calibre into a terminal.

Once you get it up and running, the preferences are extensive. You can choose how it interprets metadata from filenames (I still can't get this configured properly for some reason; it always either pulls the names in wrong or swaps the names and the titles), how converted books are formatted and an array of other options.

You can set up an email address (or use an existing email server) to send books wirelessly to supported ereaders, but be careful if you own a Kindle because Amazon charges for anything that gets emailed to your device (15 cents per MB at the time of this writing). This in mind, I'll be sticking with USB transfers.

My personal favorite feature of Calibre is the Web interface, which makes your library available to you anywhere in the world via Web browser. To set it up, you just turn on the included Web server via Preferences > Content Server and set up a username and password to use for login. If you have a dynamic IP address (as is the case with most cable internet), you'll also need to get a static IP address from one of the many providers, such as dyndns.com, and port-forward port 8080 through your router. This server feature works well with the minimize-to-the-notification-area feature, which makes it less obtrusive while running in the background.

Furthermore, for people with an iPad, the free ereader app Stanza can connect to your Calibre Web library to sync, which is very cool.

What with the conversion, organization and available-anywhere-there's-internet server, Calibre is a pretty awesome piece of software. However, I do have some complaints:

No native .doc support
. It would have been really nice if Calibre could at least convert .doc files to other formats, presumably via an external library (similar to the rar support) or an API. Maybe the author could work out something like this. As it is, it just coughs up errors if you try to convert them and opens OpenOffice.Org if you try to view them.

Crashy as hell. Any time I try to do anything with a batch job, Calibre freezes and crashes. This is a serious annoyance, but I think it'll be fine once I get everything set up. I expect that, once I get everything tagged and converted to .mobi, I won't have to use any of the really destabilizing features--specifically, the batch stuff--with any regularity.

Wednesday, July 7, 2010

What's New In the Filthy Pants PPA

I originally created my Personal Package Archive (PPA) to distribute my SVN builds of HandBrake, which has worked out really well. However, I have since added a number of programs, the functions of which may be a bit esoteric to some, so I thought I would take some time to explain what each program does (in alphabetical order):

Anomos

An anonymous, encrypted BitTorrent client built using free and Open Source code. It uses an advanced, planned mix network to protect users' identities, though it will not magically make your illegal torrents safe.

The client routes packets through a number of intermediary nodes before they reach their final destination. The intermediary nodes cannot confirm what information their neighbors are sharing or whether they are the source of the information or simply a relay point. Additionally, all communication with both peers and trackers via Anomos is end-to-end encrypted.

Here's a screenshot:

BSNES

In the simplest terms, BSNES is an emulator for the Super Nintendo Entertainment System (SNES). However, it was written primarily by a determined individual known as byuu whose goal was to create the cleanest, most accurate SNES emulator in the world, using entirely from-scratch code (i.e., without reusing code from existing emulators, such as SNES9X).

BSNES is so accurate, in fact, that it has reached nearly 100% compatibility with commercially produced games--even the ones with crazy, specialized chips, like Star Ocean--without the use of buggy, game-specific speed hacks.

Unfortunately, great accuracy comes at the price of insanely high hardware requirements: to get full 60 fps, users will need a high-clocked Core2Duo or AMD Phenom/Athlon II cpu.

BSNES supports a number of attractive video filters, including HQ2X and SuperSAI, as well as OpenGL-based pixel shader filters with compatible graphics cards and drivers.

I include 3 separate packages for bsnes in my PPA: the standard Phoenix/GTK-based GUI (bsnes-gtk), the older, now-deprecated Qt GUI (bsnes) and an optimized build of the Qt GUI (bsnes-sse2).

Here's a screenshot:

Curse

Another piece of software from BSNES author byuu, Curse is a barebones, no-frills hex-editor. It must be invoked from the command line (e.g., curse [file to edit]) but will then open a window displaying the hexadecimal data. Like BSNES, Curse's code is written with a focus on cleanliness and efficiency. It's missing some features that would be handy, but it's still one of the only programs of its kind on Linux that doesn't require Mono....

Here's a screenshot:

Dolphin-Emu

Dolphin-Emu (not to be confused with Dolphin the file manager) began its life years ago as a Nintendo GameCube emulator. When Nintendo decided to keep the Wii's architecture relatively unchanged from that of the GameCube (the proverbial "two GameCubes duct-taped together"), the emulator was adapted to work with Wiimotes via bluetooth and Dolphin-Emu became the first game console emulator to play commercial games at full speed during the life cycle of the console.

In addition to full-speed emulation of many commercial games, Dolphin-Emu offers support for high-definition upscaling of graphics, along with high-resolution textures and support for shader-based filters.

Here's a screenshot of it playing Ikaruga:

HandBrake

A highly useful video transcoding tool designed originally to rip DVDs but later updated to support transcoding of nearly any file that can be decoded by FFmpeg/libavcodec.

The program is geared mostly toward the use of the open-source x264 codec, but it also supports FFmpeg's mpeg4 and Ogg/Theora codecs. It also supports AC3 and DTS passthru, as well as support for soft subtitles and some support for vobsubs.

My packages are updated whenever major features are added or codecs are updated.

Here's a screenshot:

HandBrake-QT4

This was an attempt by a fellow named gonza to update HandBrake's extremely deprecated QT4 GUI code to something functional. Unfortunately, he didn't stick with it long enough to satisfy the other HB devs and his code was never included in the official SVN repository.

This package is here in case anyone wants to use it, though it is sorely outdated.

LibWiiUse

This is a library used to interface with Wiimotes via bluetooth. It is required by Dolphin-Emu and is supposed to be built automatically as part of its build process. Unfortunately, this did not work on the Launchpad build farm, so I just bundled it separately and included it as a dependency for Dolphin-Emu.

NEStopia

This is a build of the highly accurate, cross-platform NES emulator, NEStopia. This emulator is cycle-accurate and supports some nice filters, including blargg's awesome NTSC filter, which is also available in bsnes.

Here's a screenshot:

Nvidia-Cg-Toolkit (no longer available)

This is a set of proprietary libraries and tools created by Nvidia. They are used in several modern emulators, including Dolphin-Emu and PCSX2. Since the source code is not available for these tools, the package contains the binaries for both i386 and AMD64 architectures, and the appropriate ones for your system will be copied into the appropriate directories.

PCSX-DF

This is a Debian fork of the popular PCSX Sony Playstation emulator. It is highly compatible with commercial games and uses plugins to add features and functionality. This is a more recent version than the one included in the Ubuntu Universe repository.

pcsxr

Another fork of PCSX, this variation has a bit more active development.

Here's a screenshot:

rtmpdump (no longer available)

This is a toolkit for RTMP streams, with all forms of RTMP supported, including rtmp, rtmpt, rtmpe, rtmpte and rtmps. In addition to rtmpdump, this package includes the server programs rtmpsrv and rtmpsuck. These tools can be used to listen in on rtmp server handshakes and give access to unencrypted streams. This allows users to download encrypted and obfuscated streams, such as those used to stream television shows from network websites (e.g., southparkstudios.com).

snespurify

This tool automatically removes headers from ROM images and is a must-have for use with bsnes' Phoenix/GTK GUI, which does not support libsnesreader and all of the fancy file formats that entails (.smc and all of the other myriad SNES file extensions, as well as compressed archives). This version presents a fancy, polished GUI that facilitates large-scale, batch purification.

Here's a screenshot:

Upset

Yet another piece of software from BSNES author byuu, Upset is a lightweight UPS patcher for use by ROM-hackers. As with his other software, Upset's code is focused on portability and cleanliness.

VBA-M

An actively-developed variant of the popular Gameboy Advance emulator, VisualBoy Advance. It has a nice GUI and performs a bit better than the standard VBA package.

Here's a screenshot:

VLC (no longer available)

This media player is based on FFmpeg/libavcodec and will play almost any video file you throw at it, without the need for external codecs. While there is already a package included in the official Ubuntu repositories, this updated version includes support for Google's WebM format.

Wednesday, June 23, 2010

Nvidia 256.35 Driver on Ubuntu 10.04 Lucid Lynx

(If you just want installation instructions, skip down to the section labeled 'How to Install It')

I don't normally devote entire blog posts to something as trivial as a driver update. In fact, I usually don't update my proprietary driver blob until Ubuntu updates its packages and pushes them out through the software update tool, but this particular driver update has brought about a couple of major changes (for me) that I thought warranted stepping out of my comfort zone a bit.

These features are:
1.) From Nvidia's website, "Fixed an interaction problem between Compiz and 'screen-scraping' VNC servers like x11vnc and vino [ed.: the VNC server that Ubuntu uses by default] that caused the screen to stop updating."

This bug has been around since Hardy Heron and has been a thorn in my side. The only workarounds were to use a different VNC server (big hassle), constantly close/reopen the VNC connection to manually refresh (even bigger hassle), or switch to metacity instead of Compiz whenever you want remote access (not a huge hassle, but still inconvenient).

2.) The driver now supports overscan compensation (via a slide-control in the nvidia-settings application) for displays that have overscan issues, though this isn't mentioned in the changelog specifically.

This feature has been in the Windows driver control (and the Linux Catalyst driver for ATI users) for a very long time, but it is new for the Linux Nvidia driver and solves another long-standing issue for me. Now, I no longer need to cover the screen edges with transparent GNOME panels to keep new windows from drawing offscreen or worry about toolbars getting chopped off of fullscreen applications. W00t! (Edit: overscan compensation has been in the driver since Karmic, but it never showed up for my specific card until this latest driver).

The new driver also includes a host of new VDPAU improvements, which always a good thing.

How to Install It

Update (7/27/10): If you don't feel like doing all of the steps I have laid out, there's a handy PPA provided by the same cats who package the standard Ubuntu graphics drivers that has a package for the 256.35 driver. You can find it here. Whether you use the PPA or my steps, the Bonus sections below are still applicable and handy.

I have an Nvidia 8600 GT graphics card and, when I first tried to install the driver, I followed the procedure that has always worked in the past of simply downloading the installer, dropping down to console, and running the installer. However, when I tried that, the installer failed with the error "THE DISTRIBUTION-PROVIDED PRE-INSTALL SCRIPT FAILED." Others have apparently also gotten another error, which I will include here for search engine purposes: "Error: Unable to load the kernel module 'nvidia.ko'. This happens most frequently when this kernel was built against the wrong or improperly configured kernel sources, with a version of gcc that differs from the one used to build the target kernel, or if a driver such as rivafb/nvidiafb is present and prevents the NVIDIA kernel module from obtaining ownership of the NVIDIA graphics device(s), or NVIDIA GPU installed in this system is not supported by this NVIDIA Linux graphics driver release."

In either event, these steps should clear things up:

1.) First, we'll gather some resources: download the driver from Nvidia's website and put it somewhere, such as your Home directory. Then, open a terminal and type:
sudo aptitude install linux-headers-`uname -r` build-essential nvidia-settings
2.) Now, we need to get rid of some modules that conflict with our soon-to-be-installed proprietary driver. So, in a terminal, type:
sudo gedit /etc/modprobe.d/blacklist.conf
Scroll down to the bottom and copypasta these lines:
# Allows proprietary Nvidia driver to install
blacklist vga16fb
blacklist nouveau
blacklist rivafb
blacklist nvidiafb
blacklist rivatv
Save and exit.

3.) Next, we need to clear out any previously installed Nvidia drivers, so (still in a terminal) type:
sudo apt-get remove --purge nvidia-*
4.) Reboot and, when the error window pops up saying Ubuntu can't load drivers blah, blah, blah, choose to 'Exit to console,' and it should dump you down to a command prompt. Login and navigate to the directory where you put your Nvidia driver installer from Step 1 (if you put it in your Home directory, you should already be there upon login).

5.) Start the Nvidia driver installer by typing (hint: you can use the TAB key to autocomplete instead of using the * wildcard character):
sudo sh NVIDIA-*
When it asks if you want it to automatically configure things, select 'yes' and it will auto-generate an xorg.conf file with the appropriate fields.

Reboot again and everything should be hunky dory (if not, and your installation still poops out, scroll down to the bottom of the post for some more things you can try).

Bonus Points

Unfortunately, if you install the driver this way, each time a new kernel update comes down the pipe, your Nvidia driver will stop working and have to be reinstalled, which can be a big bummer. However, you can set up a script that will run anytime the driver needs to be reinstalled:

1.) First, we'll move our Nvidia driver installer to a safe location. In a terminal, navigate to wherever it is and type (again, use TAB to autocomplete instead of the wildcard):
sudo mv NVIDIA-* /usr/src
2.) Next, we want to make a symlink (kinda like a more powerful shortcut) to the installer so our script won't need to be rewritten any time a new Nvidia driver comes out. In a terminal, type:
sudo ln -s /usr/src/NVIDIA-* /usr/src/nvidia-driver
Now, since our script will reference the 'nvidia-driver' symlink we can just overwrite the old driver and make a new symlink using the same name if there's a driver update.

3.) Now for the script itself. Open a text editor and, in a new file, copypasta this:
#!/bin/bash
#

# Set this to the exact path of the nvidia driver you plan to use
# It is recommended to use a symlink here so that this script doesn't
# have to be modified when you change driver versions.
DRIVER=/usr/src/nvidia-driver


# Build new driver if it doesn't exist
if [ -e /lib/modules/$1/kernel/drivers/video/nvidia.ko ] ; then
echo "NVIDIA driver already exists for this kernel." >&2
else
echo "Building NVIDIA driver for kernel $1" >&2
sh $DRIVER -K -k $1 -s -n 2>1 > /dev/null

if [ -e /lib/modules/$1/kernel/drivers/video/nvidia.ko ] ; then
echo " SUCCESS: Driver installed for kernel $1" >&2
else
echo " FAILURE: See /var/log/nvidia-installer.log" >&2
fi
fi

exit 0

Save the file with the name update-nvidia and exit the text editor.

4.) Now, we want to make our script executable, so, in a terminal, navigate to wherever you made your script and type:
chmod a+x update-nvidia
5.) Finally, we need to install it. Still in your terminal from the last command, type:
sudo mkdir -p /etc/kernel/postinst.d ; sudo install update-nvidia /etc/kernel/postinst.d
Now, you should be all set. Whenever you install a new kernel, this script should kick in and install the Nvidia driver for you.

Even More Bonus Points

Now that we've got our driver installed and ready to update itself, we should consider our power consumption! Usually, the Nvidia driver will default to a power profile that always leaves your card running full-tilt, which sucks battery life and raises your utility bill. Thankfully, there's something we can do about it.

Open up a terminal and type:
sudo gedit /etc/X11/xorg.conf
This will get us into the auto-generated configuration file created by the driver installer.

Scroll down to the Nvidia driver section (you can tell it's the right section because it will be labeled 'Device,' the 'Vendor' field will say 'Nvidia,' and the 'Driver' field will say 'nvidia') and add these three lines:
Option "RegistryDwords" "PowerMizerEnable=0x1; PerfLevelSrc=0x3333"
Option "Coolbits" "1"
Option "OnDemandVBlankInterrupts" "True"
Save the file and exit.

In the first line, the PowerMizerEnable switch tells the driver that we're interested in using the dynamic clocking features (this switch may not be entirely necessary, as PowerMizer is supposed to be enabled by default, but it can't hurt anything and may be required for some cards/cases), while the PerfLevelSrc field tells it which performance level we want to use. The 0x3333 setting means we want dynamic, on-demand clocking for both battery and AC power.

The second option we added enables the 'Coolbits' feature, which lets us manually control the clock and memory speeds on our card via the nvidia-settings application.

The third option enables a checkbox in the nvidia-settings application to allow or disallow vBlank interrupts. Allowing them (checking the box) reduces the number of wakeups from idle, and thus reduces power consumption somewhat.

Now, in my case (8600 GT), there is only one performance level and dynamic clocking does nothing (you can find out your performance levels by typing in a terminal: nvidia-settings -q GPUPerfModes -t). This really stinks, since I do all of my gaming in Windows and all I really need my video card for in Ubuntu is compositing and fancy desktop effects. With this in mind, I used the Coolbits option via the nvidia-settings application to drastically underclock my card. It now runs at approximately 150 mhz on the core clock and 180 mhz on the memory, which generates far less heat and uses way less power than before. Compiz still looks great and has no performance issues, even while playing HD video, rotating the desktop cube with 3D windows, etc.

If Installer Still Fails
(I didn't have these problems, so I didn't try these instructions myself. They are only here in case you need them)

So, if the above instructions didn't work, we still have some things we can try:

1.) Drop back down to the command line (by pressing Alt+F1), login and type:
sudo gdm-stop
and then
sudo apt-get remove --purge xserver-xorg-video-nouveau
2.) Now, we need to fiddle with some blacklisted framebuffer stuff, so type:
sudo nano /etc/modprobe.d/blacklist-framebuffer.conf
Once in, we need to comment out (put a # in front of) blacklist vesafb and add a new line that says:
blacklist vgafb16
Save and exit (hit ctrl+x).

3.) Now, type:
sudo nano /etc/initramfs-tools/modules
Scroll down to the bottom and add these two lines:
fbcon
vesafb
Save and exit, then update the initramfs by typing:
sudo update-initramfs -u
4.) Now, we need to tell grub about our framebuffer, so type:
sudo nano /etc/default/grub
Once in, search (ctrl+w) for GRUB_CMDLINE_LINUX= and add either vga=771 or vga=795 (which one you'll add depends on your resolution, so for now, just pick one). Save and exit, then update grub by typing:
sudo update-grub
If things are messed up, you can redo the above step using the other vga value to hopefully correct things.

Now, once you reboot, you should be able to pick back up from Step 4 in the original instructions and install the Nvidia driver normally without any further errors.

Analytics Tracking Footer