I lease a prime location, buy high quality equipment, and bring aboard some experienced (and expensive) chefs. But instead of letting my chefs build out the menu, I give that responsibility to a small team of Meal Designers.

These people are passionate about food; they love to eat it, and to talk about it. They are familiar with current food trends, and regularly try new dishes from nice restaurants around the city. In fact, they’re a lot like the chefs. The difference is that they don’t actually make food, and plenty of them don’t think it’s important to know how.

This causes a number of problems. For a start, they have a hard time communicating to the chefs how the elements of their meals should taste, smell, and feel. They also have a hard time articulating what they like or need changed. Dishes are worked out by a long process of iteration, which is made more difficult because cooking involves making tradeoffs, and changing the salt level (for example) might affect the texture.

They also don’t have a complete understanding of what sort of things are hard to make, and which are easy. Or which things can be cranked out in big batches by a line cook, and which much be done to order by a more skilled worker. Or which techniques are “forgiving” in a hectic kitchen, and which require strict attention to come out properly (reducing overall kitchen bandwidth). Or what can be done readily with existing kitchen equipment, and what would require costly upgrades to do efficiently. Or the alternatives that might be available for a dish, to get a similar result with lower food cost, less labor, fewer expensive specialists, shorter cooking time, or less risk. You get the idea.

By the time opening day rolls around, the meal designers will generally be pretty satisfied with what they’ve got, though the menu will be shorter than they had hoped. The kitchen will have been trashed a bit by the elaborate cooking procedures that evolved in response to design changes as the dishes were tuned. Everyone will be a bit burned out. And most people will wish they could go back and start over, given all that they had learned, and do things a bit differently.

That’s crazy, right? Well, that’s how a lot of big budget games are made. In fact, as projects get larger, the hierarchy that emerges makes it more likely to be true. And in some places, the line between designer and manager is a blurry one, which puts the designers in charge of running the kitchen too.

Now, before my game designer friends get all huffed up, I’m not trying to pick on you, and I have to point out that this goes both ways. A chef can learn every technique under the sun, but still not have the taste or imagination to create new food. Plenty of developers spend all their time repairing small pieces of a big machine, but have no idea how to make “fun” happen. How empty would it be to spend your career building guitars, but never learn to play music? Code doesn’t make video games, any more than wood can make songs. Those things are just tools… the good stuff only happens when they move.

However, I do think that if you want to design games, you have a responsibility to understand how they work, and be at least able to cobble together prototypes independently. This is your medium, after all. You don’t have to be an elite hacker. The important thing is to have some clay under your nails. Designing systems is hard, and getting good at it requires a lot of practice and failure. You need to be able to do that for yourself, or you’re going to waste a lot of time. Trust me, you can do it.

Good games are art machines. They are sensitive living systems. They are huge, twisted robots with shiny faces, made out of clicks and taps and a little bit of luck. If we want to make great ones, we all need to embrace both the craft and the art in what we do, and whether you’re a developer or designer, there is plenty of both required.

Anyway, that’s what I think.

[If you'd like to follow my own development work, go to pureenergygames.com and subscribe to the RSS feed!]

A good way to get an authentic look for retro-pixel art is to simulate the distortion caused by encoding the image into an NTSC signal, decoding it again (as a TV would), and projecting it onto a virtual CRT. This gives you natural-looking artifacts, like fringing and color bleeding.

It also makes copyrighted hedgehogs look even more dashing.

Console emulators do this sometimes, and if you’re old enough to have actually played games on a CRT TV, it really helps with the sense of immersion. This post gives a quick overview of the process, in case you’d like to try it for yourself. All of these steps are texture operations performed by pixel shaders.

1. We start by encoding the low resolution input image as an NTSC signal. Each input line is converted into voltage over time, in the same format an NTSC signal would be sent across a wire (except for the sync and color pulse stuff).
2. A “cable reflection” shader smears the signal out a little to the right. I’m not sure how much it looks like cable reflection, but it does kind of evoke the streaking artifacts you see on some old TVs.
3. The luma is split out of the signal, and then used in the NTSC decoding process. This is also where the standard OSD parameters (brightness, contrast, sharpness, etc) are applied. Now our image is RGB again.
4. The image is projected onto a curved tube. This step also takes care of tracing the scan lines and applying the phosphor pattern.
5. The phosphors from the previous frame are decayed, and the new values are accumulated. This allows for ghosting of moving images.
6. A standard post-processing stack is applied (bloom, glare, and tone mapping). This give users a taste of the eye-burning glow produced by a real CRT. (Do you remember when staying up late to play games caused physical pain? Kids these days are soft.)

Naturally, there are a few problems.

Moire

AUGH, TEH MOIRE.

The combination of the scanlines and the phosphor texture make a little bit of moire pretty much unavoidable. Tuning the brightness, contrast, bloom, scaling, or NTSC parameters can produce moire, or move it from one place to another. The good news is that it looks far worse in screen shots than it does on a running game.

(I cranked it up for the screen shot above… it’s not that ghastly under normal circumstances).

Crosstalk

Crosstalk between chroma and luma is an key part of the effect, so it’s a feature, not a bug. The problem is getting it to look bad in a “good” way.

A band-stop FIR filter can be used to chop out the chroma signal, but it’s tough to find the right balance between soft (filter too wide) and stripy (filter too narrow).

Phosphor resolution

My original goal was to have RGB phosphors visible in the image when you examined it up close. That’s really hard, because if you make the phosphors small enough to look realistic, the RGB pattern blurs out and all you can see is a sort of vertical striping. If you don’t mipmap or supersample the phosphors, you again get more moire.

These examples have the phosphor texture enlarged and strengthened to exaggerate the problem. I wasn’t able to get individual phosphors to look good at 720, and they are only barely tolerable at 1080.

Balance

Cranking all these techniques to the max gives you a delightfully bad video signal, which is also no fun to look at for more than 10 seconds. There is also a fair bit of interplay between the parameters: adjusting the scanline gap changes the brightness, and so on. Tuning all this can be a touchy process.

Below are the parameters that control the effect. As you can see, there are a lot of ways I can screw things up.

Up next

In future posts I’ll get into the technical details and show you what the shaders look like. If you’re interested, go to pureenergygames.com and subscribe to the RSS feed!

A very long time ago, I was curious how to detect the strength of the bass and treble in music, in order to synchronize some graphical effects. I had no idea how to do such a thing, so I tried to figure it out, but I didn’t get very far. Eventually I learned that I needed something called a Fourier transform, so I took a trip to the library and looked it up (which is what we had to do back in those days).

What I found was the Discrete Fourier Transform (DFT), which looks like this:

This formula, as anyone can see, makes no sense at all. I decided that Fourier must have been speaking to aliens, because if you gave me all the time and paper in the world, I would not have been able to come up with that.

Eventually, I was able to visualize how it works, which was a bit of a lightbulb for me. That’s what I want to write about today: an intuitive way to picture the Fourier transform. This may be obvious to you, but it wasn’t to me, so if you work with audio or rendering, I hope there’s something here you find useful.

Disclaimer: my math skills are pitch-patch at best, and this is just intended to be an informal article, so please don’t expect a rigorous treatment. However, I will do my best not to flat-out lie about anything, and I’m sure people will set me straight if I get something wrong.

A quick bit of background – what does the Fourier transform do? It translates between two different ways to represent a signal:

• The time domain representation (a series of evenly spaced samples over time)
• The frequency domain representation (the strength and phase of waves, at different frequencies, that can be used to reconstruct the signal)

The picture on the left shows 3 cycles of a sine wave, and the picture on the right shows the Fourier transform of those samples. The output bars show energy at 3 cycles (and, confusingly enough, negative 3 cycles… more on that below).

The inputs and outputs are actually complex numbers, so to feed a real signal (like some music) into the Fourier transform, we just set all the imaginary components to zero. And to check the strength of the frequency information, we just look at the magnitude of the outputs, and ignore the phase. But let’s never mind all that for now.

What are we trying to accomplish? We’ve got a sampled signal, and we want to extract frequency information from it. The Fourier transform works on a periodic, or looping signal. This seems like a problem, since we don’t actually have any signals like that. In practice, you just take a small slice of a longer signal, fade both ends to zero so that they can be joined (which is a whole topic unto itself), and pretend it’s a loop.

Let’s make things simple and say that our loop repeats once per second.

Picture it as a bead, sliding up and down along a thin rod, tracing out the signal. So as this bead is bobbing up and down, look what happens if we spin the rod at a rate of, say, 10 revolutions per second:

We get a scribble, as you’d expect. And it is roughly centered on the origin.

Now, let’s assume we know there’s some energy in the signal at 3 Hz, and we want to measure it. What that means is that on top of whatever else is causing the signal to wobble around, we’ve added a wave that oscillates 3 times per second. It has a high point every 1/3 of a second, and corresponding low points in between, also spaced 1/3 of a second apart. You can probably see now how we might be able to detect it… let’s try spinning our signal at a matching rate of 3 revolutions per second.

Since the signal completes a rotation every 1/3 of a second, all the high points in our 3 Hz wave line up at the same part of the rotation, and this pulls the whole scribble off-center. How can we quantify that? The easiest way would be to record a bunch of points as we rotate, and average them to find their midpoint:

It makes sense that the distance of this midpoint from the origin is proportional to the strength of the signal, because as the high points in our signal get higher, they will move the scribble farther away. But what if the signal contains no energy at 3 Hz? Let’s remove the 3 Hz wave and see:

Now there is nothing to pull the scribble off center, and all of the other oscillations tend to (approximately) balance each other out.

This looks promising as a way to detect energy at a given frequency. Time to translate it into math! For a looping signal of N samples:

(Raising e to an imaginary power produces rotation around a unit circle in the complex plane, according to Euler’s formula. How? Magic, as far as I can tell. But apparently it’s true).

So this equation is a little different from what we started with. I’ve added a normalization factor of 1/N, and changed the sign of the exponent. I also rearranged the terms slightly for clarity. This form is normally called the inverse DFT, which is confusing, but apparently the difference between the DFT and IDFT is a matter of convention, and can depend on the application. So, let’s call that close enough.

Anyway, once you can “see” what’s going on in your head, a lot of the quirks of working with the DFT become much less mysterious. If you’ve had to work with DFT output before, you may have wondered:

• Why does the first element in the result (k=0) contain the DC offset? Because in that case, our samples don’t spin at all, so all we’re doing is averaging them.
• Why doesn’t the DC offset affect the frequency information? Because adding a constant value to all the samples just makes the whole scribble bigger, which doesn’t affect the midpoint.
• Why does the second half of the output array contain a mirror image of the first half? It’s just our old friend aliasing. When calculating the last element (k=N-1), we’re rotating by (N-1)/N at each step, which is almost all of the way around. This is the same as taking small steps (1/N) in the wrong direction. That’s why the result at (k=N-1) has the same magnitude as (k=1). It’s equivalent to processing a negative frequency of (k=-1).
• Why does a sine wave with amplitude 1.0 come out of the DFT as 0.5? When we spin the sine wave, we get a circle of diameter 1.0, but it’s midpoint is only half that distance away from the origin.
• Where is the other half of the energy then? It’s hiding in the negative frequency part!

Hopefully this was more helpful than confusing.

And if you’d like to get updates on my game development work, come subscribe to my RSS feed at pureenergygames.com!