Introduction

While trying to look for a set of accessibility standards for Stem Stumper I couldn’t find any (Update: checkout the BBC’s guidelines for games used on any of its sites). While sites like ablegamers.org can give really useful papers and statistics about disabled gamers, I couldn’t find a clear list of guidelines to follow for making sure as many people as possible are able to play my game. While I understand blind adherence to rules isn’t the best way to achieve accessibility, game developers lives are a lot easier when we have a standard list of items to test against. Developers interested in accessibility should also check out the IGDAs Game Accessibility Interest Group. They seem to be working on a gaming specific set of guidelines.

The closest thing I could find are the Web Content Accessibility Guidelines. They are a set of best practices that should be followed for making a website and its content accessible. Each guideline also comes with an explanation of the reasoning behind it. So for this post, I’m going to use these standards to break down the immensely popular mobile game Angry Birds made by Finnish developer Rovio. In the spirit of the Gamasutra article “Resetting Accessibility in Games” , I’m also going to show how following these standards leads to better usability for everyone.

Scope

I’m not including Angry Bird’s social network menus in this review. I’m only focusing on content you see after hitting “Play”. I’m assuming this is the vast majority of a player’s experience in the game. Also, I’ve only made it up to level 1-21 on my iPhone. I really like the game, I just have next to no free time.

General Gameplay

Relevant Guidelines:
Guideline 1.2: Provide alternatives for time-based media.
Guideline 1.4: Make it easier for users to see and hear content including separating foreground from background.
Guideline 2.1: Make all functionality available from a keyboard.
Guideline 2.2: Provide users enough time to read and use content.

The core mechanic of Angry Birds is using a slingshot to fling birds at green pigs. The pigs can be protected by ice, wood, metal, or other materials. Some birds have special abilities. Like the blue bird that can split into three and the yellow bird that that can boost its speed in mid air.

That’s it. The core of the game is that simple. No time limits or crazy combination of gestures needed. You don’t have to have the “twitch” reaction associated with most modern games. Since “twitch” is something you can only get from other video games, its lack in Angry Birds makes it usable not just for the disabled but by people who aren’t used to gaming at all.

Rovio does a good job off using multiple graphical cues to differentiate birds, pigs and obstacles from each other. By making sure the outlines and sizes of the birds and pigs differ significantly, they avoid confusing the color blind ^[1]. Even for the non-disabled, the variation helps enforce the difference between these objects.

While not playable by keyboard, the game can be played with only one finger. There’s no need for “chording”, having to press multiple keys/use multiple fingers at the same time. This helps people with mobility related disabilities like cerebral palsy or arthritis play the game. It also simplifies the game and makes it easier for everyone to pick up and play right away.

Lack of VoiceOver Support

Relevant Guidelines:
Guideline 1.1: Provide text alternatives for any non-text content so that it can be changed into other forms people need, such as large print, braille, speech, symbols or simpler language.
Guideline 2.4: Provide ways to help users navigate, find content, and determine where they are.
Guideline 3.1: Make text content readable and understandable.
Guideline 4.1: Maximize compatibility with current and future user agents, including assistive technologies.

Apple’s User Interface tech gives apps the ability to tag buttons, labels and other UI elements with accessibility information. When VoiceOver, the iPhone’s screen reader, is turned on it can then read that information out loud to blind users. This tech has opened up the world of modern mobile devices to the blind. ^[2]

However, using Apple’s off the shelf tech ties you to iPhone. Rovio, and many other mobile device developers, use their own cross-platform User Interface tech. It makes it easier to not only have a game on the iPhone but also on Andriod and other emerging mobile platforms. Rovio, however, doesn’t seem to use any of Apple’s accessibility tech since nothing in its User interface is tagged to work with VoiceOver ^[3].

While there is no true cross platform way for providing accessibility information, WCAG suggest always making the following available from code:

Name: The name of the element.
Role: What it does.
Value: The state its currently in

This information lines up nicely with what Apple’s accessibility code expects. Using the sound toggle in the pause menu as an example:

Name: “Toggle Sound”
Role: “Switch Between the sound being off and on”
Value: “Sound is currently off”

Its worth noting that in a game as visually based as Angry Birds, VoiceOver support is the least of its problems. Even if every label and button was tagged and setup properly, the act of pitching a slingshot to hurl birds at pigs behind obstacles is still fundamentally inaccessible to someone with a visual impairment. I’m learning that its better to make a game fundamentally accessible and not to rely on VoiceOver support to save the day. VoiceOver should be the icing on a cake of general accessibility.

User Interface

Relevant Guidelines:
Guideline 1.1:Provide text alternatives for any non-text content so that it can be changed into other forms people need, such as large print, braille, speech, symbols or simpler language.
Guideline 2.2: Provide users enough time to read and use content.
Guideline 2.4: Provide ways to help users navigate, find content, and determine where they are.
Guideline 3.2: Make Web pages appear and operate in predictable ways.
Guideline 3.3: Help users avoid and correct mistakes.

Being a Finnish developer developing for an American market, Rovio must have been very conscious about the need for translating any text in the game. The game’s menu doesn’t rely on text at all but on intuitive symbols.

The text that’s used for level names and numbers isn’t important for understanding the game. Even for crucial information, like the secondary abilities of birds, Rovio relies of descriptive pictures and good level design to teach the player.

The use of symbols instead of text not only makes the job of localization easier but helps players that have trouble reading but can still see well enough to play the game.

Rovio also makes sure to keep the User Interface very shallow. Every important feature is only two taps away. The pause menu only has 5 buttons on it leaving it very uncluttered. Since the levels are small and restarting them is so easy, Rovio doesn’t have to implement “undo” functionality to let players fix mistakes. This keeps the game simple which again makes it more usable for everyone.

Other Guidelines

Guideline 1.3: Create content that can be presented in different ways (for example simpler layout) without losing information or structure.
Guideline 2.3: Do not design content in a way that is known to cause seizures.

I don’t think Guideline 1.3 is relevant. If VoiceOver support was implemented Angry Birds could support the standard VoiceOVer gestures. But given some quirks with how input is processed when VoiceOver or Zoom is on, using a slingshot wouldn’t be convenient. Messing with that core mechanic would make this another game entirely.

Angry Birds doesn’t have any flashing animations which eliminates the risk of seizures. While video induced seizures are relatively rare^[4] most people who suffer from them don’t know they’re susceptible until the first attack. ^[5]

Conclusion

The WCAG seems like a really good starting point for judging accessibility in games. Not only does it contain the main guidelines above but multiple levels of success criteria to judge how well you’ve implemented a guideline. See Web Content Accessibility Guidelines for the nitty gritty details.

One thing that’s lacking on most platforms is screen reader support. Apple has been ahead of the curve in terms of accessibility and hopefully other platform owners will follow. With the advent of motion gaming being used as a way to entice non-traditional gamers, I’d be disappointed if accessibility isn’t on the minds of Sony, Microsoft and Nintendo.

Its also important to note that ease of usability in Angry Birds goes hand in hand with most of these guidelines. Not only can the guidelines be used as a way to accommodate special needs players, but as a way to judge the usability of your game for everyone.

Notes:

All images of Angry Birds were captured from my iPhone and used under fair use.
Cake images taken from Wikimedia Commons. http://commons.wikimedia.org/wiki/File:Gabe-birthday-part.jpg and http://commons.wikimedia.org/wiki/File:Cake_from_WHR%28P%29.jpg

Note 1: According to Wikipedia up 10% of American males are red-green color blind. Color blindness is rare amongst women.

Note 2: Austn Seraphin’s account of getting his iPhone

Note 3: For iPhone developer’s Apple’s guide to Accessibility Programming on iOS. The link shows up in Google searches so I’m assuming this doesn’t break iOS NDAs.

Note 4: From the Epilepsy Foundation:

“…the annual incidence of visually provoked seizures in the United States general population is estimated to be one in 91,000. Among young people 7 to 19 years of age – a population that is most susceptible to these provoked seizures and among the most frequent users of video technology – the annual incidence may be five-times as high, or one in 17,500. Only individuals who are susceptible, meaning who are photosensitive, are at risk for developing seizures provoked by light.”

Note 5: Also according to the Epilepsy Foundation.

A broken clock is right every other sprint

My name is Kwasi Mensah, and I’ve fallen out of love with agile development. While I bought into it when I first started working, over the years I’ve lost faith in our ability to make reliable time estimates, the cornerstone of this methodology. My experience, and that of many other colleagues I’ve informally asked, is that while the majority our estimates may be in the ballpark,[1] when they’re off they’re really off. There are several reason proposed for this behavior (such as distractions and vacation time). In this post however, I’m going to focus on the aspect I think effects game developers the most: the cost of innovation.

How are we supposed to make an estimate on figuring out the color palette for a game about a psychic summer camp? Or how many iterations are going to be needed to get people to dance to Lady Gaga in their living room? Each game we develop hopefully has work that’s never been done before (or is at least new to the team making it). We have to deal with tasks outside the scope of our experience which agile development doesn’t seem to deal well with. And I don’t think there’s much we can do about it. We don’t get better at estimating how long it will take us to do unknown tasks. We gain experience doing similar tasks and then draw on that to give better estimates.

To be fair, my natural sense of driving directions is much worse than this

Humans are fundamentally flawed at telling how long tasks are going to take. The planning fallacy tells us that we always overestimate how quickly we can do a task. There’s a famous example of this with psychology students severely underestimating how long it would take them to finish their theses. And this is only one of the many psychological quirks that stop us from getting estimation right[2]. Breaking tasks down, documentation and technical design documents help give us a clear road map and disseminate knowledge to the rest of the team and managers. But when you’re uncertain about a key dependency in your task any estimate you put down is useless[3]. And there are several cases when doing the work required for an estimation is a lengthy task in and of itself.

So how can we have any hope of shipping a product “on time”[4]? Deadlines are still important for talking to the press, managing PR pushes and getting paid by publishers. I think we can put better emphasis on becoming familiar with the unknown. This is where having a strong pre-production period helps. Having a period up front that says we’re willing to spend x amount of time exploring these y number of specific questions. And then having the discipline to limit production to the questions you know the answers to[5]. If you’re going to try to do things your team has no experience in understand that you’re going to need a lot of experimentation time or you’re going to have to cut down on the number of new things to add. I think the power in being “indie”[6] is that the lack of resources forces you to be conscious of this balance at all times.

Because we could all be making much more money outside of gaming

So once you’ve answered your games core questions and are in production what can you do to help the team stay its course? Assuming there are no shenanigans with your bug database, keep your bug count as low as possible before starting new features. Some places advocate charging the time back to the original task that caused them so you can better gauge the true cost of the task. I just think it’s important to keep the total number of bugs down[7]. Even if you’ve gathered the knowledge needed to have a semi-reliable estimates on getting the first pass of a task done, by definition bugs represent unknown quantities of work. There’s also the memory cost of having to remember the specifics of work done months ago instead of days ago.

While I’ve had a nebulous version of this problem in my head I can’t say we followed all of this advice when making Stem Stumper. And this post is largely based on fixing problems I’ve experienced before and not on how the practical application of these ideas went[8]. Even if you think I’m overreacting (which is quite possible) I think we have to come to terms that we work in an industry where inspiration can’t always be forced by milestones.

Notes

Note 1: Small deviations from initial estimates is a problem in and of itself since these add up over the course of the project. As do distractions. Joel on Software has an article on developing a statistical model to take these into account. I like the idea of a confidence range instead of a date. But I think making a game has so many more creative unknowns than making issue tracking software that I think pre-production and not statistical modeling is going to have the most impact.

Note 2: Wikipedia has more examples of why we’re psychologically bad at this. Check out this Wikipedia section.

Note 3: Actually the estimate is worse than useless because it gets written down and gives production false hope.

Note 4: The psychology of deadlines is fascinating. There’s plenty of research that says intrinsic motivation can have a better effect than deadlines. But there’s also the non-scientific but often observed Parkinson’s Law, “Work expands so as to fill the time available for its completion”. There are parts of the process that we could stay on forever and parts we would rather skip over.

Note 5: But everyone knows this right? So why don’t we take the time to answer these questions before going full steam ahead? The problem with this type of planning is that it doesn’t fit well with how we currently staff projects. Unless you have ton of independent questions you want answered by your team, you’re going to have a fair amount of people with nothing to do.

As a fiscally responsible company, you have to try your best to not let that happen but that last thing you want to do is tear apart the relationships and expertise you just finished building up on the last project. “Indie” developers have teams that come together and break apart as projects start and finish. Everyone has enough on their freelancing plate that they’re ok doing this. But at larger developers, which try to attract people who don’t want to live on ramen, job security is paramount. Extra Creditz has a really good episode discussing pre-productions which you should all go watch now.

By the way, Extra Creditz seems to be having contractual problems with The Escapist. All I know is the videos are a constant source of intelligent conversation about games in accessible language aimed at non-developers. You can follow them on twitter @ExtraCreditz to see what they’re up to next.

Note 6: Hopefully the quotations marks around “indie” give away how much of a loaded term I think its become. For the sake of this post, “indie” means a tiny team working out of their bedrooms eating ramen like its going out of style.

Note 7: Yes, I know 100 game crashing bugs is different than 100 bugs about a dumb feature not working properly. You’d probably need some type of stratified system where there are no crash bugs, up to 20 second tier bugs, 40 third tier bugs, etc. And you’d also have to take account bugs that might be fixed by future work. The numbers are completely arbitrary but your team should consciously decide on an acceptable balance between making progress and quantifying the current amount of outstanding work.

Note 8: I guess I’m suggesting time-bounded period for pre-production where you deal with issues in a preassigned priority. This sounds a lot like a sprint, except it’d be over a longer period and you have the disciplne to drop issues that couldn’t be figured out in time. I think agile can still be useful during production when you have a better handle on the amount of work in front of you.

Broken Clock image from: http://2.bp.blogspot.com/_0S4RTTd6-v0/Sc2ED9W_IYI/AAAAAAAAA0Q/XA5eZgFTjSE/s320/broken+clock.jpg
Car estimation image from: http://xkcd.com/612/
Balance image from: http://www.dailyhaha.com/_pics/good_balance.jpg

Guerilla PR at PAX East

Everyone at Ananse Productions is heads down working on Stem Stumper and we’re finally starting to see the light. But it would be nice if people knew about the game when we release in Mid-April. So I armed myself with business cards, packed up our stickers and buttons and brought Ananse Productions to PAX East.

Below are some thoughts on how our weekend went and what we’ve learned about trying to show our game to the 70,000+ people in attendance.

Be Confident

I’m normally not a very pushy person. But for Stem Stumper, Ananse Productions’ first title, I honestly believe we’re doing something that sets us apart from other mobile games. Stem Stumper is a blind accessible puzzle game designed for both the non-sighted and sighted.

Stem Stumper Screen Shot

This makes me confident that I’m not wasting your time when I talk to you about Stem Stumper. And remember that everyone at PAX East is excited about gaming in some way shape or form. Even if Stem Stumper isn’t their cup of tea no one’s going scream bloody murder if you politely introduce yourself and try to start a conversation about your game.

Demos are different from Betas!

We’ve been preparing to get a build of Stem Stumper out to remote testers. I’ve been worrying about things like tutorials, getting the levels to unlock in order and what to show on that final screen when you beat the game.

The problem is none of that matters on an expo floor. While its important to the overall timeline of getting Stem Stumper out the door, on the expo floor you have a very very short amount of time to show your game. Count out 10 seconds to yourself right now. It’s a surprisingly large amount of time when people have 100 booths with long lines that they want to see. I needed to be obsessive about shortening our pitch, cutting every extraneous word, every advanced feature, every small fact that we wanted people to know about Stem Stumper. I practiced the pitch until it rolled off my tongue. And then, only when someone is deeply interested, did I introduce them to the advanced features.

The clock's ticking!

If you can’t shorten your sell that means your game’s learning curve is too steep. We showed Stem Stumper at Made in MA, the PAX East pre-party that highlights games made in the New England Area. We still had some big usability issues that got in the way of people picking up Stem Stumper and playing it right away. While seeing everyone fumble over the same issue was disheartening, we had a clear point of attack for what needed to be fixed and had it addressed the next day. When we showed the game to people on the first day of PAX East Stem Stumper clicked with them right away.

Steep learning curves make it impossible to demo

Always Be Demoing

The great thing about working on a mobile game is that you can walk around the expo floor with an iPhone and headphones in hand and have people play it then and there. I walked up to everyone that had a yellow media pass and didn’t seem like they were speeding to another booth.

Glengarry's mantra modified just a smidge

While I did talk to ~100 press members over this entire weekend, I know I’ll be lucky if 10 of them followup. But even if I don’t hear from them again, we just got 100 people who’ve never seen Stem Stumper before giving us valuable information about our initial user experience. While I did use our quick spiel instead of our actual tutorials, we got a huge amount of people to help figure out what the most pressing issues with our controls are. So now we can make Stem Stumper even better for all of our players and for the 10 journalists that do followup.

Go to Panels!

The expo floor is great for meeting sheer amounts of people. But panels were our chance to meet people who are directly aligned with accessibility and diversity and gaming. Everyone in the room already has something in common which makes it easier to introduce myself.

Don’t be afraid to ask questions! At a conference like PAX East, standing up in front of a packed room and asking an intelligent question is an easy way to set yourself apart. Even when I wasn’t able to ask my question during the session, I would talk to the panelist afterwards. They’re panelists because they’re passionate about the session idea and in my humble experience always want to chat more about it.

Don't be afraid to raise your hand!

The Press is Friendly but Busy

Last, but definitely not least, remember that the press are people too. At an event like this you’re going to start running into people who’s names sound eerily familiar. I made sure to tell myself the Stem Stumper and Ananse Productions aren’t too small for their attention. They’re at PAX East because they’re excited about games and they want to make sure their readers find out about the good ones no matter who makes them.

However, I kept in mind that a lot journalists already worked out who they’re talking to and what panels they’re covering. Especially ones from from the bigger organizations like Kotaku, Gamespot, and Joystiq. Respect the fact that they’re doing their job just like you are. But I was always ready to exchange cards so I could follow up with our press kit (you’re never too small for a press kit!).

An Overworked Journalist

I wasn’t picky about who I talked to. If they have a press pass, they belong to an organization that talks to gamers that we’re never going to meet face to face. There’s no way to tell on the floor whether they reach 10 or 10,000 people a month and frankly it doesn’t matter at this point for Ananse Productions. Establishing relationships, getting people excited about Stem Stumper and honing my ability to quickly describe what stands out about Stem Stumper are things I can work on no matter who I’m talking to.

Monday was email madness day. I emailed everyone that gave me a card with our press kit and more information about Stem Stumper. For people who I remembered being especially excited about Stem Stumper I made sure to include a couple of lines about our meeting. Remember, this person probably has a million stories from PAX East that they’re trying to keep in their head at the same time. A friendly email reminder makes sure that we’re still visible on their todo list.

Conclusion

It’s too soon to see how much press Stem Stumper is going to get. But in my humble opinion all of PR comes down to establishing a relationship with your player base. Game journalists are an important part of that. Ananse has some other leads that we’re following for media exposure but this weekend was an important step in getting everyone to know about Stem Stumper.

Cards from everyone we talked to this weekend

Notes

All external images used under fair use.

Clock ticking image from: http://campyellowfever.blogspot.com/

Learning curve image from: http://dementiagaming.wordpress.com/2010/08/23/learning-curve-of-eve-online/

Gleegarry Ross image from: http://www.michaelfruchter.com/blog/2009/02/abc-always-be-closing/

Image of hands raised from: http://vickicaruana.blogspot.com/2011/01/are-you-afraid-to-raise-your-hand.html

Overworked Journalist Image from: http://www.aspectuspr.com/blog/2011/02/churning-it-out-a-question-of-quality-content/

Is engineering your specialty?

Because you're not a real programmer unless you read XKCD

I’ve always been a generalist, knowing just enough about graphics and audio to be dangerous. The elegance of managing complex relationships in real time, not just getting games to run blazingly fast, is what drew me into game programming[2]. However, in the game industry I’ve found a lot of programmers that have the “hack it together until works” philosophy which left me really frustrated.

I’ve always believed that good structured coding leads to better tools that empower content creators to iterate more and lets the team focus on making or changing features without running into a cavalcade of bugs. So with Ananse Productions I’m testing that theory. Plus it keeps me interested as a programmer which is important if I’m going to live and breathe this company.

Can you keep your scope small?

Mmm, delicious scope creep

Stem Stumper was made in 6 months. And we only had our Artist, Audio Guru and Level Designer for 3 of those months. I think this was only possible because we kept our scope very tiny because our engine could only do so much. Constraints are a great source of inspiration[3].

Our main priority was making a great game. But just underneath that was making sure we were battle tested and I had a clear understanding of what worked and what needed to be fixed to make our pipeline smoother. I committed to limiting the scope of the game to what the we could handle and being really hesitant to add new features. This is a paradigm shift from the industry standard of forcing in features because your scope demands it.

How much risk are you willing to take on to grow expertise?

Don't worry, she'll land on her feet

Like most “entrepreneurs” I have age on my side. If Ananse Productions doesn’t work out (knock on wood) I have a lifetime of earnings in front of me to make up for it. I have no one dependent on me and no one’s retirement funds to watch out for. On top of that if Ananse isn’t around this time next year (again knock on wood), I get to walk away with the experience of writing an engine, starting a company, managing a team, and learning how PR works. For me, that’s a win. My risk tolerance is about as high as its ever going to be.

Even if you’re not a kamikaze start up like us, there’s still an enticing risk/reward dynamic to doing things in house. If your artist needs to know how animation works in the game, someone on the team can answer her right away. If our sound person needs to know what audio format to use she can get an answer right away. Yes, proper documentation helps but having someone on the team who’s an expert in every system can be immensely helpful. You’ll not only have someone who understands how a physics system works but who has also actually implemented one. Engineers know there’s a huge difference between the two. You’ve just grown your human capital and cultivating talent can outweigh the time savings of a engine in the long run.

Are you OK with the external dependencies?

An interesting take on the trust fall

One thing that was very important in Stem Stumper was that we were able to talk to the screen reader tech of the platform we’re running on. It’s very easy for us to make sure our engine can do that because well… we built it from the very start to include that.

When we wanted to the game to work on Android (something we’re currently working on and will post about soon) we reimplemented our platform specific layer for Android. It sucked and it was painful but next time around we should have something that just works (although nothing ever does on Android). And learning the ins and outs of the Android platform set us up to do contract work for others. We weren’t reliant on working on an engine that already did this for us. And we didn’t have to touch the innards of an unfamiliar code base.

What’s your long term plan?

Because we all want to make it rain

I’ll admit the questions above say more about my personality than my business sense. But one serious business reason for writing your engine is company stability. When you look at the big independent studios like Valve and Epic they each have ways of making money outside of game development. Valve has Steam, which allows them to take as long as they want with Half Life 3. Epic has Unreal which everyone and their grandma uses for console development (yes its starting to branch out onto other platforms). Even The Behemoth uses merchandising to help pay the bills. While riding an initial blockbuster can help setup a studio, for long term growth and stability I think you need something that isn’t as hit or miss as a game. And and making the code a potential product is hopefully a smart move for us. We’re obviously a while a way from acting on this and it slows us out of the gate, but we’re in it for the marathon not the sprint.

Summary

You’re not dooming yourself when you decide to roll up your sleeves and take care of things yourselves. But just like any other business decision you have to carefully evaluate the risks and see if its in line with what you’re trying to accomplish. We use WordPress for our site because we’re not about web development. And we’re not going to implement our own custom source control. I hope this helps others afflicted with NIH lead long and successful lives.

Notes

Note 1: Joel Spolsky of Joel on Software and Stack Overflow fame has already written a post in defense of Not Invented Here syndrome which you can find here. I wanted to focus on how it applied to Ananse Productions and game development.

Note 2: You should check out Insomniac’s presentation on their component based architectures. And Radical Entertainment’s. And check out the Game Programming Gems books. I fully read the sections on Architecture and the rest of the articles are gravy.

Note 3: According to legend, Chuck Jones (of Bugs Bunny fame) stuck to some pretty strict rules when making Bugs Bunny and Wile E. Coyote cartoons. While there’s debate on the Wile E. Coyote rules, constraints get rid of indecision which frees us to focus in on what’s important.

“Real Programmers” comic from http://xkcd.com/378/
“Scope Creep” from http://test.ical.ly/tags/scope-creep/
“Risk Assessment Cat” from http://cheezburger.com/TemplateView.aspx?ciid=4401869
Trapeze artist image from http://usrportage.de/archives/911-Not-having-globals-state-doesnt-mean-youre-doomed.html
“Scrooge McDuck” image from http://listsoplenty.com/blog/?p=13032

Introduction

(This post assumes you’re familiar with C, Objective-C and Cocoa.)

Just like TV shows and films a video game’s visual representation can be broken down into frames, single snapshots of the state of the game world at any given time. By showing these frames many times per second we give players the illusion of continuous motion.

The loop below is a high level description of what the game has to do each frame to draw the current snapshot.

//Basic game loop
while(1)
{
    ProcessInput();
    DoLogic();
    Render();
}

Even though the iOS API doesn’t make this clear there’s a similar loop going on, we just have less control over it.

All iOS apps start with a call to UIApplicationMain which kicks off a NSRunLoop which for our purposes is a fancy event queue. Whenever something interesting but non-critical happens (input for example) it gets put on the back of this queue and processed in the order it’s received.

So how do we emulate our loop above using this? We know with UIApplicationDelegate we don’t need ProcessInput anymore. We can care of input events as soon as they come in with touchesBegan, touchesEnd and touchesCanceled. In general, input is going to require some type platform specific callbacks so as long as we leave breathing room in our game loop for processing these we should be fine.*1

Naive Try at a Game Loop

We know we want to calculate a new frame fairly frequently. Let’s pull the number 60 times per second out of thin air for now.

//Bad iPhone game loop
-(void) doFrame:(id)data
{
    DoLogic();
    Render();
}

- (BOOL)application:(UIApplication *)application
    didFinishLaunchingWithOptions:(NSDictionary*)launchOptions
{
    doFrame();
    [NSTimer scheduledTimerWithTimeInterval:(1.0/60.0) target:self
        selector:@selector(doFrame) userInfo:nil repeats:YES];
}

The above won’t cut it. The resolution of an NSTimer is 50-100 milliseconds. *2 1/60th of a second = 16 milliseconds. Our resolution, at best, is thrice the frequency at which we want our function to be called. There are also a host of other problems with using repeating NSTimer‘s *2. We could try messing with the NSObject performSelector family of functions but you’re going into run into other problems, especially with our friend v-sync.

The Basics of Vertical Sync(v-sync)

Monitors take a real (but very small) amount of time to draw the images on our screens. Furthermore, there’s a gap between when we finish drawing one image and when we’re ready to draw another.

Draw–Rest–Draw–Rest–Draw

The vertical refresh rate of a monitor is how often the monitor draws a frame per second. This number varies in different monitors in different regions. Since we’re only concerned about iOS in this post, we’re going to focus on 60 frames per second = 16 milliseconds (remember the number I pulled out of thin air) because that’s the vertical refresh rate of iOS devices.*3

Vertical synchronization (v-sync) refers to waiting for the screen to finish being draw i.e. waiting for the start of a “Rest” state.

With this information, let’s take a closer look at our Render function:

void Render()
{
    InitDrawing();*6
...
    FinishDrawing();*7
}

InitDrawing() waits for v-sync, if it hasn’t happened yet, before letting us draw any objects.*4,*5. FinishDrawing() signals that we don’t have any more objects to draw this frame. In between them we put our special sauce that tightens the graphics on level 3.

So what happens if we don’t take v-sync into account when we draw? We’ll be stuck inside of InitDrawing() waiting for v-sync when we could be doing other useful things like dealing with touches. In fact, it was debugging the lagginess of the VoiceOver (Apple’s screen reader tech) in my game that made me realize that I was stuck inside of InitDrawing().

Note, even if we were in a perfect environment and able to call doFrame every 16 msecs, if it’s not synchronized with vsync we’ll still waste a lot of time waiting for it.

We’re Halfway There

So how do we make sure doFrame gets called 60 times a second and we won’t have to wait for v-sync? Fortunately, CADisplayLink comes to our rescue.*8 This functions like the NSTimer above but it has a good resolution and is timed with v-sync (sort of, we’ll talk about it later).

//Better but not quite right yet.
-(void) doFrame:(id)data
{
    DoLogic();
    Render();
}

- (BOOL)application:(UIApplication *)application
    didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    mFrameLink = [CADisplayLink displayLinkWithTarget:self
        selector:@selector(doFrame:)];
    mFrameLink.frameInterval = 1;
    [mFrameLink addToRunLoop:[NSRunLoop mainRunLoop]
        forMode:NSDefaultRunLoopMode];
}

If we can assure that every call to doFrame will always take less than 16 msecs than the above will work and you go home and get back to coding. And dear God, blog to the rest of us on how you’re doing that. However, if any of our frames take even a little bit longer you’re gonna be in for a shocking surprise. Let’s draw some pretty pictures.

Let’s say we have a frame that takes 21 msecs (needed a little extra time to decode your awesome background music).

Timeline: 0---16---32---48
Frame 1:  0----21

Since we took longer than our alloted 16msecs, CADisplayLink won’t wait untl the 32msec mark to call doFrame. It will call it immediately.*9 It’s easy think “that’s ok. I normally only take 10 msecs so this will be straightened out by the next frame”. But remember our section on v-sync. By calling doFrame right away we’re going to be stuck in InitDrawing() until the 32msec mark. So our timeline after the second frame will look like*13:

Timeline: 0---16---32---48
Frame 1:  0-----21
Frame 2:        21----42

Our second frame has to wait until the 32 msec mark before it can start rendering and then it will take 10 msec to draw. And remember, you passed the 32 msec mark while drawing this second frame which means doFrame is going to be called immediately again.

Timeline: 0---16---32---48---64
Frame 1:  0-----21
Frame 2:        21----42
Frame 3:              42---58

The timelines above only take into account calling doFrame. There’s no breathing room for input processing which will make your app unresponsive and laggy until the storm settles. We know this will settle itself out because the amount of time spent waiting for vsync decreases with each frame, but the fact remains that one frame of suckiness can cause an avalanche of woe.

It’s hard to track down if you don’t know what you’re looking for. Instruments will still tell you that you’re rendering at 60 frames per second. But you’ll see a lot of time spent in some combination of semwait_signal and usleep because you’re stuck waiting for v-sync.

We could’ve put a quick end to this by enforcing that if we miss a v-sync we just suck it up and wait for the next one i.e. don’t get run over chasing the bus you missed.

Timeline: 0---16---32---48
Frame 1:  0-----21
Frame 2:           32--42

This gives us the breathing room we need to still process input and makes sure that one errant frame doesn’t screw us over in the long run. Fortunately, this is eerily similar to problems physics engines have run into and Glenn Fiedler has an excellent article that’s the basis of how we’re going to deal with this. *10

Conclusion

We’re going to remember how long the past frame took and bank it.*11 Then the next time we call doFrame we subtract 16 msecs from the bank. If the bank has no more time left in it then we know we won’t have to wait for v-sync. Otherwise, we do nothing because we know we’re being triggered for a v-sync that we missed.

Also note, the call to fmod if we take longer than a frame. If we miss multiple v-sync (using a breakpoint for example) there’s only one call to doFrame *9. If we didn’t have this, we can get a huge amount of time our bank and we won’t render until its all subtracted out, one frame at a time.

//Psuedo-code for what I'm currently using.
-(void) doFrame:(id)data
{
    static double bank = 0;
    double frameTime = mFrameLink.duration * mFrameLink.frameInterval;
    bank -= frameTime;
    if( bank > 0 )
    {
        return;
    }
    bank = 0;
    PlatformHiResTimer timer;*12
    timer.Start();
    DoLogic();
    Render();
    double elapsed = timer.ElapsedTime()*PlatformHiResTimer::sNanoSecToSec;
    bank = elapsed;
    if( elapsed > frameTime )
    {
        bank = frameTime + fmod( elapsed, frameTime );
    }
}

This should take account sporadic frames being longer than usual. This isn’t meant to protect against a stream of frames that consistently take longer that 16 msecs. If that’s the case you need to lower your frame rate.

It took a fair amount of experimenting to figure this out. While this discussion focuses on iOS its meant to serve as a baseline for setting up game loops for any platform.

Notes

Note 1: If you’re really set on delaying input handling you can use the -touches* functions to queue them until you call ProcessInput. Be careful of lag, especially if you’re game is supposed to work with VoiceOver( Apple’s screen reader tech). The VoiceOver cursor relies on touch events being processed off of the run loop even if the events aren’t passed down to the app.

Note 2: NSTimer accuracy issues discussed here.

Note 3: Our final solution actually isn’t dependent on knowing vertical refresh rate. It’s just easier to discuss with a firm number in mind.

Note 4: EAGlContext presentRenderBuffer returns before v-sync happens. It’s the next render command that will be forced to wait for v-sync if it’s called too early. A lot of the google results for glClear being slow on the iPhone are probably related to not waiting for v-sync.

Note 5: A lot of pc/mac games have options to not wait for v-sync. This lets you render faster but can cause visual artifacts. On iOS we’re stuck with v-sync. For a full discussion see this wiki article.

Note 6:

void InitDrawing()
{
    [EAGLContext setCurrentContext: sGLContext];
    glColor4f( 0, 0, 0, 1 );
    glClear(GL_COLOR_BUFFER_BIT);

    //You can also draw a screen filling black quad. This hasn't been
    //a bottleneck for me.
}

Note 7:

void FinishDrawing()
{
    glBindRenderbufferOES(GL_RENDERBUFFER_OES, sColorRenderBuffer);
    [sGLContext presentRenderbuffer:GL_RENDERBUFFER_OES];
}

Note: it seems like iOS always uses double buffering.

Note 8: CADisplayLink is only available in iOS 3.1 and later. Reference doc.

Note 9: CADisplayLink‘s behavior is actually more complicated. If your frame misses multiple v-sync events the selector seems to only be called once for all of the missed v-sync events. I also believe that the CADisplayLink object’s duration property is for the most recent v-sync event. Furthermore, the CADisplayLink object’s duration property can vary significantly from the time between calls to the selector even when we don’t miss a frame. Performing the selector seems to be put off if the NSRunLoop is busy when v-sync happens. The moral of the story: While CADisplayLink gives us a much better resolution that a NSTimer, calls to doFrame aren’t guaranteed to line up with v-sync.

Note 10: Glenn Fielder’s article “Fix your Timestep!” can be found here.

Note 11: Glenn would use the term accumulator but I think “bank” is more intuitive.

Note 12: PlatformHiResTimer is a helper class that’s a wrapper around platform specific functions that give us nanosecond timing resolution. For the iPhone/Mac I use mach_absolute_time() and mach_timebase_info which are mentioned in this Mac OSX Q&A

Note 13: For the timelines we’re assuming DoLogic() doesn’t take a noticeable amount of time. It makes drawing the timelines easier. Note that without this assumption, you could get lucky and have the work in DoLogic() push the call to Render() past v-sync. But you’re playing with fire as it’s going to be very hard to make sure that happens on a consistent basis.

We could also call Render() before DoLogic() to make sure we start rendering as soon as v-sync is done. However, we’ll be rendering a frame behind the game. This can cause wonkiness, especially if we’ve processed input that’s changed the state of the game but we haven’t called DoLogic to validate/fix it.

It makes more sense to me to always call DoLogic() first. When doFrame() takes less than 16 msecs it doesn’t make a difference. When doFrame() takes longer than 16 msecs I think the final version of doFrame() handles it better than the trickeration needed to call Render() first. I’ve seen a fair amount of game programming books advocate calling Render() first so I might be missing out on something.

BioWare Co-founders, each has a medical degree. We'll leave them out too

Last month I had the privilege of presenting at the Games for Health Conference.^[1] It was a big gathering of people that want to use games to positively change the world. However, most of attendees were from the the education or healthcare space. While everyone knew they were on to something powerful, there were also a lot of misconceptions on how gaming works as a motivator.

The “G” Word

One way to apply games to real life

I heard the word “gamification” many times during the three day conference. While the term has been around for a while, Jesse Schell popularized this concept in his DICE 2010 speech.^[2] A lot of attendees used it to describe adding points systems, social web interactions, and other extrinsic rewards to get people to do healthy and educational tasks. In their minds, the “game” was all about getting the reward. However, extrinsic rewards in and of themselves aren’t good replacements for intrinsic motivations.^[3] Those presenters didn’t understand what it means to make a “game”. True conventional^[4] video game design is about finding the enjoyable parts of a process and exemplifying them while skipping over and minimizing the parts that aren’t fun.

The Points Don’t Matter

Whose Line Is It Anyway

As Drew Carey would say at the start of “Whose Line is It Anyway?” the points don’t matter. A lot of presenters focused on the extrinsic rewards we give players (points, achievements, badges, etc.) because its the part of making games that’s most visible. What they don’t realize is that good games use these rewards as an icing on the cake of an inherently compelling experience. Kendra Markle, founder of AlterActions.org and another presenter at the conference, used a powerful metaphor of the rider and the elephant.^[5] The rider is our rational mind which reacts to things like money and tweets. The elephant is our emotional side where it takes a lot of effort to do something we don’t want to do. When there’s a disagreement about which way to go the elephant almost always wins out.

Good game designers understand that while we need to convince the rider to pick up and start playing the game, we should spend most of our time getting the elephant to not want to put it down. That’s why we spend so much time on polish, usability, and getting players into “flow”^[6]. Being fixated on the things which most people don’t consciously notice is what separates designers from the rest of the world.

Let’s take Stack Overflow for example. It’s a popular question and answer forum used by programmers. Stack Overflow uses a system of upvoting the best answers and keeping track of how good members are at answering questions. On its surface it sounds the same as the forums and mailing lists coders normally go to for answers but with a score added. But just like “Who’s Line Is It Anyway” the points in and of them self don’t matter. It’s the fact that they act like filters, making sure the best answer is easily found at the top and that trusted posters are easily identifiable, that has made it the go to site for coding questions. And this in turn feeds into founder’s bigger intentions: to get programmers writing more.^[7]

Stack Overflow: More useful than English classes?

Bad Teacher

The problem with trying to target intrinsic motivations is that not everything can be “gamified”. There are things in life that are necessary and are fundamentally not fun. “Games” implies being able to voluntarily start and stop at any time. Let’s take a hypothetical doctor that doesn’t like to fill out forms. We can use game design skills to try and streamline the process and make a better connection between filling out forms and the health of patients. But at the end of the day doctors have to do it or patients’ health will be compromised. When you also take into account that what’s considered fun changes from person to person, leaning on “games” as a primary teaching method becomes really tricky.

What happens when you force people to play games

Good Willed Propaganda

The take away is that games are a great way to have people voluntarily reinforce a set of behaviors. Truly inspiring change should be about getting players to exercise the right behaviors on their own instead of relying on a carrot which won’t always lead them. We should be focusing on embedding our intent in what makes the game fun, even if the player has to go outside the game to get the specifics of the message. Good luck elephant handling!

Control the elephants, control the world!

Notes:

Note 1: My company is Ananse Productions. We talked about Game Design and Accessibility using our iPhone game Stem Stumper as a use case (you should pick it up!). You can find our slides here and you can find the other Games For health Conference presentations here

Note 2: Watch Jesse Schell’s DICE 2010 speech here

Note 3: There’s a lot work in the social psychology space about how motivation works. For example, there’s the
overjustification effect where people will start aiming for the rewards instead of for the behavior you were trying to instill in them.

Note 4: I made sure to use “conventional” because there is interactive media that aims to do something different. Check out Chris DeLeon’s post on not games

Note 5: You can find Kendra Markle’s full presentation here

Note 6: Flow is an interesting concept in and of itself. It’s about keeping a player challenged enough to be engaged but not so challenged that they give up. You can read more about it here.

Note 7: Jeff Atwood, one of the co-founders, admits that Stack Overflow wasn’t just meant to be a good technical resource. It was also meant to sneakily teach programmers how to write. Jeff wanted to help programmers become better writers and he knew the only way to do that was to well… write. Blogging would seem too insurmountable to people who aren’t used to it. So they honed in on two things every programmer wants: quick, reliable answers and the ability to show other programmers how smart they are.

Picture of Ray Muzyka and Greg Zeschuck from: http://venturebeat.com/2008/03/29/qa-with-bioware-founders-on-mass-effect-and-life-at-ea/
Picture of “Reality: Worst Game ever.” from: http://foggslog.wordpress.com/2011/04/27/gamification/
Picture of Whose Line Is It Anyway from: http://heyitscaitlinx3.tumblr.com/post/6331187274
Picture of Stack Overflow logo from: http://stackoverflow.com/
Picture of bunny from: http://www.ps3trophies.org/forum/dead-space-2/93628-hardcore-time-workaround.html
Picture of baby elephant from: http://www.inspirationline.com/rss/10APR2006.htm