#AltDevBlogADay

C/C++ Low Level Curriculum Part 12: Multiple Inheritance

Alex Darby — Wed, 22 May 2013 11:16:20 +0000

Hello, and welcome to the 12th part of the C / C++ low level curriculum. Really soon after part 11! (No, of course part 11 didn’t get too big and need to be split. Why would you ask?)

Last time we looked at the basics of how inheritance was implemented at the low level; and this time we’re going to examine how using multiple inheritance affects this picture (note: we’re leaving the keyword virtual til next time).

Before We Begin

I will assume that you have already read the previous posts in the series, but I will also put in-line links to any important terms or concepts that you might need to know about to make sense of what you’re reading. I’m helpful like that.

Another big assumption I’m going to make is that you’re already very familiar with the language features of C++ and comfortable using the language features we’re discussing. If I need to demonstrate anything out of the ordinary I’ll explain it – or at least link to an explanation.

In this series I discuss what happens with vanilla unoptimised win32 debug code generated by the VS 2010 compiler – whilst the specifics will differ on other platforms (and probably with other compilers) the general sweep of the code should be basically the same – because it’s assembly that has been generated by a C++ compiler – and so following the same examples given here with a source / disassembly debugger on your platform of choice should provide you with the same insights we get here.

With this in mind, in case you missed them, here are the backlinks to the previous posts in the series:

I won’t lie – it’s not light reading :)

I’ve just read post 11 and it all seemed pretty obvious…

Good! It should do!

In my experience, most good solutions to problems appear obvious when explained well ;)

Now we’ve got an understanding of how single inheritance behaves, let’s examine how multiple inheritance affects this picture…

Multiple Inheritance

Sample time!

Like last time I have lovingly zipped up a hand-crafted VS2010 solution / project / source code combo to go with this sample which contains the following code:

class CTestBaseOne
{
public:
    int _iA;
    int _iB;
 
    CTestBaseOne( int iA, int iB )
    : _iA( iA )
    , _iB( iB )
    {}
 
    int SumBase( void )
    {
        return _iA + _iB;
    }
};
 
class CTestBaseTwo
{
public:
    int _iC;
    int _iD;
 
    CTestBaseTwo( int iC, int iD )
    : _iC( iC )
    , _iD( iD )
    {}
 
    int SumBaseTwo( void )
    {
        return _iC + _iD;
    }
};
 
class CTestDerived
: public CTestBaseOne
, public CTestBaseTwo
{
public:
    int _iE;
    int _iF;
 
    CTestDerived( int iA, int iB, int iC, int iD )
    : CTestBaseOne ( iA, iB )
    , CTestBaseTwo ( iC, iD )
    , _iE ( iB )
    , _iF ( iD )
    {}
 
    int SumDerived( void )
    {
        return return SumBase() + SumBaseTwo() +_iE + _iF;
    }
};
 
int main( int argc, char* argv[] )
{
    CTestBaseOne    cTestBaseOne( argc, argc + 1 );
    CTestBaseTwo    cTestBaseTwo( argc, argc + 1 );
    CTestDerived    cTestDerived( argc, argc + 1, argc + 2, argc + 3 );
 
    return      cTestBaseOne.SumBase()  + cTestBaseTwo.SumBaseTwo() 
            +   cTestDerived.SumBase()  + cTestDerived.SumBaseTwo() + cTestDerived.SumDerived();
}

Once you’ve unzipped it, go ahead and build it.

Don’t forget to pay attention to the build output – it shows the memory layout which we’re going to talk about next.

Memory Layout

As you can see, we now have two base classes, and one class that derives from both of them.

When you build the project, you should see that the memory layout of these classes looks like this:

1>  class CTestBaseOne	size(8):
1>  	+---
1>   0	| _iA
1>   4	| _iB
1>  	+---
1>  
1>  class CTestBaseTwo	size(8):
1>  	+---
1>   0	| _iC
1>   4	| _iD
1>  	+---
1>   
1>  class CTestDerived	size(24):
1>  	+---
1>  	| +--- (base class CTestBaseOne)
1>   0	| | _iA
1>   4	| | _iB
1>  	| +---
1>  	| +--- (base class CTestBaseTwo)
1>   8	| | _iC
1>  12	| | _iD
1>  	| +---
1>  16	| _iE
1>  20	| _iF
1>  	+---

This is all as we might expect – given what we found out last time.

In particular note that:

the memory layout of both base classes is embedded into CTestDerived
CTestBaseOne and CTestBaseTwo appear in the memory layout in the same order they are declared in the base-specifier-list of CTestDerived.

n.b. the base-specifier-list is the part of the declaration of a class where the base classes are specified.

In the simple case of single inheritance we considered in the last post, we saw that the functions of a base class B could be called on instances of a derived class D because:

the memory layout of D contains a literal instance of B at an offset of 0 bytes within itself and…
…this means that the member data of an instance of B is at the same offset relative to the memory layout of an instance of D…
…and so the hard coded offsets used to access these members within functions belonging to B are also valid for instances of D

Looking at the memory layout for this multiply inherited class we can see that:

this relationship still holds for CTestBaseOne and CTestDerived - CTestBaseOne is an an offset of 0 bytes within the memory layout of CTestDerived
however, this same relationship is not true of CTestBaseTwo and CTestDerived

Given this situation, how do functions of CTestBaseTwo work with instances of CTestDerived?

As usual the best thing to do do is take a look…

Calling a function of CTestBaseTwo on CTestDerived

Put a breakpoint on the return statement from main(), run the code, and when it stops right click then choose ‘Go To Disassembly’.

Rather than paste the disassembly as text this time, I’ve inserted a screenshot of my debugger window – this allows more formatting and highlighting options.

N.B. in this screenshot I have “Show symbol names” checked under viewing options. Whilst this typically makes it easier to relate disassembly to C or C++ code, it does hide detail (i.e. the addresses of the symbols) .

Let’s pick this apart then, starting at the current line indicator where the breakpoint is and working down:

we can see that (following the x86 thiscall convention) before each function is called, the address of the corresponding object is stored into ecx using lea.
First it loads the address of cTestDerived into ecx and then calls CTestDerived::SumDerived()…
then it…
Oh, wait… it’s loading the address [ebp-20h] into ecx…
that symbol isn’t resolving in the disassembly window, so what witchcraft is this!?

I have helpfully highlighted the most salient areas of the screenshot with red boxes :)

If you look at the function calls made in the disassembly, and compare them to the calls in the C++ code, you will see that all of the high level function calls have an analogue at the assembly level except for cDerived.SumBaseTwo().

CTestBaseTwo::SumBaseTwo is getting called, but with [ebp-20h] as the this pointer in ecx, not [cTestDerived] (n.b. see the top red box in the screenshot).

So, the question is: how does the address [ebp-20h] related to the address of cTestDerived?

This would be a good time to reiterate that the watch window is your friend. We can use the watch window to Sherlock Holmes our way to an answer.

If you look in the watch window below the disassembly view (shown again below by itself for those of you who are vertical resolution challenged) you can see that I have used watch window expression evaluation to find out some information about these values:

This shows us that:

the address of cTestDerived is 0x0048fa84…
… and the address of cTestDerived cast to a pointer to CTestBaseOne has the same address, …
…but when the address of cTestDerived is cast to CTestBaseTwo we get 0x0048fa8c…
…which is the same value as [ebp-20h]…
…or an 8 byte offset from the address of cTestDerived…
…which is the offset of CTestBaseTwo within CTestDerived

Should this be surprising?

Here’s the memory layout of CTestDerived again:

1>  class CTestDerived	size(24):
1>  	+---
1>  	| +--- (base class CTestBaseOne)
1>   0	| | _iA
1>   4	| | _iB
1>  	| +---
1>  	| +--- (base class CTestBaseTwo)
1>   8	| | _iC
1>  12	| | _iD
1>  	| +---
1>  16	| _iE
1>  20	| _iF
1>  	+---

Since we know that:

(within non-static member functions) member variables are accessed via constant offsets from their this pointer

the memory for CTestBaseTwo starts at an offset of 8 bytes from the start of the memory layout of an instance of CTestDerived

it follows that CTestBaseTwo::SumBaseTwo() wouldn’t work if the compiler passed the address of an instance of CTestDerived because the constant offsets used to access the members of CTestBaseTwo would be off by 8 bytes.

Consequently, any time a CTestBaseTwo member function is called on an instance of CTestDerived the compiler must ensure that a compatible this pointer is generated to pass to the function - i.e. pointing at the start address of CTestBaseTwo within the instance of CTestDerived.

Frighteningly obvious once you know isn’t it?

I honestly don’t think it should be surprising though – given the way that we know data within user defined types is accessed at the assembly level (see part 10), it pretty much had to work like this.

…one more little thing

In the example above, cTestDerived is a Stack variable – so the compiler knows exactly where it is located in the current Stack Frame.

This means that the compiler can calculate the address of the instance of CTestBaseTwo within cTestDerived at compile time, and can therefore access it at no extra cost compared to any other Stack variable.

We should probably check whether this is this any different when we’re dealing with a pointer to a CTestDerived at an arbitrary point in memory, just to be thorough.

Luckily I have already thought of this :)

If you place a breakpoint on the return statement of CTestDerived::SumDerived you can check the disassembly yourself, but here are the relevant lines from my disassembly window:

   52:     int SumDerived( void )
    53:     {
001010A0  push        esi  
001010A1  mov         esi,ecx  
001010A3  push        edi  
    54:         return SumBase() + SumBaseTwo() +_iE + _iF;
001010A4  lea         ecx,[esi+8]  
001010A7  call        CTestBaseTwo::SumBaseTwo (101050h)

As you should be able to see by now, the code in this function is adding a constant offset of 8 bytes onto the this pointer it is passed to generate the this pointer it is passing to CTestBaseTwo::SumBaseTwo

If you’re having trouble seeing it, remember that the ‘thiscall‘ win32 member function calling convention uses ecx to pass the this pointer.

Most significantly, looking back to the last post, we can see that this is essentially the same way that member variables of user defined types are accessed when we had a pointer to an instance of a user defined type – in fact, at the level of the assembly code, there is really no difference between a member variable and a base class; this distinction is really only meaningful at the level of the C++ code.

We now also know that multiple inheritance can cause your code a small additional cost in pointer arithmetic when calling member functions of any of its base types that has a non-zero offset within its memory layout.

What was that earlier? about declaration order?

If you’re paying attention, you should have noticed that when we looked at the memory layout of CTestDerived I mentioned in passing that the ordering of CTestBaseOne and CTestBaseTwo within it matches the textual order they were listed in its base-specifier-list.

This is obviously significant, since it implies that if the textual order in which CTestBaseOne and CTestBaseTwo are listed changes, then the memory layout of CTestDerived will change to reflect this.

If you swap the order of CTestBaseOne and CTestBaseTwo around here’s the memory layout printed during the build process:

1>  class CTestDerived	size(24):
1>  	+---
1>  	| +--- (base class CTestBaseTwo)
1>   0	| | _iC
1>   4	| | _iD
1>  	| +---
1>  	| +--- (base class CTestBaseOne)
1>   8	| | _iA
1>  12	| | _iB
1>  	| +---
1>  16	| _iE
1>  20	| _iF
1>  	+---
1>

Given what we have discovered so far, we can see that this new memory layout means that CTestDerived can now be treated as an instance of CTestBaseTwo.

We can also see that with this new layout, the compiler would need to adjust CTestDerived pointers in order to call CTestOne functions.

I leave it as an exercise for you, o budding expert reader of x86 disassembly, to check this for yourself :)

Aside: construction and destruction with single inheritance

Something we entirely skipped past in last post was construction and destruction of inherited types.

This was intentional – construction and destruction behaviour is straightforward with single inheritance.

We all should know the expected high level behaviour for single inheritance (of arbitrary depth) - in summary:

each constructor calls the constructor of its base class before it does the work of its own function definition – i.e. classes are constructed in order ”from inside to out” or “least to most derived“.
destructors do the opposite, each destructor does its own work before calling the destructor of its base class – i.e. classes are destructed in order “from outside to in” or “most to least derived“.

The disassembly matches the high level behaviour in a very straightforward way and I leave it as an exercise for the reader to step through the disassembly of construction & destruction in some test code to see this in action.

Like the rest of the behaviour we’ve discovered so far, when you think about it, it’s actually pretty obvious that this sort of ‘stack-like’ construction / destruction behaviour is required in order to make inheritance work correctly.

Construction and destruction with multiple inheritance

It was pretty obvious that we were coming to this, right?

What happens with construction and destruction when multiple inheritance is involved is less simple.

For example, what order do the constructors of multiple base classes get called in? … and what order do their destructors get called in?

We also assume that – since the constructor and destructor are member functions – there must be some fiddling with this pointers during this process too.

Luckily, this is very easy to empirically determine: we can just add some text output into the constructors and destructors of the sample classes to print the name of the function and the value of their this pointer.

Here’s a link to a VS2010 project I prepared earlier to do just that, I’ve just added a little extra code to the original example code.

Below is the command line output produced when it is run:

You can see that:

constructors are called in the textual order they appear in the class declaration for CTestDerived - from least derived (i.e. CTestBaseOne) to most (CTestDerived).

destructors are called in the opposite order – this is to ensure that work done in the constructors is un-done in the opposite order.

this also shows that the this pointers are changed for the constructor and destructor of CTestBaseTwo just as they did when we were calling regular member functions

At this point you should feel free to swap around the order of CTestBaseOne and CTestBaseTwo in CTestDerived‘s base-specifier-list to check that construction and destruction follow the same rules as the ordering of base types in derived type’s memory layout (they do, I promise).

Summary

That’s it for this time and it was massive! I bet you’re glad I split this off from post 11 now :)

The astute amongst you will have noticed that we have not looked at any code using the keyword virtual. This is entirely deliberate, and that’s for next time.

So, let’s recap what we’ve discovered so far about inheritance…

First, what we learned about single inheritance in part 11:

1) We know that the memory layout of user defined types is fixed at compile time…

2) …and so code accessing a data member of a user defined type can use a constant offset relative to the start address of an instance of the type (see part 10).

3) In single inheritance, the memory layout of a derived type D literally extends that of its base type B.

4) This ensures that the inherited members of B are at the same constant offsets relative to the start address of an instance of D as they would be relative to the start address of an instance of B…

5) …which means that a pointer to an instance of type D can safely be treated as a pointer to an instance of type B…

6) …which in turn guarantees that member functions of type B can safely be called on instances of type D.

We’ve also discovered that if a derived class D class inherits from multiple base types A and B, then this multiple inheritance breaks the convenience of the single inheritance approach somewhat:

7) As with single inheritance, the memory layout of an instance of D contains the member data of both A and B, laid out exactly as it was in each base class.

8) Member functions of both type A and type B will use constant offsets relative to the their this pointers to access their data members.

9) Logically; only either A or B may have an offset of 0 bytes within the memory layout of an instance of D…

10) … consequently a pointer to an instance of type D can only be safely treated as a pointer to whichever of A or B has a 0 byte offset within its memory layout

11) Which base type has a 0-byte offset is determined by the textual ordering of the A and B types within the base-specifier-list of D‘s class declaration

12) …if A were at the 0 byte offset within D, the compiler would need to calculate a compatible ‘this’ pointer whenever a member function of B called on an instance of D (and vice versa)

13) …when an instance of D is created, the instances of A and B contained within its memory layout will be constructed by their own constructors before the constructor for D is called, and…

14) …the order in which A and B are constructed depends on their textual ordering within the base-specifier-list in D‘s class declaration (i.e. they will be constructed in memory offset order).

Disclaimer

The above numbered bullet points are facts we have discovered empirically by examining the behaviour of win32 x86 code created by the Visual Studio 2010 compiler.

Do not assume that code generated by other platforms / compilers will behave identically. It should behave very similarly, but you should check.

In general, for any POD type you should be able to save out its memory to file as binary and load it back into the memory of a different instance of the same type.

Obviously, this will hold true iff you don’t change target platform, compiler, compiler options, alignment specifications, or the declaration of the type.

Thanks

Thanks for peer review go out to Bruce Dawson and Amir Embrahimi; and for general #altdevblogaday admin assistance to Luke Dicken.

Appendix: What does the C++ standard have to say about all this?

I spent some time reading the C++11 ISO standard (or a near final revision of it at least), but even after consulting the source it is not 100% clear to me exactly what is and what isn’t guaranteed by the standard – see the below for more information.

Below is a collection of snippets of information I found in this near-final draft of the ISO C++ 11 standard document when I was looking up various bits for this post (you have to pay ANSI for the actual one!).

If you want more information on any of the points below, click the link above to download the .pdf and search for the indented text – that will get you to the page it was on. This document does not make for light reading!

1. Ordering of base classes in memory within a multiply inherited class.

As far as I can tell, this is not guaranteed by the standard – in fact the draft standard says this:

10.1 Multiple base classes [class.mi]
1 A class can be derived from any number of base classes. [Note: The use of more than one direct base class
is often called multiple inheritance. — end note ] [Example:
class A { /∗ ... ∗/ };
class B { /∗ ... ∗/ };
class C { /∗ ... ∗/ };
class D : public A, public B, public C { /∗ ... ∗/ };
— end example ]

2 [Note: The order of derivation is not signiﬁcant except as speciﬁed by the semantics of initialization by
constructor (12.6.2), cleanup (12.4), and storage layout (9.2, 11.1). — end note ]

which more or less says that it’s not guaranteed.

2. Ordering of data members within a class.

So, it appears that a C++ compiler is allowed to reorder data members of a class in memory vs. their textual declaration order if (and only if) their access control (i.e. public, private, protected) is different:

15 Nonstatic data members of a (non-union) class with the same access control (Clause 11) are allocated so that later members have higher addresses within a class object. The order of allocation of non-static data members with diﬀerent access control is unspeciﬁed (11). Implementation alignment requirements might cause two adjacent members not to be allocated immediately after each other;

I can’t imagine this would ever be a problem for you unless you’re writing a reflection library or similar.

3. Ordering of constructor calls when constructing types with base types

Thankfully, there does appear to be some sanity remaining in the universe; as I managed to find the part of the standard that specifies the order in which constructors are called for types that use inheritance:

In a non-delegating constructor, initialization proceeds in the following order:
— First, and only for the constructor of the most derived class (1.8), virtual base classes are initialized in the order they appear on a depth-ﬁrst left-to-right traversal of the directed acyclic graph of base classes, where “left-to-right” is the order of appearance of the base classes in the derived class base-speciﬁer-list.
— Then, direct base classes are initialized in declaration order as they appear in the base-speciﬁer-list (regardless of the order of the mem-initializers).
— Then, non-static data members are initialized in the order they were declared in the class deﬁnition (again regardless of the order of the mem-initializers).
— Finally, the compound-statement of the constructor body is executed.
[Note: The declaration order is mandated to ensure that base and member subobjects are destroyed in the
reverse order of initialization. — end note ]

TL;DR – (if you are not using virtual base classes) each constructor initialises its base classes in declaration order, then the class members in declaration order, then the constructor’s body is executed called.

Life can be better (part 2 of 3)

Amir Ebrahimi — Sun, 19 May 2013 21:34:24 +0000

In the previous article, I set the context of a personal journey of health within the backdrop of larger issues of quality of life in our industry. In this article, I continue that journey with my own personal experience of burnout and depression and what it took me to regain my own health, vitality, and enthusiasm for the work that we all do. My personal wish is to not see any more bright, creative people suffer from work-related stress. Extended stress has odd effects on both the body and the mind.

NOTE: Please consult with a health professional before making any changes with your own health. I am certainly not a doctor and will assume that you are an adult that will ultimately be responsible for his or her own health. I’ve done my best to portray this post as personal truth and not accepted science. I realize that every person’s body is different and therefore may respond differently to a specific health regimen.

Background

I grew up playing role-playing games (RPGs) like other hackers, geeks, nerds, and escapists out there. However, fantastical worlds like that of Dungeons & Dragons didn’t grab my interest as much as sci-fi did. Although I remember getting lost in the dark, gritty world of Shadowrun on my Super Nintendo (based on the RPG), for some reason I chose Cyberpunk 2.0.2.0. (a different RPG) when it came time to run a campaign. I loved the idea of a futuristic world where man and machine could meld. Through Cyberpunk, I became acquainted with the use of the term “meat,” crassly used to describe the human body. “Cyberspace” was the place where netrunners existed, while their “meat” resided in “meat-space,” otherwise known as the physical world. Essentially, a netrunner could bring consciousness into an alternate world without reliance on the body.

I connected with Cyberpunk as a teenager mainly because of my growing interest in computers. As I tried to figure things out with computers, I found that my mind gravitated towards combing through my thoughts to better process them, as a memory manager might do to defragment allocations. I also found that I was entertained simply with thought itself, traversing different paths to see where my mind would end up. I realize that these behaviors are what primarily distinguish introversion from extroversion and as a result, believe that they lend the identification of self to the thoughts and processing of the brain. Instead of “I am what I can do” it becomes “I am what I can think”. I also think that those same behaviors help us become better software engineers. Even though I enjoyed skateboarding, football, baseball, paintball, and other physical activities, I somehow managed early on to form a mental model where the brain/mind and body had little, if nothing at all, to do with each other.

Oddly enough, my naivety around health later in life didn’t stray far from that teenage fascination with cyber-augmented posthumanism. In the last article, I mentioned that after starting to work out w/ co-workers at lunch, I was able to get into the best physical shape that I had been in since high school football (hey, in Alabama, we did two-a-days). Unfortunately, even with having increased strength, my energy, mood, and mental health were declining. At age 25, I thought that health had more to do with the way I looked than how I felt. In all honesty, I’d rather have a sexy mind rather than trying to look sexy. Well, during the summer of 2005, in the midst of my third extended crunch for a console game, I finally hit rock bottom and decided that I needed help.

Depression

I used to think that it was shameful to talk about depression and/or any mental impairment. I believed that this was one thing that I was supposed to figure out on my own. In my judgment, sitting with shame around depression let me run circles around questions of “Why?”, rather than looking at the “What?”. As I wrote in my previous article, I did notice the warning signs of feeling chronically tired, feeling “heavy” to move around even though I wasn’t grossly overweight, and starting to think that most things I had to get done were weighty tasks. For me, this was a dramatic change from how I remember feeling when I got my first internship in the industry. I went to a general practitioner for a check up and when the blood work came back inconclusive I decided that I just had to push on. I continued that way until things got worse and I didn’t know what else to do, so I finally decided to see a psychologist.

According to the National Institute of Mental Health (NIMH), “depression is caused by a combination of genetic, biological, environmental, and psychological factors”. In other words, we don’t quite know the causes. Trying to answer those questions of “Why do I feel this way?” or “How did I come to this place?” resulted in many years of sitting with different psychologists. I think a good psychologist can provide temporary help to calm acute anxiety or depression with talk-therapy, but dialogue is less useful at resolving any root physiological cause(s) of emotional disturbance when they exist. While I’ll agree that psychotherapy has its own benefit to healing emotional wounds and looking at long-standing issues, on a practical level of my day-to-day experience it didn’t provide the answers.

Let’s be clear about one thing: I am not saying that stress in our industry causes depression for everyone. Some level of stress is actually healthy in helping us to achieve our goals. Chronic stress, on the other hand, seems to show up in different ways for different people. Separate from major health problems, I have seen stress have impact in minor ways, too, such as frequent irritability or negativity, relationship problems, or motivational challenges to name a few. What I can say is that, at the time I experienced depression, I was working more (including weekends), sleeping less, had little time to socialize, and eating outside for most of my meals if not skipping them altogether. My worst night was working until 6 am on a build that we had to send off to magazine reviewers and being asked to come back in around 10 am. Most crunches I experienced at different companies from 2003-2008 lasted for more than 4 months out of the year. Even though there was some time off and a slower pace following the release of a game, somehow my batteries were not getting recharged.

Moving past the shame allowed me to see the questions of “What can I do?”. Well, in parallel with psychotherapy, I also met with a psychiatrist to hear another point of view. I worked through different SSRIs, SNRIs, and anti-anxiety medications; battling different side-effects in an effort to find the “right one for me”. My judgment of these drugs are that they can work as a crutch, but in the same way that caffeine can keep me operating out of touch with my natural energy levels. The drugs provided relief and got me back on my feet, however, in the end they pale in comparison to my current experience at a holistic level.

When I experienced depression, it was damn near impossible to distinguish that I was not my thoughts. At work, I was unhappy with most of what I was tasked with doing, irritable with requests from others, not motivated to get my work done, and negatively thinking about what the company was doing and the manner in which we were doing it. Now, I’m not saying that some form of each of those thoughts/feelings/behaviors aren’t welcome in order to alert me to doing or suggesting something better. However, preponderance of these thoughts/feelings/behaviors as my M.O. led me to believe that I am a negative, unhappy, and unmotivated person. My invitation to you if you’re in that space: don’t draw conclusions there. From my experience, I can attribute more of the ways I was thinking at that point in my life to my overall physical health.

After experiencing a gradual decline in my state of being over the years, I realize now what negligence towards health in the face of stress has afforded me. I also carry deep concern for any others that, like me, are/were not as well equipped to deal with stress. As an employer, I think it is ever important to protect our employees from burnout. In the process of learning about my own health, I’ve come to a rudimentary understanding of how stress works in the body.

Stress

The PAT (pituitary-adrenal-thyroid) axis is an important system that regulates the hormones in the body in order to keep the body in balance. I think of them as good friends who have my back during good and bad times (regulators, mount up!). When stress comes and goes, each gland does their part. Everything I encounter in the day impacts my PAT axis from my alarm clock in the morning to the traffic on the way to work to the tasks I’m trying to complete on any given day. Additionally, how I choose to react to those stressors can also tax my system. All of these stressors deplete the stores of hormones and endorphins in my body, which is why I’m sometimes exhausted by the end of the day and not able to handle much more. The food I eat helps replenish what has been used after being digested and synthesized into what the body needs. However, under chronic stress the body is producing elevated and continued levels of hormones and endorphins in an attempt to keep the body in balance. Unfortunately, the immune and digestive systems are suppressed, among other things that are affected, which can lead to a non-optimal state for both energy absorption and perception of well-being.

I mentioned perception in the previous article because I wanted to allude to the possibility of being stuck in a fearful, anxious, doubtful, exhausted, etc. state that elevated stress levels can help bring about. To put this in perspective, I consider what happens when adrenaline rushes through my body and how much my thinking can change in the moment. I’ve experienced this while driving when I am in a happy, easy-going mood and with a flash of a potentially dangerous situation with another car I have both a physical and emotional/mental reaction. The world I was seeing as beautiful only moments ago has now become “out to get me.” In a similar way, I see elevated levels of stress having effects on my perception of how I’m performing, the relevance or efficacy of what I’m working on, how others are doing, and so on.

Beyond Depression

Even though I got back on my feet eventually, I somehow never felt as good as I did when I entered the industry. I no longer felt terribly bad, but I didn’t feel energized or great either. Unrelated to energy levels, stress, depression, or any other things I’ve spoken to already, about a year and a half ago I had started working with a nutritionist in preparation for a surgery. The belief was that I would have a quicker recovery from the surgery if I was in a healthier state. So, laughably now, I considered “getting healthy” as something I’d do for a short while up until the surgery. Gradually, I changed my diet, started drinking more water (which was up from almost none at all), reduced my caffeine intake, took tailored supplements and started to feel radically different in the first three months. I think it is more a testament to the shoddy state I was in more than any one thing that I was doing in particular. I started noticing that when I woke up I had energy to boot and was ready to take on the day. I also noticed that my brain function seemed to change. I no longer would get stuck worrying about something incessantly. My brain processing was becoming razor sharp again and my focus improved. When I started to feel the anxiety that I carried around ease up and become more manageable, I became dumbfounded. I realized only after feeling much better that the one question I recall none of the health professionals asking me from 2005 all the way up until 2011: what was I actually eating?

Food for Thought (and Energy!)

If I try to remember what my meals looked like when I started working professionally in 2003 (and to some extent all the way back to 1999 at GaTech) up until a year and a half ago:

Breakfast

Cereal w/ milk, muffin, or bagel w/ peanut butter or almond butter; pancakes or eggs, bacon on the weekends

Lunch

Usually Mexican/Indian/Thai food (varied over the years depending on where I was working), burger w/ fries, occasional salad, sandwich with chips

Dinner

If working late, then whatever food was brought in if the company catered; otherwise, whatever food was close around (likely the same places I ate at during lunch); or skipping dinner and eating some kind of snack at home later in the evening (usually another bowl of cereal or PBJ or whatever could be made quick)

When not working late – frozen pizzas or pre-packaged food that could be warmed up (sometimes in microwave, sometimes in oven, sometimes in saucepan); pasta w/ tomato sauce; mac and cheese; thai/indian take-out

Fluids

Orange juice in the morning
Coffee many times throughout the day, which also killed my appetite
Very little water if none at all (would drink water during workout sessions)
Occasional soda

Certainly anyone can look at this and say, yeah, that doesn’t look like the healthiest diet, but at first glance I wouldn’t think someone would say there was anything particularly wrong with it either. However, there is one thing that is missing in significant quantity – vegetables. There would rarely be any sort of vegetable or green on my plate unless it snuck onto the plate (luckily, Thai or Indian food usually has vegetables).

Fast forward to today and I now consume the following:

Before breakfast

10 oz. of carrot juice (used to buy this in the store and then decided to simply buy my own 5 lb bags for the week and get a juicer); I recently started juicing the leftover stalks of whatever green I’ve been eating in addition to the carrot juice

Breakfast

Two eggs, parsnips w/ cinnamon or seasoned cauliflower / rutabagas w/ diced onions + some sort of green sauteed in butter (chard, spinach, kale, or collard), two slices of toast

Before lunch

I usually try to drink 1-2 quarts of water before lunch. I have a stainless steel water bottle that is close to a quart that I keep filled at my desk. I don’t usually have any liquids when I’m eating a meal to aid in digestion. I’ll drink something before or an hour after my meal.

Lunch

Usually, this is some form of leftover from dinner the previous night(s) that I bring to work and warm up in a toaster oven. Occasionally, I eat something from around the office, but this was difficult initially because I find that most places don’t include fresh vegetables or greens with a good sized portion of protein

Dinner

Two small lamb loin chops (pan-fried, then broiled) w/ brussel sprouts (sauteed)
Ground lamb and sweet potato (steamed and mashed w/ butter) tacos w/ onions, garlic, and chard, collard greens, or kale (sauteed) in a brown rice or corn tortilla + additional sweet potato, onions, garlic, and greens on the side
Turkey burger w/ broccoli, cauliflower, beets (both gold and purple), or whatever else I have left in the fridge
Seasoned, baked chicken breast w/ blanched and then sauteed green beans
Turkey meatloaf w/ kale chips, baked cauliflower or turnips/rutabagas

Miscellany

Occasionally, I mix whole grains in: brown rice, quinoa, millet
I’ll eat fish and beef occasionally
Dairy products come and go in my diet and when I do eat them I go for the whole fat, raw versions when available (or at least vat or low-heat pasteurized)
I’ll have fruits occasionally as a dessert or a snack

All of these meals are makeable in about 30 minutes. When I am eating later than I like or have to run to Whole Foods to pick up groceries, then I’ll usually grab stuff from the hot food bar — again, loading up on vegetables; including ones that I don’t buy regularly, such as bok choy.

The key here is that I eat vegetables now in significant quantity due to the mix of vitamins and minerals present. Two-thirds of my plate will be vegetables and only a third will be a hand-sized amount of protein/fat (some vitamins that come from vegetables need fat to be absorbed) and occasionally some starch such as brown rice or quinoa. If I’m eating out somewhere, I’ll usually request for them to double whatever vegetables come on my plate.

My cupboard is bare for the most part – no more cereals, potato chips, dry pastas, crackers, etc.

My fridge is full of organic vegetables and meat.

I’m mindful of sugar.

I cook with butter usually for breakfast and olive oil (on low/med heat; there are better oils for high heat) for all other meals.

I drink lots of water (3 quarts) and usually only water during the day (occasionally a kombucha or decaf tea) — not during meals.

I use a rice-cooker w/ a steam basket to steam most of my vegetables (quicker and easier than the stove top) – I bought one from a Japanese food store

I still enjoy an occasional coffee, sweets, pasta, pizza, etc. from time-to-time, however, the key is that I keep none of that in the house.

My grocery bill for the workweek is roughly $75 with some carry-over to the weekends. The most expensive items are meat, eggs, and carrots (because I’m buying those in 5 lb bags). Vegetables are for the most part cheap in their individual quantities.

I didn’t eat this way before because I considered food to be a nuisance. Even though I had enjoyed great meals in restaurants or cooked by others, I treated that as a luxury. My relationship with food was that I needed to be doing something else other than preparing a meal or taking time to enjoy it calmly. So, when I was hungry, my intention was simply to eat something as quick as possible and get back to what I was doing. One way I could characterize my prior understanding of food is that I could eat anything and produce anything. My new understanding is garbage in == garbage out. Now, that’s not entirely true because I know that I was still able to develop working code under extreme stress even though I was not eating well. However, I also know now that I was not producing at my optimal performance.

I see food as potential energy. Whatever food I am taking into my body is, ultimately, a stored form of energy from the sun. It goes through millions of chemical processes to reach the stage that it is at when I ingest it. That potential energy gets used either for physical or creative work. To me, a calorie is no longer a calorie and I don’t view things in a simple “bad” food vs “good” food dichotomy. I choose organic because I’m looking to maximize the nutrients in my food, not simply because non-organic is “bad” (and yes, I’m aware that the term “organic” is prone to abuse in efforts for companies to make more money). I choose specific foods over others for the same reason. I prepare my food and use whole ingredients because I like knowing what I’m putting into my body.

Summary

I could go on about a lot more things, such as how the body needs healthy levels of certain amino acids and minerals for the optimal functioning of the brain, the glycemic index, or what foods are nutrient rich. However, the most valuable guiding strategy that I have learned from all of this is that nutrition is more about how I feel and less about how I look, which is why I misunderstood and took no interest. I knew that eating healthy was important; however, I had never experienced first-hand the effects, nor did I have a reason to. I think that nutrition is even more important than exercise and that physical activity comes naturally as an effect of having more potential energy in my body. My invitation to you is to find out what works for you. Become the patient observer. Pay attention to how you feel after meals. Notice what improves your moods. Take time to prepare your own food and enjoy it stress-free, so you can maximize the nutrients you are taking into your body. What motivates me to continue is twofold: I don’t want to go back to feeling the way I used to feel and, more importantly, I want to see what’s possible.

I’ll also add that everything isn’t perfect today. I get irritable at times. I wake up on the occasional morning and feel sluggish. I still battle with anxiety. I’m exhausted sometimes at the end of the day. I’m frustrated occasionally with the performance of my own work or with what we’re doing as a team. However, all of those things are blips now where before they were part of my routine, daily experience. Having a programming background, I simply see this as an optimization problem to figure out what can be done better. However, optimizing the human body is a bit more difficult than shaving cycles off of a routine. Maybe in a cyber-augmented posthuman world of the future, we’ll have the equivalent of a software profiler for the human body.

In the next article, I’ll shift focus to the changes I’ve made that impact how work gets done both at a studio and individual level, talk a bit about what I’ve changed in regards to sleep, some practical things I do to conserve willpower throughout the day, and tie up any loose ends with this series.

C/C++ Low Level Curriculum Part 11: Inheritance

Alex Darby — Fri, 03 May 2013 21:06:20 +0000

Hello, and welcome to the 11th part of the C / C++ low level curriculum. About time? Definitely!

Last time we looked at the basics of User Defined Types: how structs, classes, and unions are laid out in memory; and (some of) the implications of memory alignment on this picture.

In part 11 we’re going to look at how inheritance affects this picture, in particular the implications for memory layout of derived types and also for their behaviour during construction and destruction (note: we’re leaving multiple inheritance and the keyword virtual out of this picture to start with).

Before We Begin

Another big assumption I’m going to make is that you’re already very familiar with the C++ language and comfortable using the language features we’re discussing, as well as the accepted usage limitations of those features etc. If I need to demonstrate anything out of the ordinary I’ll explain it – or at least link to an explanation.

With this in mind, in case you missed them, here are the backlinks to the previous posts in the series:

I won’t lie – it’s not light reading :)

Class vs. Struct: a Gentle Reminder

The C++ keywords struct and class define types that are identical in implementational detail and what you can do with them (the only difference being at the language level: the default access specifier if none is specified is private for class, and public for struct).

So, whilst I will be using the keyword class throughout this article please take it as read that anything we talk about here applies equally to types defined using the keyword struct.

What happens when we derive from another type?

So, what does happen when you derive a user defined type from another non built-in type?

Clearly the data members you specify in the declarations have to go somewhere, and so do all those specified in the type(s) you are deriving from.

At the level of C++ there is nothing other than the standard to tell you how this works – and nothing other than looking at what happens with the code generated by the compiler you are using will tell you for definite.

As in the last post, we will be relying heavily on the frankly awesome secret 007 compiler flag /d1reportSingleClassLayout in order to tell us exactly how the (Visual Studio 2010 win32 x86) compiler has decided to lay our example structures out in memory.

It’s about time to look at some example code, so, rather than have you go through the usual rigmarole of setting up your project I have kindly set one up for you.

The zip file in this link contains a VS2010 solution with a single project and .cpp file lovingly set up to run the code shown below, which is in 00_Inheritance.cpp

class CTestBase
{
public:
    int _iA;
    int _iB;
};
 
class CTestDerived
: public CTestBase
{
public:
    int _iC;
    int _iD;
};
 
int main(int argc, char* argv[])
{
    return 0;
}

When you compile this project you should get the following in your “Build” output window (the magic of /d1reportSingleClassLayout!) :

1> class CTestBase size(8):
1>   +---
1> 0 | _iA
1> 4 | _iB
1>   +---
1> 
1> class CTestDerived size(16):
1>    +---
1>    | +--- (base class CTestBase)
1>  0 | | _iA
1>  4 | | _iB
1>    | +---
1>  8 | _iC
1> 12 | _iD
1>    +---

Looking at this, it should be fairly obvious that the data members of CTestDerived have just been concatenated onto the end of the memory layout of CTestBase - and, more importantly, that the memory layout of CTestBase within CTestDerived is identical to that when it’s not a base class.

It’s that simple! (for certain definitions of ‘it’ and ‘simple’…)

Armed with this information from last post:

“A guarantee is given in both the C and C++ language specifications that memory address of each member will be higher than that of the one declared before it (see this post on Stack Overflow for more detail of the wording).”

it is obvious that – since CTestDerived inherits all of the members of CTestBase - its members must appear after those of CTestBase in memory.

I remember when I had this explained this to me - not long after having started my first job in the industry as a fresh faced graduate – I did the internal equivalent of a double-take, because the information I had just received was so bleedingly obvious that I couldn’t believe I’d ever not known it.

If it’s that easy, why post about it?

Good question!

The fact that the memory layout of a type is identical in all situations is required by the standard – and also by logic – let’s see why…

First, download and open the second zipped VS2010 project file - this contains the code below in 01_InheritanceWithFunctions.cpp:

class CTestBase
{
public:
    int _iA;
    int _iB;
 
    CTestBase( int iA, int iB )
    : _iA( iA )
    , _iB( iB )
    {}
 
    int SumBase( void )
    {
        return _iA + _iB;
    }
};
 
class CTestDerived
: public CTestBase
{
public:
    int _iC;
    int _iD;
 
    CTestDerived( int iA, int iB, int iC, int iD )
    : CTestBase ( iA, iB )
    , _iC ( iC )
    , _iD ( iD )
    {}
 
    int SumDerived( void )
    {
        return _iA + _iB + _iC + _iD;
    }
};
 
int main(int argc, char* argv[])
{
    CTestBase       cTestBase   ( argc, argc + 1 );
    CTestDerived    cTestDerived( argc, argc + 1, argc + 2, argc + 3 );
 
    return cTestBase.SumBase() + cTestDerived.SumBase() + cTestDerived.SumDerived();
}

Put a breakpoint on the return statement from main, and then compile and run the release build configuration.

The first thing to note is that the memory layouts printed to the output window during the build are unaffected by the addition of these functions.

This is what you would expect, as we know that non-virtual member function calls are resolved at compile time just like regular non-member and static member functions.

Since CTestDerived is derived from CTestBase, we know from our high level knowledge about C++ that we can call both of these functions on an instance of CTestDerived – what we’re looking at right now is how this is implemented.

When the breakpoint is hit, right click and choose “Go To Disassembly”.

I’ve pasted the part I’d like to discuss below…

(N.B. to get the same disassembly as this you should have the following Viewing Options checked in the disassembly window: ‘Show source code’, ‘Show line numbers’, ‘Show address’, and ‘Show symbol names’)

    44:     return cTestBase.SumBase() + cTestDerived.SumBase() + cTestDerived.SumDerived();
0129109A  lea         ecx,[cTestDerived]  
0129109D  call        CTestDerived::SumDerived (1291060h)  
012910A2  lea         ecx,[cTestDerived]  
012910A5  mov         esi,eax  
012910A7  call        CTestBase::SumBase (1291020h)  
012910AC  lea         ecx,[cTestBase]  
012910AF  add         esi,eax  
012910B1  call        CTestBase::SumBase (1291020h)  
012910B6  pop         edi  
012910B7  add         eax,esi

We’ve previously covered that the win32 calling convention for member functions (‘thiscall’) passes this to member functions in the ecx register.

Correspondingly, you’ll notice that the address of cTestBase and cTestDerived are being stored in ecx using lea (‘load effective address’) immediately before calling their member functions.

Specifically, note that the address of cTestDerived is passed un-tampered with in ecx when calling the base class function CTestBase::SumBase. Remember this for later (and for the next post!).

So, let’s look at the disassembly for CTestBase::SumBase and CTestDerived::SumDerived - I tend to single step the disassembly and step into them, but putting breakpoints in them is more reliable :)

CTestBase::SumBase

    14:     int SumBase( void )
    15:     {
    16:         return _iA + _iB;
01291020  mov         eax,dword ptr [ecx+4]  
01291023  add         eax,dword ptr [ecx]  
    17:     }
01291025  ret

CTestDerived::SumDerived

    33:     int SumDerived( void )
    34:     {
    35:         return _iA + _iB + _iC + _iD;
01291060  mov         eax,dword ptr [ecx+0Ch]  
01291063  add         eax,dword ptr [ecx+8]  
01291066  add         eax,dword ptr [ecx+4]  
01291069  add         eax,dword ptr [ecx]  
    36:     }
0129106B  ret

We can see that all offsets from ecx used in both functions correspond to the memory layouts we have in the build output for the type that the function belongs to.

Since _iA and _iB are at the same offset within both CTestBase and CTestDerived (i.e. 0 and 4 bytes respectively), CTestBase::SumBase can safely be called on instances of CTestDerived.

We already know that this is possible from our high level understanding of C++, but now we know the implementational detail that makes it possible.

Whilst the specifics of the disassembly will probably differ from platform to platform, the principles underlying its operation should not.

Summary

To summarise what we’ve established so far :

1) in member functions, member data of a class is accessed via specific offsets from the this pointer

2) these offsets are constants at compile time and are baked into the assembly code for the member functions

3) this means that the memory layout of the members of a given class must always be identical or the member functions won’t work

If we follow this logic through, we can see that:

4) the memory layout of a class B that inherits from another class A must contain class A‘s members in the same memory layout as class A

5) the memory layout of any given class A is identical regardless of whether it is an instance of A, or it is included in the memory of some type derived from A.

6) note: this behaviour is required by the standard, and (more significantly) by logic.

Finally, it follows that (because each member of a struct must have a higher address than those declared before it):

7) the extra memory required by derived class B will be concatenated onto the end of the memory layout of its base class A

That’s all for now – next time we’ll look at how multiple inheritance affects this picture.

I know it’s pretty short, but this just means the next one will get here more quickly :)

Epilogue – for those who wondered what I changed in the project settings

There’s quite few changes to the default VS2010 win32 console app project properties in the projects I’ve zipped up for this post.

The changes have to do with making the optimised release build configuration leave the code structure alone (i.e. not strip out or ‘fold’ functions to save exe size, prevent functions being inlined), and prevent extraneous ‘debug checking’ code being inserted (makes function calls slower, and code less easy to follow in disassembly)

turning off ‘Whole Program Optimisation’ (Configuration Properties->General)
turning off ‘Inline Function Expansion’ (Configuration Properties->C/C++ ->Optimisation)
turning off ‘Basic Runtime Checks’ (Configuration Properties->C/C++ ->Code Generation)
getting rid of pre-compiled headers to streamline the number of files (Configuration Properties->C/C++ ->Precompiled Headers)
turning off ‘Enable COMDAT folding’ (Configuration Properties->Linker-> Optimization)

Essentially, this makes the Release configuration assembly have the same structure as the Debug one WRT function calls.

Also, I use the argc parameter to main as input to the code, and return value computed from that so that the optimiser can’t assume constant input or output values.

If you use constant inputs, or don’t output a value computed from the inputs then it’s pretty hard to convince the optimiser not to optimse the entire .exe to ‘return 0;’… ;)

Shout out

Thanks (again) to Bruce – king (or at the very least duke) of advice and peer review.

Code Share: Source Censoring, Part 2

Niklas Frykholm — Tue, 30 Apr 2013 15:38:24 +0000

A while ago I shared the tool we use for censoring source code in the Bitsquid engine.

Quick recap: We need to censor the code in our source distributions because there are parts of the code that are covered by NDAs to third parties and cannot be publicly disclosed. We do this with a tool that strips out the secret code and replaces it with blank lines, based on preprocessor definitions.

The stripping tool is only part of the solution, though. It works well if you only distribute code drops. You take a snapshot of the code, run the stripping tool to strip out secrets, zip it up. Done!

But frankly this is a terrible way of distributing source code. There is no history, no indication of what has changed from version to version and keeping local changes merged with the mainline is a constant pain in the ass.

The only sane way of distributing source code is to expose a mercurial (or git) source repository that you can pull changes from. This lets customers examine the history, find out which version introduced a particular bug, maintain their own branches that they merge with the mainline at their convenience, etc.

But of course, we cannot just share our own internal repository (because it contains secrets).

hg-clone.rb

We handle this with another tool, that we have inventively decided to call hg-clone.rb.

What hg-clone.rb does is pretty straight forward. Given two repositories as argument, a SOURCE and a DESTINATION, it checks out each revision in the SOURCE repository, runs a filter program (to strip out any secrets) and checks the result into the destination repository.

SRC:    0  --> 1  --> 2  --> 3  --> 4  --> 5  --> ...
    	|      |      |      |      |      |
    	F      F      F      F      F      F
    	|      |      |      |      |      |
        v      v      v      v      v      v
DST:    0' --> 1' --> 2' --> 3' --> 4' --> 5' --> ...

You call the program as

hg-clone SOURCE DESTINATION --filter FILTER --target TARGET-REV --cutoff CUTOFF-REV

SOURCE and DESTINATION are the source and destination repositories. DESTINATION does not need to exist, if it doesn’t it will be created. FILTER is the filter program, it will be run once in the destination directory before each revision is committed.

TARGET-REV is the target revision that should be copied from the source to the destination. hg-clone will first transfer the parent(s) of the target revision to the destination repository (if they haven’t already been transfered), then it will transfer the target revision. This process is applied recursively, so if the parents’ parents haven’t been transferred, they will be transferred first, etc. Only the revisions that are ancestors of TARGET-REV will be transferred, so you can have secret development branches that won’t be copied to the destination until they have been merged with your release branch.

If you don’t specify a TARGET-REV, the last revision in the source repository will be used.

CUTOFF-REV can be used to cutoff the recursive parent transfer at a specific revision. If you set the cutoff to revision 1000, then any revision that has a parent before revision 1000 will be reparented to revision 1000 in the destination repository. Essentially, in the destination repository, history will start at revision 1000. This can be used to hide a shady past.

hg-clone tries its best to preserve authors, dates, messages, branches, etc between the source and destination repositories. It cannot however preserve version numbers, since those are based on a content hash, which changes when the filter is applied. What it does instead is to insert a marker [clonedfrom:116:91fe33c1a569] in the commit message that specifies which revision in the source repository the current revision in the destination repository was cloned from. This commitment marker is also used to determine the mapping between revisions in the source and the destination and whether a particular revision has already been copied over or not.

To use this in practice, you would typically set up one external repository for each customer with a corresponding filter program for stripping out the things that customer is not allowed to see. Then you would set up a cron job to run hg-clone and copy revisions from the main repository to the customer’s.

Instead of having one repository per customer, you could alternatively have one repository for each possible NDA combination (e.g., +PS3 +PS4 -X360). However, this can be problematic, because if a customer becomes disclosed for a platform you will have to switch them over to a new repository, which might be painful. If you have one repository per customer you can just change the filter function.

The hg-clone program is available from our bitbucket repository.

This has also been posted to The Bitsquid blog.

“App Of The Day” type apps can help indies with app discovery

Kyle-Kulyk — Fri, 26 Apr 2013 23:11:45 +0000

App discovery has to be one of the biggest hurdles that independent developers will face in the mobile marketplace. When limited marketing budgets butt up against a crowded app space, it’s difficult for small devs to attract attention to their work. This is one of our primary obstacles we’re facing with our new release, Vex Blocks for Android. We’ve made the game, people seem to like the game, how do we get people to notice the game with our marketing budget as constrained as it is? I’d had a successful mobile developer recommend to me in the past that when you launch a new game, all you need to do is to pay the Russians $5,000-10,000 to get you to the top of the app charts and then see if you stay there. If your app doesn’t stay there, move on. That wasn’t the particular answer we were looking for with this game release and that’s where App Turbo helped out.

About a week and a half after we launched the paid version of Vex Blocks on Google Play, Yuhao Zeng of the Paris based App Turbo contacted me to ask if we’d be interested in promoting Vex Blocks with their “App of the Day” app. They expressed interest in our game and thought their users would too. The idea was simple enough. We make a full copy of our paid game available for 24 hours, free, and they’d promote it. As we had our hands full with free version of Vex Blocks, our app marketing had at that point consisted of a few press releases (where we could afford them) and emails to review sites who had historically ignored us. We received the details, talked them over among ourselves and ultimately decided on “What the hell?”

I don’t want to sound like an advertisement for their service, but I like when others share their indie development experiences and if I can help another dev in our shoes, I’m going to pass it along. Yuhao was fantastic throughout the process, quickly answering any and all questions I threw his way. The “App of the Day” app is regional, covering a good chunk of Europe as well as Russia, Canada, Australia, Mexico, Brazil and Japan. Prior to the promotion, the free version of Vex Blocks was managing roughly 100 downloads a day in the 2 weeks since it’s release. We were hoping that perhaps this promotion might give us a boost of a couple of thousand users and help us get the ball rolling a bit.

When all was said and done, over 90,000 users downloaded and played our game in the 24 hours the promotion ran this week, of which roughly 80% have so far stayed as active users. I’m sure that’s nothing to some developers, but for an indie team like ours this is a huge step forward. The next day, the free version of Vex Blocks jumped from it’s regular 100 a day to roughly 6000 where it’s stayed so far. The paid version of Vex Blocks is also experiencing a nice spike. For an English only game, I’m pleasantly surprised that so many decided to give us a try in non-English speaking countries and the feedback from users has been fantastic. The goal was to help Vex Blocks find it’s way into the hands of gamers and raise awareness, and in that this exercise was a complete success. It also served as an excellent opportunity to help narrow the list of the 2000 some Android devices that the game was available on and weed out a few devices that were incapable of playing our game.

There are a number of “Free App” type applications like “App of the Day” (http://appturbo.it/) out there but in our inexperience we didn’t even consider reaching out to them. Although I haven’t used a similar service to promote our titles, I’ll certainly contact the App Turbo team again for this type of service. If you’re a developer with a mobile app to promote, it certainly couldn’t hurt to send a quick email to Yuhao (yuhao@appturbo.it) and see what they can do for you. I know I’ll be partnering up with them again for the iOS release of Vex Blocks and hopefully for future releases after that.

Life can be better (part 1 of 3)

Amir Ebrahimi — Mon, 22 Apr 2013 17:03:25 +0000

TL;DR – If you’re happy, healthy, and have energy to make things happen most of the time, then keep doing what you’re doing. If not, or if you used to be and now aren’t, then you may be interested in what I have to say.

This May will mark the tenth year that I have worked in the video game industry. Technically, I could be classified as a “veteran”. However, whenever I think of that term — veteran — I conjure up images of other great developers, some of whom I have had the good fortune of working with, that are senior to me. Regardless of the appropriate use of the term veteran, I’m calling attention to it because one out of every three of us who came into the industry were not expected to be around after 5 years. Extend that period to ten years and it would either be you or the guy / gal next to you that would leave behind an empty chair.

I’m citing these numbers from a Quality of Life Survey put out by the IGDA that I remember coming across back in 2004; only a year after joining the video game industry. I was disheartened to read that attrition was a concern in our industry and that, among all the reasons given, long hours and crunch were omnipresent. In other words, people weren’t leaving the industry because it was unfulfilling as a career. No; people were leaving because it was a choice between their health / sanity / relationship / family / social life / etc. and a creative endeavor that was bleeding them dry.

My experience at that time with others in the industry was that neither the IGDA, nor it’s report on quality of life were to be taken seriously. In my judgment, there was cowboy bravado at play that took pride in the fact that we worked ourselves to the bone; all of which was “necessary” to produce the next hit title. If you couldn’t keep up with that pace, then you weren’t cut out for the industry. I even had a senior manager tell me in an exit interview that he was surprised I was headed to work for another company in the industry instead of leaving altogether.

I remember sifting through pages of the Department of Labor’s Enforcement Policies to understand the legalities around working unpaid, and for all effective purposes, obligatory, overtime. For reference, I’ve managed to dig up the original highlighted sections from my files as I’m writing this blog post: 49.1.2, 49.1.5, 49.2.1, 50.3, 50.6, 50.7, 51.5.2, 51.6.3.2, 51.6.4.1, 51.6, 54.1, 54.4, 54.5. If this stuff bores you to tears, then I’ll inform you that the crux of all questions around overtime as it comes specifically to “computer software workers” is whether you are classified as an exempt (as in from overtime pay) or non-exempt employee. See, California, being a hot-bed of software innovation, for reasons not known to me, decided to single us out for protection. One of the three criteria, of which all must apply, for an employee to be exempt is that you are paid at least a minimum hourly rate for your time.

You may be wondering why I would be interested about an IGDA whitepaper, legalities of overtime, and classification for pay only after a year of being in the industry. Well, although I didn’t have much awareness about my well-being back then as I do now, I did know something was wrong. My first indicator was health; noticing that my overall energy level day-to-day had declined. I found that I was chronically tired, felt “heavy” to move around even though I wasn’t grossly overweight, and started to think that most things I had to get done were weighty tasks. The second indicator was my mood; noticing that the enthusiasm and optimism that I entered into the industry with had been replaced with irritability and negativity.

I decided to see a doctor for a health check up to see if anything was gravely wrong. Oddly enough, the blood work that was done indicated that nothing was awry. In fact, the results even showed that I had a high level of “good” cholesterol (HDL) and a low level of “bad” cholesterol (LDL). Nothing the doc found was abnormal and I was told to get better sleep and exercise more (which I eventually did start doing during lunch with coworkers). Had I had the awareness that I have now to understand that health was my responsibility, I would not have accepted the blood work as conclusive. My current understanding of doctors is that they aren’t in the business of making us healthier; they’re there to make sure we don’t get any sicker than we already are.

Over the next year, I worked on my third console title, lifted weights and exercised multiple times a week with co-workers, ate salads occasionally during lunch, and, unfortunately, was also drinking numerous cups of coffee during the day to fuel the work that I was doing. Commonly said, I was working hard and playing hard. Sure enough, at some point during development, we entered our multiple-month crunch during the project and sleep and free-time became a luxury. What’s odd about that time is that I was in the best physical shape I had been in since my years playing football in high school, yet my energy level, mood, and mental health were getting worse.

In parallel, a letter from a disgruntled ‘EA spouse’ and a class-action lawsuit against EA by a collection of artists both emerged in late 2004. Both spoke to the unfair working conditions that employees faced at EA. In 2005, the class-action lawsuit was settled for $15.6M and would later be succeeded by a separate class-action lawsuit composed of programmers (also settled out of court). In comparison to the film and television industry, the labor practices of the video game industry were and in some cases still are juvenile. For film and television, professionals get paid for overtime and although the health effects of crunch aren’t mitigated, at least the costs are made material.

I experienced overtime pay firsthand, when I joined a visual effects company, The Orphanage, here in San Francisco in 2008. I learned quickly how volatile the film industry could be — projects came and went, hours went up and down, and staff increased and decreased on a week-to-week basis. However, even with the volatility, there was a fundamental understanding that everyone would get paid for his or her time. The California Department of Industrial Relations released a working order in 2001 specifically for the film industry. The working order dictated how overtime would work within a single workday and workweek, required minimum days off, alternative workweeks, meal and rest periods, and even the temperature for working conditions. Even though I had a salary, I logged every hour that I worked against different projects at the company for budgeting purposes. What I found so cool about this process is that, not only would I be paid overtime when I worked longer hours for a specific project — it would show up in the cumulative costs for that specific project. Essentially, accountability was built into the process, since having people work longer hours resulted in more costs for the project. Unfortunately, I joined the company almost immediately after the Writers Guild of America (WGA) strike concluded, which left the film industry on shaky ground. Our goals as a company to re-build the VFX pipeline from the ground up were thwarted when a major project of ours was lost and multiple rounds of lay-offs ensued.

The awareness of unspoken quality of life issues in the video game industry that 2004 brought us was much needed. In my judgment, we’ve seen some improvements come out of that awareness, such as flexible working hours and smaller studios that attempt to treat their employees better. However, there is still a prevalence of crunch time across the video game industry. I see an opportunity here to spend more of my own energy inviting others in the industry to consider the awesome effects of what a happy, healthy, and energetic workforce would bring. In that regard, I’m doing my part as a studio owner to have reasonable working hours, minimize unhealthy levels of stress with projects, and respect our employees. My hope is that at the very least I will not perpetuate the cannibalism of creatives in our industry.

However, my attention right now and the purpose of these articles is to heal the damage that has already been done at a personal, individual level of health. My story continues past 2005 with more crunch periods, stress on relationships, and more health problems (including mental health). If any of what I’ve said so far resonates with you and you’ve come to accept as “normal” the sub-optimal ways you feel, think, and view life, then know that life can be better. Whatever you’ve been getting out of your body and mind under extended periods of stress comes at a cost. I wholeheartedly believe that whatever has been taken requires replenishing.

Over the past year and a half, I have focused on improving my own health. I have noticed that changes in the health of my body have resulted in improvements with the way I think and the performance of my mind. Another benefit is that with more energy, things that I wouldn’t have considered to do in the past come easier now, such as riding my bicycle to work. My hope for us all is to feel good in our bodies at a physical level, happy at an emotional level, and energized at a creative/intellectual level. Together, we’ll be able to make our industry a sustainable outlet for human creativity.

In the next article, I will go deeper into the health side of things as best as I have come to understand. I will share briefly a few anecdotes of my naivety as it relates to health and some of the things that I have done in an effort to make improvements. I will explain a bit about the PAT (pituary-adrenal-thyroid) axis and why it is so important to your overall well-being and your perception of well-being. If I find that there is more information to share or that this next article is too lengthy, then I may continue with further articles.

EDIT: I am a (hu)man, therefore I am imperfect. I have revised this article to remove references to what, unbeknownst to me, could be considered controversial — hair testing mineral analysis. I’ve become aware of arguments on both sides and would be happy to share with you the information, so that you can make your own educated decision. I have had my own positive experiences with hair testing/nutritional balancing, tailored supplements (as opposed to a catch-all or taking supplements that my body doesn’t need), a healthier diet, reduction of caffeine, drinking much more water than I thought was necessary, and a variety of other things. My goal is to raise awareness about health in our industry, provide one story as a reference point to others that may have had a similar experience, and offer some suggestions on how you may be able to improve your own health. My intention is not to back myself into a position of having to defend a specific approach to nutrition. What I have come to understand is that health and nutrition are not a simple thing. Simply Google ‘improve health’ and see all the various approaches and opinions that are returned. As to the Game Developer Performance Challenge that was originally linked here, I’d still like to challenge you to improve your own health if your day-to-day experience is sub-optimal.

A Fast, Type-safe & Minimal String Formatting Library, miniformat

Jaewon Jung — Sat, 13 Apr 2013 04:37:13 +0000

The inspiration for this came from tinyformat. Unfortunately it relies on the C++ iostream, basically std::cout, which is somewhat slow and definitely not everyone’s favorite. So I bit the bullet and got around to implement my own with minimal dependency and speed in mind.

Origin

I didn’t want to implement all the nitty-gritty of formatting a floating-point number to a string. Fortunately there was an awesome library called stringencoders, so I used it for the number-to-string part. The original miniformat came out like that(By the way, you can find the latest code of miniformat with improvements mentioned in this article in the new repository. I created the new one with git back-end to use SourceTree, which currently doesn’t support Mercurial on Windows). Its usage was like:

std::string str;
mini::format(str, "String: %1 Int: %0, Float: %(.3)2\n", 100, "JJ", 3.141592);

Originally it supported only std::string as an output type by default, although one can support other string types by specializing function templates in string_adaptor namespace. By the way, keep in mind the caveat that it’s not supporting all the features of printf format specifiers(for instance, no hexadecimal/octal integer support).

Improvements

Recently I saw Andrei Alexandrescu’s great post on C++ optimizations(video, slides) and decide to apply the lessons there (as for number-to-string conversion) to the formatting library and see how it fares. I also wanted to add a support for raw char arrays as an output type by default since a potential heap allocation cannot be an option in some situations.

Here’s the original code for converting 32bit unsigned integer to a string:

template <typename String>
void miniformat::detail::render(String& wstr, uint32_t value, int width, int)
{
    int beginIndex = string_adaptor::length(wstr);
    // Conversion. Number is reversed.
    do 
    {
        string_adaptor::append(wstr, 1, (char)(48 + (value % 10))); 
    }
    while (value /= 10);

    // Handle the 'width' parameter.
    int spaceCnt = width - (string_adaptor::length(wstr)-beginIndex);
    if(spaceCnt > 0)
        string_adaptor::append(wstr, spaceCnt, ' ');

    int endIndex = string_adaptor::length(wstr)-1;
    // Reverse string.
    strreverse(string_adaptor::at(wstr, beginIndex), string_adaptor::at(wstr, endIndex));
}

Pretty straightforward, other than the consideration for the ‘width’ parameter(the unnamed fourth parameter is a precision value, which is irrelevant for integers).

Here is the one after Andrei’s optimizations applied:

inline int miniformat::detail::digits10(uint32_t v)
{
    static const uint32_t P01 = 10;
    static const uint32_t P02 = 100;
    static const uint32_t P03 = 1000;
    static const uint32_t P04 = 10000;
    static const uint32_t P05 = 100000;
    static const uint32_t P06 = 1000000;
    static const uint32_t P07 = 10000000;
    static const uint32_t P08 = 100000000;
    static const uint32_t P09 = 1000000000;

    if(v < P01) return 1;
    if(v < P02) return 2;
    if(v < P03) return 3;
    if(v < P08)
    {
        if(v < P06)
        {
            if(v < P04) return 4;
            return 5 + (v >= P05);
        }
        return 7 + (v >= P07);
    }
    return 9 + (v >= P09);
}

template <typename String>
void miniformat::detail::render(String& wstr, uint32_t value, int width, int)
{
    static const char digits[201] =
        "0001020304050607080910111213141516171819"
        "2021222324252627282930313233343536373839"
        "4041424344454647484950515253545556575859"
        "6061626364656667686970717273747576777879"
        "8081828384858687888990919293949596979899";

    int next = string_adaptor::length(wstr);
    const int length = digits10(value);

    // Handle the 'width' parameter.
    int spaceCnt = width - length;
    if(spaceCnt > 0)
    {
        string_adaptor::append(wstr, spaceCnt, ' ');
        next += spaceCnt;
    }

    string_adaptor::append(wstr, length, '0');
    next += length-1;

    while(value >= 100)
    {
        const uint32_t i = (value % 100) * 2;
        value /= 100;
        *string_adaptor::at(wstr, next) = digits[i+1];
        *string_adaptor::at(wstr, next-1) = digits[i];
        next -= 2;
    }

    // Handle last 1-2 digits.
    if(value < 10)
    {
        *string_adaptor::at(wstr, next) = '0'+static_cast<char>(value);
    }
    else
    {
        const uint32_t i = static_cast<uint32_t>(value) * 2;
        *string_adaptor::at(wstr, next) = digits[i+1];
        *string_adaptor::at(wstr, next-1) = digits[i];
    }
}

It now consists of two passes(and two corresponding functions) instead of one, but due to strength reduction, array writes minimization and other optimizations from Andrei’s post, should perform faster than before.

Here are timings of converting some random integers into std::string, before & after this optimization:

Win32	Release/before	Release/after	Debug/before	Debug/after
uint32_t	0.85	0.57(1.5x)	4.64	2.44(1.9x)
int32_t	0.89	0.62(1.4x)	4.63	2.44(1.9x)
uint64_t	1.47	1.03(1.4x)	4.71	2.5(1.9x)
int64_t	1.49	1.08(1.4x)	4.71	2.5(1.9x)

x64	Release/before	Release/after	Debug/before	Debug/after
uint32_t	0.93	0.54(1.7x)	1.38	0.78(1.8x)
int32_t	0.94	0.56(1.7x)	1.39	0.77(1.8x)
uint64_t	0.93	0.57(1.6x)	1.43	0.79(1.8x)
int64_t	0.94	0.63(1.5x)	1.45	0.82(1.8x)

Numbers are in seconds.
Iteration count: 10,000,000 for Release, 1,000,000 for Debug
Compiler used: VS2012
One can find the code for this in benchmark2() function template of miniformat.cpp in the project repository.

One can observe 1.4x ~ 1.9x performance improvements overall. Not bad. You can also notice that the performance degrades very much in Debug(keep in mind that the iteration count differs between Release and Debug), especially in Win32. It seems mainly due to the fact that I used std::string as an output in this benchmark, which has much more overhead in Debug.

As I said, I had bolted on a support for raw char arrays recently, but it wasn’t very efficient because I had originally structured the code on the implicit assumption that getting a length of a string is a constant operation. This holds true for std::string and similars, but not for a raw char array. For that, one should pass the current length of the output string through functions. You can see that the render() code above calls string_adaptor::length() in the beginning to get the current length. That’s far from optimal. So I fixed it.

Here are some timings of a benchmark comparing sprintf vs. miniformat(for the both cases of outputting to std::string and to char array), before & after this fix:

Win32	Release/before	Release/after	Debug/before	Debug/after
Speed-up when outputting to std::string	4.42	4.20	0.26	0.24
Speed-up when outputting to char array	2.59	3.84	1.58	1.27

x64	Release/before	Release/after	Debug/before	Debug/after
Speed-up when outputting to std::string	3.81	3.81	1.13	1.06
Speed-up when outputting to char array	2.19	5.13	3.91	3.51

Numbers mean a speed-up factor of miniformat compared to sprintf.
Compiler used: VS2012
One can find the code for this in benchmark() function of miniformat.cpp in the project repository.

Before this fix, the speed-ups in the case of outputting to char array were less than the ones in the case of outputting to std::string because of the inefficiency mentioned above(4.42 vs 2.59 on Win32/Release, 3.81 vs 2.19 on x64/Release). With the fix, the speed-ups of outputting to char array has come close to or exceeded the ones of outputting to std::string. Again, you can see the inefficiency of std::string on Debug, which actually makes the speed-up factor of miniformat less than one on Win32/Debug.

Conclusion

I think miniformat is now in a decent state. As I said, it’s not exactly functionally equivalent to sprintf and some people don’t like possible code bloat that an extensive use of C++ template brings. I definitely won’t say benchmarks above are scientific. Still, it brings type-safety and speed-up of up to 4~5x (than sprintf) as just a header-only library with no dependency.

It’s not yet using C++11 stuff. In a related node, one should still append .c_str() when putting std::string as an input argument. I tried to do without it, but without move constructor stuff in C++11, it seemed very difficult to not introduce any copying. std::ref)(also, available in C++11) can do the trick, but it requires more typing than adding .c_str()! If anyone knows a solution for this without introducing a dependency, please enlighten me.

I welcome suggestions!

(I posted this article to my personal blog, too.)

Android piracy still sucks

Kyle-Kulyk — Thu, 11 Apr 2013 22:37:02 +0000

I don’t consider myself naive. I understand piracy is rampant; I’ve known this for years and when I decided that I’d form an indie studio two years ago I like to think that I did so with my eyes open. Deciding to switch careers and go indie wasn’t a decision that was made lightly and my family and I knew beforehand that it wasn’t going to be a walk in the park. The reward of creating even a modestly successful business would be worth it. I still believe that. Still, with the release of our new mobile puzzler, Vex Blocks, there’s an emotional, open-handed slap across the ear to be had when you realize that your release is being pirated at a rate of 20:1. On the Android market, a developer’s hard work is so devalued that 95% of gamers would rather pirate your game by the thousands than pay a dollar to help support what you’re doing and enable you to feed your family. Even though I was aware that piracy on Android was bad, it’s a soul shaking realization when it hits in relation to your own product and it left me to question if it’s even possible to make a living as an independent developer. All piracy sucks and the Android market isn’t alone in fighting this scourge, but what can be done and how has it impacted the games we make?

We launched the paid, full version of Vex Blocks on Google Play first for the simple fact that it was finished first. We knew the market for paid apps on Android was limited compared to others, but I don’t think we were really conscious of just how big a role piracy played. Our first release, Itzy3d, marked our first steps into the world of mobile game development. When we released Itzy3d, our plan was to have a full, paid version and then a free, lite version. The paid version of Itzy3d was all but ignored on the Android marketplace, as we suspected it would be while the lite version was downloaded considerably more than its iOS counterpart, by roughly 5:1. There was at least an appetite for our products on Android so skipping out on a Google Play release didn’t even occur to us with our follow-up title.

What we didn’t know with Itzy3d was actual numbers on how many copies of the game were installed on devices compared to how many we sold via Google Play. With Vex Blocks, we’re able to see how many copies are out in the wild compared to how many copies we’ve sold. These statistics don’t change our approach to Android releases. The impact is more of an emotional toll when you realize the bulk of gamers out there on Android you’re trying to reach with your product don’t have any interest in supporting that product, even if they use it.

Over the years I’ve heard plenty of arguments trying to justify why piracy isn’t really an issue for developers. The main one you always hear is, “Those people would have never bought the game in the first place. No money is lost.”

The idea that certain people just never buy games is ridiculous. Arguing that pirate gamers would just cease gaming if they couldn’t pirate is utter nonsense. Add to that the cost barrier for mobile gaming is generally so low, and it just boils down to pirates are cheap. Full stop. Pirates who can’t spend half the price of a large cup of coffee to support a game they want to play shouldn’t make excuses or try to rationalise their actions. They’re cheap and they’d rather open themselves up to malware infected copies found in the gutters of the internet than compensate developers for their work. Now I’m sympathetic to the fact that in some countries, due to government restrictions, the only way you can access television shows, music and, yes, mobile games is through illegal sites, but the bulk of pirated copies we’re seeing of our game aren’t coming from these countries.

Pirates will argue that nothing is stolen as the product is only copied, so it’s not even a crime, but of course the reality is every pirated copy of a game removes that person from the pool of potential customers and with 95% of your customers not willing to pay for your product, that doesn’t leave developers much room to work with on Android. So, the piracy ship has sailed and aside from becoming bitter old men who rant on blogs and shake their fist at kids who walk in front of their house too slowly, what can be done? Price competition isn’t a factor. The race to the bottom is already over and, surprisingly, no one came out the victor. Daring to even charge a dollar your game is in itself an invitation to pirate now. The ease with which pirates can copy a developers work has fundamentally changed the way developers cope and the end result is a diminished product for all.

There are numerous ways a developer can attempt to limit the impact of piracy issues. You could simply avoid developing for Android, but there is still a market consisting of legitimate Android users out there and it’s not that other platforms are immune to piracy. I love creating games for Android devices and I’m humbled by the support we’ve received from our players. I wouldn’t ignore the gamers who do support our products. Many of the other methods to deal with piracy involve lessening the experience for legitimate gamers on mobile (and on other platforms). DRM is one route. The general thinking is, the harder you make it to pirate your work, the more chance that some pirates might actually pay to play your game due to the effort involved in cracking or finding a pirated version. There’s certainly some validity to that approach but as we’ve witnessed with a few recent, high profile examples of digital rights management such as launch issues surrounding Diablo 3 and SimCity, this has the potential to backfire and hurt your paying users.

You can require mandatory sign-up as well, but this has a tendency to alienate those who simply want to buy a game and play, hassle free. That leaves the freemium method of releasing your game filled with upgrades, advertisements or both. It’s certainly the route that more mobile developers are choosing, including ourselves, but again this can lead to a lesser experience as advertisements throughout a game become tiresome quickly, and an over reliance of micro-transactions can also turn gamers away.

Alternately developers can choose to embrace piracy. There’s the chance that a happy pirate may promote your game simply through word of mouth. For indie developers, creating a great product is job one, but getting noticed is by far the real battle. The next great novel could be sitting in a pile of thousands of books but if no one picks it up to read it in the first place the author could very likely die in obscurity. Similarly, great games need to be played in the first place for anyone to discover them and if they’re played as a result of piracy and discovered in that fashion, the argument can be made that the pirates did the developer a service. Of course it’s only a service if the pirates tell enough gamers from the 5-10% of Android gamers who legitimately support developers – ideally before those developers are forced to close their doors.

But hey, why should developers going under matter to pirates? By that point they’ve already moved on to helping themselves to the next game.

Designing a game to behave in ways for which it was not designed

Alex Norton — Tue, 09 Apr 2013 01:04:03 +0000

Prologue

I’d like to start this article by pointing out that it is merely a documenting of my own personal experience and thoughts that led from it, and in no way is a treatise on what I think everyone should do. Please take it with a grain of salt.

Introduction

Emergent behaviour is something that has really surprised and fascinated me in the past with gaming communities. A particularly clean-cut example is Minecraft. Despite being designed as a sandbox game, the game’s community has grown to accomplish some incredible feats including a working computer as well as people making games within the game, which – to my mind – is just incredible. Given enough momentum, and given the right design choices in a game project, people will start to play a game in ways that it was not designed for – sometimes to very interesting effect.

So naturally, when this sort of thing started happening – albeit in a less grandiose fashion – within my own game’s community, I really found it astounding, and it led to some interesting revelations and thoughts for expansion, which I will go into later.

My own experience with emergent player behaviour

First, let me explain what happened within my own community to lead to these revelations, and to the eventual typing of this article. Some of you may know me for my main project, Malevolence: The Sword of Ahkranox. Among those who know and play the game, it has become infamous for its difficulty – which it was designed for – and the sheer effort required just to stay alive in the game when everything in it tries to kill you in the most creative way possible. This factor alone has led to its small, but dedicated fan-base to have a somewhat gleefully masochistic view towards playing, and many community requests for expansion to the game have revolved around this mentality, including making dungeons darker and more difficult to traverse, making traps deadlier, combat less forgiving and primarily the addition of a perma-death option to greatly multiply the sense of risk.

The emergent player behaviour that led from this started out fairly simply. People wanted to know how long they could survive in the game’s hostile environment, which only gets more difficult the longer you stay alive, and so, competitions arose to see who could live the longest. To my mind, this was inevitable and very much tied into the base competitive nature of the modern gamer.

However, it soon escalated from there with the addition of functionality to be able to “claim” locations as being the first to discover it. The game world in Malevolence is infinite, and everyone is in an exact copy of the same world, so people can travel as far as they like to discover new locations which are procedurally generated by the game, but still exist in everyone else’s copy.

The procedural nature of Malevolence means there’s
always something new to find. This makes players feel
a sense of ownership if they’re the first to find something.

This has led to certain players making it their entire goal to just wander the game world claiming as many places as they can get their digital hands on. One player has already found and claimed over 1100 unique locations, and the community as a whole has claimed over 10,000. It quickly became a mark of status on the forums to have claimed the most locations, and certain players would become minor celebrities for their efforts. I marvel to think what people will manage to accomplish once the quick-travel system is implemented later this month.

But this sort of digital achievement was not enough for the community, and – just recently – a forum member posted a challenge request for four others to join. He had found a particularly difficult level and wanted to challenge a handful of other players – on perma-death – to reach the end of the level alive (no small feat, given the difficulty). A section of the forum was dedicated to the exercise, and many hundreds of members watched along eagerly as the characters battled it out to survive (they documented their progress with screenshots and narrated videos). The entire ordeal became somewhat of a grand spectacle amongst fans of the game – a bit like reality TV in RPG form – and people even went so far as to place bets on how long people would last. Play strategies and tactics were discussed at length and equipment was debated over across many posts.

The popularity of this exercise, of course, led to more tournaments – of varying types – being organised and participated in, eventually leading to a thriving sub-culture within the community. This, in turn, led to the most recent piece of emergent player psychology which really caught my attention: Legendary Items.

Many games have legendary items in them – weapons or armour which have phenomenal properties which players actively seek out for their own – however, these items are only legendary because they have been written by a game designer to be legendary. What has been happening in my own community is that certain players are managing, through quite impressive play skill, to survive exceptional levels of in-game adversity, and when they have finished their challenges (to much community fanfare) they have been distributing their (procedurally generated – and thus, unique) equipment to the community through the game’s internal item sharing system. These armour items and weapons have become highly sought-after legendary items not because they have been written to be valuable, but because actual people have made them famous through very public shows of in-game heroism and skill. To the best of my knowledge, this is unique in video games (though I could be wrong – I don’t profess to know every game and its contents). Regardless, it is fascinating to see these purely digital items gain a real sense of perceived value not due to rarity in the game, or clever writing, but by actual achievement by real people.

I am sure that other bits of emergent player behaviour will appear over time, but thus far, the above is what I have experienced. This alone has led to some very interesting thoughts, which I felt that I should share.

Players congregate around well-known discovered places,
such as The View of Inca, because of its abundance of
resources. But the heavy visitor traffic simply begets more
visitor traffic as word spreads.

Designing a game to behave in ways for which it was not designed

It’s no new revelation that a strong community is a valuable asset to an indie game project, but how can one work to strengthen that community?

From my own experience, a community is often strengthened by a sense of ownership over the title. This – at least for me – was achieved by leaving the gameplay style quite open and not pigeon-holing people’s playing angle. An open world with procedurally generated content affords people the feeling that there is always something that only they could have found before – be it an item or location. This makes people feel that it is “theirs”. Naturally, this leads to a desire to show off their “acquisition” to help aid their social standing within the community.

This factor, and the emergent tournaments, has led me to begin serious planning to start implementing in-game functionality which will allow people to set up their own challenges for people to take on and handle for them, as well as showing off their character and gear that they’ve found, but it’s interesting how this could be applied to other projects as well.

If a game project is a solitary experience, then it will remain so in the player’s mind, however, if you add in a sense of unique accomplishment – separate to normal game goals which everyone is doing – and make something an experience for which they can know that no-one else has done, then that makes them want to share it and talk about it. This communication builds community bonding, and the community talks about it, leading to more people hearing about the project. People start discussing their achievements as though it was an actual thing that they, themselves, personally did on the weekend.

Conclusion

In an industry that has an ever-increasing indie push, the importance of marketing and publicity is becoming more and more relevant, and that onus is usually placed on the developer at the early stage, as marketing departments and publicists are expensive for independent developers. As such, anything that can help word spread about a project is a good thing. Talk is cheap, and with a little bit of clever design and forethought regarding emergent player behaviour, this can be turned into a very valuable part of your game’s functionality.

Thank you for taking the time to read about my personal experiences. I hope they’ve helped!

Addendum

After publishing this I had a developer friend make an interesting point which I thought I’d add to the end of this article.

One interesting observation made about “emergent” behaviour is that players latch onto it due to the very fact that what they’re doing is outside the realms of “what has been designed”.

Case in point:

Look at all the stunt videos for Codename Eagle, Battlefield etc… communities pop up with people breaking the game in marvellous ways. Now compare that to the videos from a game like Crackdown2, in which the “stunts” have been designed into the game structure, ie “park your car here and work out how to flip it to here”… (tip: There probably are none).

By writing that stuff into the game you’ve transferred ownership away from the players.

If Quake 2 came with a tutorial on corner-jumping, how many people would care?

A good developer needs to act as an amplifier… when people do cool stuff, they should use their power over the codebase to amplify the players’ cool stuff. Provide the Lego bricks for them to build with, but be careful not to try and show them what or how to build it.

This is, precisely, what I aim to do, given recent developments on my project, and what I will aim to do for future projects as well :)

Designing “10000000″ as a Free To Play Game

Raul Aliaga Diaz — Fri, 05 Apr 2013 22:38:18 +0000

1000000 [iOS, Android, Steam] is a puzzle-runner-rpg-like game developed by Eightyeight Games. If you haven’t played it, I seriously recommend to check it out since it’s awesome, and its story of development and success is as amazing and inspiring as the game itself.

The game in its PC/Mac port.

What’s also interesting is that the game allows itself to be designed as a Free to play game. It is a particularly good example because is indisputably a great game, is accessible for any platform you’d like to play it and we can use it as an example for this thought experiment of making it Free to play.

So here we will discuss a brief overview on how the game can be tweaked and expanded in its design to suit a Free to play strategy, and what would imply doing so, changing from a defined product to an ongoing service. From this point, we’ll assume you have played the game. If you feel like just reading, check here or here (not recommended).

Design changes and expansions

Using the Free to play triangle lens we can discuss three venues for monetization: Skips, Unlocks and Boosts. They are based on traditional “sim management” games, so we can adapt the triangle here to Content, Unlocks and Boosts.

Content: Pretty much all the game’s content is available behind locks achievable through gameplay and achievements, but the game could include more dungeons, sets of tiles, new achievements, episodic content or new heroes. Also, maybe just selling the music can be a good and simple trick.
Unlocks: All the locks on things that are accessible through “repairs”, the list of upgrades, access to dungeons, the alchemist potions. An additional element could be putting caps or limits on the amount of resources (wood, rock, coins and XP) that can be had at a time and unlock them through monetization/other resources. In 10000000, you can carry at most 4 items on a run: another potential source of revenue could be selling the ability to increase that limit so players can carry more.
Boosts: This is relatively straightforward, since when you’re playing you receive weapons, food, keys, orbs and scrolls that you can carry to the next run and act as boosts. All of these can be sold separately.

There are also two key areas for feature expansion. The first is a storage room that can hold any extra boosts the player found while running or purchased at the store. This causes the player to manage more effectively how they proceed on each run. These limits could be expanded through purchases.

Second, the Alchemist room enables a particular effect at the cost of a trade off. This should be changed to a boost fashion in which these alchemist effects can last only for one run, possibly with less strict tradeoffs and also stored in your storage room.

The Alchemist’s room

A third, possibly controversial expansion would be “crafting”. Retention is a huge issue in mobile, and something that could help to promote players to craft a plan to come back can be precisely the tradeoff of time versus money for alchemist effects, in which traditional game resources can be used so their “resource flow” closes. This is particularly important since puzzle/running games might have good retention but low frequency of use per week. Hence, this add-on would allow players to increase their chances to move through an engagement/monetization funnel in a better way.

However, the Triangle suggests to monetize with at most two of the methods, which is sound advice. Given that 10000000 isn’t a place to “enjoy watching an aquarium” such as DragonVale or Hay Day, the clever thing would be to stick with unlocks and boosts.

So that would be it, if we were only considering monetization aspects of Free to play games. As of now, we have discussed monetization venues for in-app purchases (which many paid games include), but free to play games must include social aspects within their designs. In a nutshell, non-paying players can be your best sources of new players (for another more general discussion, see here).

What social features can we include in this game? Endless runner games usually allow players to share their scores as a text on social networks such as Twitter or Facebook, but that’s rather limited given the unique aesthetic of the game, and specially its music. The guys at Nimblebit know this very well with their feature of sharing towers in Tiny Tower, planes on Pocket Planes and now video on their latest game: Nimble Quest, through the Everyplay service. See it at work here.

10000000 is an excellent game to share video replays to feel closer to the excitement of the game itself and sharing amazing feats that can be achieved in it. One additional content tweak would be increasing the animations and “pyrotechnic effects” of achieving chain reactions with the tiles in the running zone. When you’re playing and doing amazing feats, you feel great but you’re focused on the tiles, and when someone else is viewing, it would be nice to have something for other people to watch that is spectacular.

For example in Sonic Dash, they control the pacing of races and allow the player to do some “quick time swipes” for Sonic to breakdance in the air. This looks awesome and also wins some rings if done right.

If we want to go nuts, we can even discuss a room dedicated to store these glorious gameplay images and/or videos so the player herself can revisit them in the game or have it for her friends to see, but that’s a lot of additional work, development-wise.

Execution tradeoffs

We have discussed monetization and social aspects for the game to be considered a Free to play game, is that it? Well, not so much. Free to play games start when the game launches, and operate as a service, not a product. This cannot be stressed enough as it is easy to overlook in practical decisions and prioritizations how to approach the execution of these designs and handling the expectations associated to them.

So what does all this mean? In addition to the elements mentioned above they need to pass a check for game balance from a qualitative perspective and a quantitative one, hopefully informed through analytics -especially when the game is already live. Additional checks must be passed regarding UI, accessibility, and timing of the monetization elements. There’s a huge amount of games that hide away the options to pay more under too many screens, unclear menus or the wrong timing for monetization, making it annoying instead of convenient.

Adjusting the expectations is another important issue. It is fairly easy to fail at Free to play with a great game, even spectacularly. Many times this is because developers fail to see the service aspects and just throw in some IAP without too much thought about them, only to claim foul when they don’t get disgustingly rich overnight.

Then all the work needed with art, programming, balancing and testing needs to be estimated and considered to be split in several iterations that have different tradeoffs on different platforms when they go live. For example, monetization re-designs would allow the game to be set free and include more revenue venues but it would not achieve all of its strength without the social features. On the other hand, working the social features first would put the game in the eyes of more potential players that still would be stopped at a paywall if they’re not willing to pay anyway. However, the social features discussed here can be integrated in the game without making it Free to play at all and it could potentially be a good thing anyway.

In my particular opinion, I would thoroughly design all the aforementioned elements up to a point of actual estimation of the development work needed and consider the tradeoffs with the actual resources available and any other strategic considerations for the game. 10000000 is already present on several platforms and the best course of action would be to set up a minimal analytics service on all platforms to monitor the game’s performance and then decide how to go from there, possibly making it free to play on Android first (and possibly an Asian localization). Then moving these changes to all platforms including the social features, that can be developed for the game to scale once you have data and revenue to support such scaling.

Where do we go from here?

This was a general discussion to a deeper level of how to take an already great game to millions of players through the Free to play model, considering several aspects of the decisions involved.

For more detailed discussions on monetization and Free to play, I recommend to check:

To understand all the nuances involved in the design of a Free to play “casual” successful title, Candy Crush Saga:

The controversial decision made by Candy Crush developers of not to use a virtual currency in their game and going with direct purchase of boosts draws special attention. This was a risky and bold decision (albeit a good one given the game’s performance).

For expectation handling and pitfall avoidance: 7 Ways to Fail at Free-to-Play. On social aspects, a previous post of mine: Some social game design principles before launch. More resources in general, my previous post: Almost everything you need to know to enter mobile game development on 2013.

In summary, 10000000 is a great game, that includes within itself many options to make it a game that allow players themselves to put a better price on the value of the whole experience -a fun, convenient and social experience. When we analyze the tweaks and additions we can make to the game, we must take into consideration the costs and benefits of each improvement and how they can add value to the whole game incrementally. This will change from a finished, bounded product to an ongoing live service dedicated to its community of fans.

What is gimbal lock and why do we still have to worry about it?

Niklas Frykholm — Fri, 15 Mar 2013 14:17:48 +0000

If you have ever worked with rotations and Euler angles you are probably at least somewhat familiar with the phrase “gimbal lock”. But like many things concerning rotations, angles and spaces it can be tricky to visualize and get a good grasp of.

Sometimes it feels like every time I need to think about gimbal lock I have forgotten everything about it and have to go back the beginning and ask myself: OK, but what is it really that is happening?

Hopefully, this article will take care of that problem.

The Wikipedia page shows how gimbal lock can happen in a mechanical system. But it isn’t necessarily self-evident how this translates to the computer game world. In the computer there are no mechanical limitations, we can rotate an object however we like. How can anything be “locked”?

Euler angles

When we are using Euler angles, we represent an object’s orientation as three consecutive rotations around the object’s axes. We can choose the axes and the order in which we apply the rotations arbitrarily, and depending on what we choose we get different Euler representations. So XYZ is the Euler representation where the first angle rotates the object around its X-axis, the second around its (new) Y-axis and the third around its (new) Z-axis. YZX gives us a different representation. We can even have representations with repeated axes, such as XZX.

So if we want to talk about the “Euler angles” of an object, we really must also talk about what axes we are rotating around and in what order. Otherwise we have no idea at all what we are talking about. Unfortunately, many articles about Euler angles are pretty sloppy with this and throw around terms like yaw, pitch and roll as if they had completely well-defined and unambiguous meanings. I prefer to use more wordy, but descriptive names, such as euler_xyz[0] that unambiguously state the axis rotation order and the index of the angle we are talking about.

An object has three rotational degrees of freedom and it is quite easy to see that the three Euler angles for a particular axis order (XYZ) are enough to define any possible orientation of an object. Note though that the representation is not unique. There are many possible Euler angles that represent the same orientation. For example, adding 360 degrees to any of the three angles will give us a different representation that results in the same object orientation.

So “gimbal lock” doesn’t mean that there are rotations that can’t be expressed as Euler angles. We can express any rotation in Euler angle form. Given an object, we can convert its orientation to Euler angles, and from that orientation we can rotate the object however we like and convert the new orientation to other Euler angles.

So what exactly is it that is “locked”? It seems we can do whatever we like.

The “lock” in gimbal lock

The term “gimbal lock” comes from the mechanical world. If the problem had originated in the world of computers, it would probably have been called something less confusing, such as “Euler angle flip” or “coordinate singularity”.

Because, in the computer world, there is really nothing that gets “locked”. Instead, the problem is this: when the Euler angles have particular values, there are orientations that are very similar to the current orientation which can’t be achieved by just making small changes to the Euler angles. In particular, this happens when one of the angles is at 90 degrees, so that two rotation axes coincide.

So even though the orientations are “close” in the real world, they are not close in the Euler representation. In fact, at least one of the Euler angles will have to flip 180 degrees in order for us to represent the new orientation.

So one of the angles have to flip? What is the big deal? Can’t we just flip it and get on with our stuff?

We can, as long as the angles only represent instantaneous “snapshots” of the object’s orientation. However, if the angles represent key frames in an animation and we want to interpolate between those key frames we run into trouble. If one of the angles flips 180 degrees between two key frames and we interpolate between those values, we will see the the object animating through all those 180 degrees. In the viewport, we will see the object doing a “flip” or “roll” that shouldn’t be there.

Note that it is only the interpolation that creates this unwanted behavior. If we just displayed the actual key frames and didn’t interpolate between them — everything would look right. We could work in Euler angles as much as we liked and be as close to the gimbal lock position as we wanted and no-one would ever know.

So the only thing we need to fix to get rid of gimbal lock is the interpolation. If you have done any work with 3D graphics you probably already know the answer — to use quaternions instead of Euler angles to represent angles. Quaternions don’t have the weird singularity points that Euler angles have and we can interpolate between any keyframes by just lerping the quaternions.

It doesn’t matter if the animation package is using Euler angles internally, as long as we convert everything to quaternions before we do the interpolation. Note that interpolation in quaternion space is not the same as interpolation in Euler space though, so to get as close as possible to what the animator intended, we probably want to sample the animation at our target frame rate and generate our quaternion key frames from those samples, rather than directly converting the animator’s key frames (which may be further apart).

Well, there is one caveat actually. If we have more than 180 degrees of rotation in a single frame we can’t represent that nicely with quaternions. Quaternions always lerp the shortest path between two orientations and you can’t represent several “laps” of rotation with quaternions as you can do with Euler angles (by setting one of the angles to 9000 degrees, for example). But you can fix that by sampling at a higher frame rate if you need to represent really fast rotations with quaternions.

So with that we can say good bye and good riddance to Euler angles and never have to worry about their sorry gimbal locking asses ever again.

Or so you may think…

The return of gimbal lock

I certainly thought so, until I started working on the new cutscene animation system for our level editor.

You see, animators really like to work with curves. They like to see a visual representation of what the animation will do to an object over time with key points that can be moved and handles that can be adjusted to change the slope of the curve.

Curves! Animators love them!

Quaternions are great for interpolation, but they are no good for curve editing.

Sure, you could probably draw some curves that represented a quaternion (the laziest thing would be to just draw the x, y and z components of the quaternion), but those curves wouldn’t mean anything to an animator, the way the Euler angle curves do. They wouldn’t be able to do anything with them.

So, animators want curves with keyframe interpolation. Curves need Euler angles. But what happens when we mix Euler angles with keyframe interpolation? Presto! Our old friend the gimbal lock is back again! Haven’t we missed him.

That’s it. We’re stuck. Gimbal lock is here to stay and the animators will just have to work around it.

And we have to add support for all the usual tricks and workarounds that animators use to get around gimbal lock, such as changing the axis order (from XYZ, to XYZ, XZX or another of the twelve possible permutations), converting to quaternion and back again, applying an “Euler filter”, etc.

But who said this game engine gig should be easy?

This has also been posted to The Bitsquid blog.

Launch Day 2.0

Kyle-Kulyk — Tue, 12 Mar 2013 22:48:40 +0000

I didn’t sleep well the night before our Sunday launch of Vex Blocks. Despite the weeks we’ve had the game in the hands of testers, a new bug had come to light late Saturday evening. While not game breaking it still needed to be addressed. I stayed up working on the problem until solved. Chat messages, frantic on my part, went back and forth between myself and my partner until between the two of us we were able smooth things out. I then prepped the build for the next morning’s launch and finally after a long day allowed myself to push back away from my desk just after 1am. I tried and failed to sleep after that.

The late night adrenalin rush of fixing a bug on the eve of our self-imposed deadline had fired up my brain and my thoughts erratically leap from point to point like popcorn popping while anxiety knotted up in my stomach and settled in for a winter’s nap. I stared at my bedside table for an undetermined length of time before I eventually drifted off into a fitful sleep.

At some point during the night my wife woke up to comfort our baby girl, Hanna and had fallen asleep in her room. My day started when my wife came back to bed and informed me that our 3-year-old, Jake, was awake and calling for Daddy. I squinted, noticed the sun was up so there was no chance at another hour of sleep and rolled out of bed just as Jake hit the living room and started honking the horn of an obnoxiously noisy pink riding car meant for his sister. I settled him down with a cup of a fruit/veggie juice I have yet the had the courage to taste and turned on “The Cat in the Hat knows a lot about that” before heading to my office to launch our second title.

Launch day this time was different. With our first title, our core group hunkered down in my home office in a marathon of testing the ultimately cumulated with “Well. That’s it. Let’s launch.” Someone suggested we should have a drink so we did toasting each other and then that was it. Game launched. The next day I sat, staring at my computer screen knowing I needed to “market” somehow but not having a clue even where to start. This time out there was no empty void of “What the hell do I do now” waiting for me after launch. There’s no one else in the room. No one to raise a toast to. The artwork was finished months ago by very talented student artists. The testers I haven’t actually seen face to face in who knows how long. People complete their assigned tasks and then scatter leaving only us, the two founders of our indie studio, each waking up with their respective families on a cold, Canadian Sunday morning. There was still a long list of of objectives that needed to be completed to launch Vex on other platforms, in other app stores. There’s another team working on our third title that needs checking in on. There’s communications that have been piling up, unanswered while we pushed to complete. The gaping maw of uncertainty that greeted me after the first game released didn’t exist this time out. I know what needs to be done and we can’t afford to slow down.

So while the Cat in the Hat explains to my 3 year old the benefits to singing underwater with a whale I sat down to release our new game. The sun tries to poke through the clouds outside my office window but it just can’t decide whether it wants to clear up or add to the snow on my back deck. I noticed the time and remembered that Daylight Savings had struck again. I could almost feel that hour suddenly sucked away from me like a voodoo spell setting in as I lamented the time shift. I shook the sudden tiredness off and a quick check of my emails included a brief message from our other developer, my brother-in-law Will. No doubt he had a late night as well. If he wasn’t awake due to work, I’m sure his sleep would have been interrupted by their first child only weeks old, waking and bellowing his displeasure. I took the few cosmetic changes to Vex Blocks that Will had pushed up to our server, installed the new build and then uploaded the apk file to Google Play. Then I hit “Publish”.

That was it. I leaned back in my chair and despite being told by an online message that it could take hours to populate to Google’s servers, I punched in the url just to see if it was visible. It wasn’t. I went through my email and then checked a few links to press releases that were scheduled to go live that morning to herald the Vex Blocks release. I clicked refresh on the Google Play site again. Still nothing. I flipped back and forth between press releases, finding the announcements somehow comforting and real, like the launch of the game after ten months wasn’t just a dream. I’d then refresh the Google Play link. I don’t know how many times I did this before the baby monitor in the living room announced that Hanna was awake. I fired off a message to Will for whenever his sleep deprived old bones made his way into his office that morning, informing him that the deed was done.

The links to the game on our Facebook and web pages were live. The press releases were out. The game was published. I went to rescue my daughter from the confines of her crib. She smiled her huge, baby smile when I entered her room, just like she always does and I took her into the living room where she proceeded to make short work of some Cheerios I placed out on the coffee table for her to keep her occupied. My son jumped up on the couch beside me and we talked and watched TV, letting my wife sleep in a bit. Little Einsteins was on now. The rocket ship apparently runs off leg pats, so we patted our laps faster and faster to rev up Rocket as we were instructed, raised our hands as high as we could and announced together “Blast off!” Hanna turned from where she was standing, leaning against our ottoman, swaying a bit on her unsteady little legs. She giggled at us.

It was a good day.

30 Weeks of Game Development

Forrest Smith — Fri, 08 Mar 2013 12:00:24 +0000

Outland Games is an iOS game I’ve been working on at Uber Entertainment since late last summer. We just released the game to the app store this week (iTunes – $0.99). This post is the first in a short series on it’s development.

Evolution

On every project I’ve ever shipped I wished I could have chronicled the game’s production from start to finish. It’s a really interesting process that the public never gets to see.

The scale and starting point of Outland Games enabled us to record progress in a way that I’ve never been able to do before. We used fraps to capture a video at the end of almost every week of development [1]. Eka took those videos and edited them into the following video.

We also took that video and compressed it to a single image.

That’s what game development looks like – warts and all. I’m not sure I’ve ever seen a video put together like this one. Certainly nothing so raw.

Scope

The team behind Outland Games was small and consisted of just five primary team members.

Forrest Smith – coding/design – 7 months
Chandana Ekanayake – art/ui/design/trailer – 4 months
Aung Zaw Oo – animation/modeling – 3 months
Ben Golus – effects/misc awesome – 3 months
Howard Mostrom – audio – 2 months

Unity3d is all the rage these days but this game was made using a custom engine. It’s actually the same codebase as Planetary Annihilation. I’ll cover that in more detail in another post.

The game never had any official milestones. I think it can be broken down into the following stages.

Prototype (weeks 1-6)

When we started the goal was to create an endless runner using internal tech and the Monday Night Combat IP. What that meant precisely we didn’t know.

After implementing and playing with various jump mechanics we settled into the double jump you see in the final version. The remarkably basic grey and red block based gameplay from the very first week was actually pretty fun!

We also tested various art creation strategies. Our initial plan for character animation was hand drawn sprite flipbooks. This idea was quickly scrapped in favor of a 3dsmax skeleton with attached sprites. (Which would later become full 3d meshes.)

Much experimentation was done in regards to physics, dash mechanics, level design, and art style. Howard wanted audio hook ups from week 1 and insisted it wasn’t a “real game” until it had sound. Unfortunately sound kept getting pushed back and didn’t get configured until the end of the prototype phase. Sorry Howard!

Europe (weeks 7-8)

During September I went to Germany and Amsterdam on a two week trip. At this point I was still the only full time member on Outland Games so development was temporarily stalled. Beer consumption levels however reached an all-time high.

Tech (weeks 9-13)

Because we used early stage internal tech there were a lot of feature gaps to fill. Adding build tool support for iOS. Fleshing out the bare bones UI system. Defining and/or implementing export/import processes for compressed textures, meshes, and animations. Plus various game specific systems.

Much of this work didn’t actually show up on screen. It’s easy to work for weeks on systems that are invisible to the eye. This can be super frustrating at times. Particularly when attempting to record weekly videos.

Content (weeks 14-25)

The basic components of functionality were finally in place at the start of week 14 despite the game looking quite meager. If you exclude the non-productive Thanksgiving week and out town Christmas week the content phase was ten weeks.

By week 25 the game had two environments, animated enemies, coins, pickups, profile save/load, upgrade progression, full UI including store, layered music, voice overs, and the whole shebang. Although not quite ready for primetime the game was functional and, more importantly, fun.

Ship (weeks 26-30)

The final push was focused on one clear objective – shipping.

I was tempted to call this phase polish but that’s not quite right. A lot of polish did go in. Final animations. Carefully tuned audio. Continuously tweaked level design. But too many features weren’t finished to call it just polish. GameCenter leaderboards and app store IAP weren’t even started until the very end. Even the in-game intro cinematic was a last weekend addition.

After three submissions the game was finally approved. We gave a great sigh of relief and consumed much scotch.

History Keeping

The hardest part of capturing footage is having the willpower to record every week. It’s easy to push back the capture because a major feature is almost done. Game dev is often an emotional rollercoaster and when you’re in a low it’s hard to muster the motivation to click record.

I actually stopped making the videos around week 16. Every week felt like too little progress was being made to justify a video. After going gold and watching the early videos I was hit with a wave of inspiration so I synced perforce week by week to re-build and capture [2].

In hindsight progress was much more visible than I realized! It’s easy to feel perpetually “one month away” which it turn feels like no progress is being made. The truth is that you are getting a lot done even if it’s taking longer than initially planned [3].

All I can say is to stick with it. Capture every single week. Even if you are almost done with some cool new thing. When the project is shipped and you look back you may be surprised at how much was getting done.

Conclusion

That’s a brief review of the development of Outland Games. We’re super happy with the final product and thrilled to finally release an evolution video. It’s a unique insight to the dev process that I hope folks find informative and entertaining.

Footnotes

[1] Despite being an iOS game 95% of development was done on PC.
[2] synced? sync’d? sunk? sank? These all feel wrong.
[3] Game dev always takes longer than initially planned.

A Bug in Object Replication and Message Reordering

Niklas Frykholm — Fri, 01 Mar 2013 16:13:59 +0000

The Bitsquid network system supports a peer-to-peer model with migration of network objects — i.e., changing the owner of a network object from one peer to another. This recently lead to an rare race condition.

To understand this bug you must first understand a little bit about how our network system works.

Background

The entire network model is based on an packet delivery system (on top of UDP) that provides ACKs for unreliable packets as well as a reliable (and ordered) packet stream between any two network endpoints. At the next layer we have implemented a remote-procedure-call service for Lua as well as an object replication system.

Games can use these services however they like, but our recommendation is to do as much as possible with the object replication system and as little as possible with RPC calls, since using explicit RPC messages tends to require more bandwidth and be more error prone.

The network be run in both client-server and peer-to-peer mode. The only difference is that in client-server mode, the server relays all messages (clients never talk directly to each other) and owns most of the network objects. (Clients can own objects in client-server mode, in that case the changes to the objects are relayed by the server.)

Objects are replicated with a message stream that looks something like this:

A: CREATE [wait for ack] UPDATE_1 UPDATE_2 ... UPDATE_n DESTROY

Here, A (the owner of the object) first sends a reliable message that creates the object. When it has received an ACK for that message, it starts sending updates, informing the other players about changes to the object. (By monitoring ACKs, it knows which changes the other players have received, so it only sends updates when necessary and it will resend updates if the messages are lost.) Finally, at some future point, the object is destroyed, through another reliable message.

The UPDATE messages are sent on the unreliable stream (for maximum performance), so they can potentially arrive before CREATE or after DELETE. But this is not a problem, because we simply ignore UPDATE messages that arrive out of order.

This approach ensures that everybody that participates in the game session will see the same set of objects with the same properties (at least eventually, keeping in mind that messages can be delayed).

Migration

Migration complicates this picture somewhat.

Migrating a network object means changing the owner of the object from one peer to another. There are a number of reasons why you might want to do that. First, if a player drops out of the game, the objects owned by that player may need to be taken over by somebody else. Second, in a peer-to-peer game we may want to load balance, so that each peer is managing about the same amount of objects. Finally, sometimes a particular player is interacting directly with a particular object (picking up a rock, etc). It can then be beneficial to make that player owner of the object, so that the interaction is not affected by network latency.

In our network, migration is implemented with a reliable MIGRATION message that tells everybody in the session about the object’s new owner. The migration message is always sent by a special peer, the HOST of the game session. (To ensure that peers do not compete for the ownership of an object.)

So if we look at a message stream with migration involved, it looks something like this:

   A:  C Ua Ua Ua Ua Ua
HOST:                    M_ab
   B:                          Ub Ub Ub Ub Ub Ub Ub Ub D

If you are an experienced network programmer you should start to smell trouble at this point.

The problem is that while the message system provides an ordered stream of messages between any two endpoints, there is no ordering of messages between different endpoints.

Consider an additional network peer X. There is an ordered stream of messages A → X. There is also an ordered stream of messages B → X. But there is no guaranteed ordering between the messages sent from A and the messages sent from B and HOST. So, suppose the messages from A → X are delayed. Then X could see the following message stream:

M_ab Ub Ub Ub D C Ua Ua Ua

So X gets a request to migrate the object before it has been created. And the creation message arrives after DELETE. In other words, a complete mess.

To be sure, this only happens if the object gets migrated really close to being created or deleted and if there are asymmetric network delays on top of that. But of course, it always happens to someone.

The Fix

There are many possible ways of fixing this. Here are some:

We could impose a global message ordering. We could make sure that the reliable message streams are globally ordered to prevent “paradoxes” of this kind. I.e., if HOST sends M_ab after receiving C, no peer should receive M_ab before C. Unfortunately, this is not as easy as it sounds. For example, what if A dies before it has sent C to X? In that case, that failed delivery will also block the channels HOST → X and B → X, since they are not allowed to deliver any messages before X has received C.
We could use a migration handshake. We could do some kind of handshake procedure to make sure that everybody has received M_ab, before B takes over ownership. But this would require a lot of extra messages and temporarily put the object in limbo.
We could fix the ACKs. We could make it so that X doesn’t ACK M_ab until C has arrived, thus forcing HOST to keep resending it, until we are ready to receive it. This would work, but would require us to implement ACKing of individual messages. Currently, we just ACK an entire UDP packet (containing many messages) on reception, which is simpler and more performant.
We could create an internal message queue. We could queue up migration, create and delete messages in some sort of internal queue if they arrive out of order and try to fix things up later. This is a truly horrible “solution” that increases code complexity and is likely to cause lots of confusing bugs in the future.

All these solutions are probably workable, but they all have the drawback of increasing complexity. And I really don’t like to increase the complexity of network code. Reasoning about network code is hard enough as it is, we should always strive for the simplest solution possible.

So, instead, the first thing I did was to simplify the problem by eliminating the host from the equation. I simply let the new owner send out the migration message instead of the host:

   A:  C Ua Ua Ua Ua Ua
   B:                    M_ab Ub Ub Ub Ub Ub Ub Ub Ub D

This is already a lot better. Now we only have two parties to worry about (apart from X), instead of three.

We still want the host to be in charge of migration. Otherwise we run into tricky problems of what should happen if several peers try to assume ownership of an object at the same time. So we let the host initiate the migration by sending a message to the new owner (B). Then, B is responsible for notifying everybody else about this.

With this approach, we can use the same “wait for ack” trick that we used during creation to make sure that B doesn’t send any updates to peers that haven’t acked the migration:

   A:  C [wait] Ua Ua Ua Ua Ua
   B:                            M_ab [wait] Ub Ub Ub Ub Ub Ub Ub Ub D

We still haven’t completely solved the problem, X can still see weird message orderings such as:

M_ab   C   D
M_ab   D   C

But this won’t be a problem as long as we do two things:

We let MIGRATE act as a CREATE message, if we get MIGRATE for an object that doesn’t exist.
We ignore “old” CREATE messages. (The C that arrives after M.)

To be able to distinguish old messages I introduced a migration counter. This is just a number that starts at zero when the object is created and is increased (by HOST) every time the object is migrated.

We tag all CREATE, DESTROY and MIGRATE messages with the migration counter and simply ignore “old” messages. With this approach, the message streams will look like this:

   A:  C_0 [wait] Ua Ua Ua Ua Ua
   B:                             M_ab_1 [wait] Ub Ub Ub Ub Ub Ub Ub Ub D_1

We can now verify that all possible message orderings that X can see work correctly:

C_0      M_ab_1  D_1  -- ok, the expected order
M_ab_1   C_0     D_1  -- ok, M_ab_1 creates the object with migration counter 1 and C_0 is ignored
M_ab_1   D_1     C_0  -- ok, M_ab_1 creates the object with migration counter 1 and C_0 is ignored

The system works equally well if there are multiple migration steps:

   A:  C_0 [wait] Ua Ua 
   B:                   M_ab_1 [wait] Ub Ub Ub
   C:                                            M_bc_2 [wait] Uc Uc Uc D_2

No matter in which order the messages arrive we will end up in the correct state.

This has also been posted to The Bitsquid blog.

Latency Mitigation Strategies

John-Carmack — Fri, 22 Feb 2013 17:25:46 +0000

Abstract

Virtual reality (VR) is one of the most demanding human-in-the-loop applications from a latency standpoint. The latency between the physical movement of a user’s head and updated photons from a head mounted display reaching their eyes is one of the most critical factors in providing a high quality experience.

Human sensory systems can detect very small relative delays in parts of the visual or, especially, audio fields, but when absolute delays are below approximately 20 milliseconds they are generally imperceptible. Interactive 3D systems today typically have latencies that are several times that figure, but alternate configurations of the same hardware components can allow that target to be reached.

A discussion of the sources of latency throughout a system follows, along with techniques for reducing the latency in the processing done on the host system.

Introduction

Updating the imagery in a head mounted display (HMD) based on a head tracking sensor is a subtly different challenge than most human / computer interactions. With a conventional mouse or game controller, the user is consciously manipulating an interface to complete a task, while the goal of virtual reality is to have the experience accepted at an unconscious level.

Users can adapt to control systems with a significant amount of latency and still perform challenging tasks or enjoy a game; many thousands of people enjoyed playing early network games, even with 400+ milliseconds of latency between pressing a key and seeing a response on screen.

If large amounts of latency are present in the VR system, users may still be able to perform tasks, but it will be by the much less rewarding means of using their head as a controller, rather than accepting that their head is naturally moving around in a stable virtual world. Perceiving latency in the response to head motion is also one of the primary causes of simulator sickness. Other technical factors that affect the quality of a VR experience, like head tracking accuracy and precision, may interact with the perception of latency, or, like display resolution and color depth, be largely orthogonal to it.

A total system latency of 50 milliseconds will feel responsive, but still subtly lagging. One of the easiest ways to see the effects of latency in a head mounted display is to roll your head side to side along the view vector while looking at a clear vertical edge. Latency will show up as an apparent tilting of the vertical line with the head motion; the view feels “dragged along” with the head motion. When the latency is low enough, the virtual world convincingly feels like you are simply rotating your view of a stable world.

Extrapolation of sensor data can be used to mitigate some system latency, but even with a sophisticated model of the motion of the human head, there will be artifacts as movements are initiated and changed. It is always better to not have a problem than to mitigate it, so true latency reduction should be aggressively pursued, leaving extrapolation to smooth out sensor jitter issues and perform only a small amount of prediction.

Data collection

It is not usually possible to introspectively measure the complete system latency of a VR system, because the sensors and display devices external to the host processor make significant contributions to the total latency. An effective technique is to record high speed video that simultaneously captures the initiating physical motion and the eventual display update. The system latency can then be determined by single stepping the video and counting the number of video frames between the two events.

In most cases there will be a significant jitter in the resulting timings due to aliasing between sensor rates, display rates, and camera rates, but conventional applications tend to display total latencies in the dozens of 240 fps video frames.

On an unloaded Windows 7 system with the compositing Aero desktop interface disabled, a gaming mouse dragging a window displayed on a 180 hz CRT monitor can show a response on screen in the same 240 fps video frame that the mouse was seen to first move, demonstrating an end to end latency below four milliseconds. Many systems need to cooperate for this to happen: The mouse updates 500 times a second, with no filtering or buffering. The operating system immediately processes the update, and immediately performs GPU accelerated rendering directly to the framebuffer without any page flipping or buffering. The display accepts the video signal with no buffering or processing, and the screen phosphors begin emitting new photons within microseconds.

In a typical VR system, many things go far less optimally, sometimes resulting in end to end latencies of over 100 milliseconds.

Sensors

Detecting a physical action can be as simple as a watching a circuit close for a button press, or as complex as analyzing a live video feed to infer position and orientation.

In the old days, executing an IO port input instruction could directly trigger an analog to digital conversion on an ISA bus adapter card, giving a latency on the order of a microsecond and no sampling jitter issues. Today, sensors are systems unto themselves, and may have internal pipelines and queues that need to be traversed before the information is even put on the USB serial bus to be transmitted to the host.

Analog sensors have an inherent tension between random noise and sensor bandwidth, and some combination of analog and digital filtering is usually done on a signal before returning it. Sometimes this filtering is excessive, which can contribute significant latency and remove subtle motions completely.

Communication bandwidth delay on older serial ports or wireless links can be significant in some cases. If the sensor messages occupy the full bandwidth of a communication channel, latency equal to the repeat time of the sensor is added simply for transferring the message. Video data streams can stress even modern wired links, which may encourage the use of data compression, which usually adds another full frame of latency if not explicitly implemented in a pipelined manner.

Filtering and communication are constant delays, but the discretely packetized nature of most sensor updates introduces a variable latency, or “jitter” as the sensor data is used for a video frame rate that differs from the sensor frame rate. This latency ranges from close to zero if the sensor packet arrived just before it was queried, up to the repeat time for sensor messages. Most USB HID devices update at 125 samples per second, giving a jitter of up to 8 milliseconds, but it is possible to receive 1000 updates a second from some USB hardware. The operating system may impose an additional random delay of up to a couple milliseconds between the arrival of a message and a user mode application getting the chance to process it, even on an unloaded system.

Displays

On old CRT displays, the voltage coming out of the video card directly modulated the voltage of the electron gun, which caused the screen phosphors to begin emitting photons a few microseconds after a pixel was read from the frame buffer memory.

Early LCDs were notorious for “ghosting” during scrolling or animation, still showing traces of old images many tens of milliseconds after the image was changed, but significant progress has been made in the last two decades. The transition times for LCD pixels vary based on the start and end values being transitioned between, but a good panel today will have a switching time around ten milliseconds, and optimized displays for active 3D and gaming can have switching times less than half that.

Modern displays are also expected to perform a wide variety of processing on the incoming signal before they change the actual display elements. A typical Full HD display today will accept 720p or interlaced composite signals and convert them to the 1920×1080 physical pixels. 24 fps movie footage will be converted to 60 fps refresh rates. Stereoscopic input may be converted from side-by-side, top-down, or other formats to frame sequential for active displays, or interlaced for passive displays. Content protection may be applied. Many consumer oriented displays have started applying motion interpolation and other sophisticated algorithms that require multiple frames of buffering.

Some of these processing tasks could be handled by only buffering a single scan line, but some of them fundamentally need one or more full frames of buffering, and display vendors have tended to implement the general case without optimizing for the cases that could be done with low or no delay. Some consumer displays wind up buffering three or more frames internally, resulting in 50 milliseconds of latency even when the input data could have been fed directly into the display matrix.

Some less common display technologies have speed advantages over LCD panels; OLED pixels can have switching times well under a millisecond, and laser displays are as instantaneous as CRTs.

A subtle latency point is that most displays present an image incrementally as it is scanned out from the computer, which has the effect that the bottom of the screen changes 16 milliseconds later than the top of the screen on a 60 fps display. This is rarely a problem on a static display, but on a head mounted display it can cause the world to appear to shear left and right, or “waggle” as the head is rotated, because the source image was generated for an instant in time, but different parts are presented at different times. This effect is usually masked by switching times on LCD HMDs, but it is obvious with fast OLED HMDs.

Host processing

The classic processing model for a game or VR application is:

Read user input -> run simulation -> issue rendering commands -> graphics drawing -> wait for vsync -> scanout

I = Input sampling and dependent calculation
S = simulation / game execution
R = rendering engine
G = GPU drawing time
V = video scanout time

All latencies are based on a frame time of roughly 16 milliseconds, a progressively scanned display, and zero sensor and pixel latency.

If the performance demands of the application are well below what the system can provide, a straightforward implementation with no parallel overlap will usually provide fairly good latency values. However, if running synchronized to the video refresh, the minimum latency will still be 16 ms even if the system is infinitely fast. This rate feels good for most eye-hand tasks, but it is still a perceptible lag that can be felt in a head mounted display, or in the responsiveness of a mouse cursor.

Ample performance, vsync:
ISRG------------|VVVVVVVVVVVVVVVV|
.................. latency 16 – 32 milliseconds

Running without vsync on a very fast system will deliver better latency, but only over a fraction of the screen, and with visible tear lines. The impact of the tear lines are related to the disparity between the two frames that are being torn between, and the amount of time that the tear lines are visible. Tear lines look worse on a continuously illuminated LCD than on a CRT or laser projector, and worse on a 60 fps display than a 120 fps display. Somewhat counteracting that, slow switching LCD panels blur the impact of the tear line relative to the faster displays.

If enough frames were rendered such that each scan line had a unique image, the effect would be of a “rolling shutter”, rather than visible tear lines, and the image would feel continuous. Unfortunately, even rendering 1000 frames a second, giving approximately 15 bands on screen separated by tear lines, is still quite objectionable on fast switching displays, and few scenes are capable of being rendered at that rate, let alone 60x higher for a true rolling shutter on a 1080P display.

Ample performance, unsynchronized:
ISRG
VVVVV
..... latency 5 – 8 milliseconds at ~200 frames per second

In most cases, performance is a constant point of concern, and a parallel pipelined architecture is adopted to allow multiple processors to work in parallel instead of sequentially. Large command buffers on GPUs can buffer an entire frame of drawing commands, which allows them to overlap the work on the CPU, which generally gives a significant frame rate boost at the expense of added latency.

CPU:ISSSSSRRRRRR----|
GPU:                |GGGGGGGGGGG----|
VID:                |               |VVVVVVVVVVVVVVVV|
    .................................. latency 32 – 48 milliseconds

When the CPU load for the simulation and rendering no longer fit in a single frame, multiple CPU cores can be used in parallel to produce more frames. It is possible to reduce frame execution time without increasing latency in some cases, but the natural split of simulation and rendering has often been used to allow effective pipeline parallel operation. Work queue approaches buffered for maximum overlap can cause an additional frame of latency if they are on the critical user responsiveness path.

CPU1:ISSSSSSSS-------|
CPU2:                |RRRRRRRRR-------|
GPU :                |                |GGGGGGGGGG------|
VID :                |                |                |VVVVVVVVVVVVVVVV|
     .................................................... latency 48 – 64 milliseconds

Even if an application is running at a perfectly smooth 60 fps, it can still have host latencies of over 50 milliseconds, and an application targeting 30 fps could have twice that. Sensor and display latencies can add significant additional amounts on top of that, so the goal of 20 milliseconds motion-to-photons latency is challenging to achieve.

Latency Reduction Strategies

Prevent GPU buffering

The drive to win frame rate benchmark wars has led driver writers to aggressively buffer drawing commands, and there have even been cases where drivers ignored explicit calls to glFinish() in the name of improved “performance”. Today’s fence primitives do appear to be reliably observed for drawing primitives, but the semantics of buffer swaps are still worryingly imprecise. A recommended sequence of commands to synchronize with the vertical retrace and idle the GPU is:

SwapBuffers();
DrawTinyPrimitive();
InsertGPUFence();
BlockUntilFenceIsReached();

While this should always prevent excessive command buffering on any conformant driver, it could conceivably fail to provide an accurate vertical sync timing point if the driver was transparently implementing triple buffering.

To minimize the performance impact of synchronizing with the GPU, it is important to have sufficient work ready to send to the GPU immediately after the synchronization is performed. The details of exactly when the GPU can begin executing commands are platform specific, but execution can be explicitly kicked off with glFlush() or equivalent calls. If the code issuing drawing commands does not proceed fast enough, the GPU may complete all the work and go idle with a “pipeline bubble”. Because the CPU time to issue a drawing command may have little relation to the GPU time required to draw it, these pipeline bubbles may cause the GPU to take noticeably longer to draw the frame than if it were completely buffered. Ordering the drawing so that larger and slower operations happen first will provide a cushion, as will pushing as much preparatory work as possible before the synchronization point.

Run GPU with minimal buffering:
CPU1:ISSSSSSSS-------|
CPU2:                |RRRRRRRRR-------|
GPU :                |-GGGGGGGGGG-----|
VID :                |                |VVVVVVVVVVVVVVVV|
     ................................... latency 32 – 48 milliseconds

Tile based renderers, as are found in most mobile devices, inherently require a full scene of command buffering before they can generate their first tile of pixels, so synchronizing before issuing any commands will destroy far more overlap. In a modern rendering engine there may be multiple scene renders for each frame to handle shadows, reflections, and other effects, but increased latency is still a fundamental drawback of the technology.

High end, multiple GPU systems today are usually configured for AFR, or Alternate Frame Rendering, where each GPU is allowed to take twice as long to render a single frame, but the overall frame rate is maintained because there are two GPUs producing frames

Alternate Frame Rendering dual GPU:
CPU1:IOSSSSSSS-------|IOSSSSSSS-------|
CPU2:                |RRRRRRRRR-------|RRRRRRRRR-------|
GPU1:                | GGGGGGGGGGGGGGGGGGGGGGGG--------|
GPU2:                |                | GGGGGGGGGGGGGGGGGGGGGGG---------|
VID :                |                |                |VVVVVVVVVVVVVVVV|
     .................................................... latency 48 – 64 milliseconds

Similarly to the case with CPU workloads, it is possible to have two or more GPUs cooperate on a single frame in a way that delivers more work in a constant amount of time, but it increases complexity and generally delivers a lower total speedup.

An attractive direction for stereoscopic rendering is to have each GPU on a dual GPU system render one eye, which would deliver maximum performance and minimum latency, at the expense of requiring the application to maintain buffers across two independent rendering contexts.

The downside to preventing GPU buffering is that throughput performance may drop, resulting in more dropped frames under heavily loaded conditions.

Late frame scheduling

Much of the work in the simulation task does not depend directly on the user input, or would be insensitive to a frame of latency in it. If the user processing is done last, and the input is sampled just before it is needed, rather than stored off at the beginning of the frame, the total latency can be reduced.

It is very difficult to predict the time required for the general simulation work on the entire world, but the work just for the player’s view response to the sensor input can be made essentially deterministic. If this is split off from the main simulation task and delayed until shortly before the end of the frame, it can remove nearly a full frame of latency.

Late frame scheduling:
CPU1:SSSSSSSSS------I|
CPU2:                |RRRRRRRRR-------|
GPU :                |-GGGGGGGGGG-----|
VID :                |                |VVVVVVVVVVVVVVVV|
                    .................... latency 18 – 34 milliseconds

Adjusting the view is the most latency sensitive task; actions resulting from other user commands, like animating a weapon or interacting with other objects in the world, are generally insensitive to an additional frame of latency, and can be handled in the general simulation task the following frame.

The drawback to late frame scheduling is that it introduces a tight scheduling requirement that usually requires busy waiting to meet, wasting power. If your frame rate is determined by the video retrace rather than an arbitrary time slice, assistance from the graphics driver in accurately determining the current scanout position is helpful.

View bypass

An alternate way of accomplishing a similar, or slightly greater latency reduction Is to allow the rendering code to modify the parameters delivered to it by the game code, based on a newer sampling of user input.

At the simplest level, the user input can be used to calculate a delta from the previous sampling to the current one, which can be used to modify the view matrix that the game submitted to the rendering code.

Delta processing in this way is minimally intrusive, but there will often be situations where the user input should not affect the rendering, such as cinematic cut scenes or when the player has died. It can be argued that a game designed from scratch for virtual reality should avoid those situations, because a non-responsive view in a HMD is disorienting and unpleasant, but conventional game design has many such cases.

A binary flag could be provided to disable the bypass calculation, but it is useful to generalize such that the game provides an object or function with embedded state that produces rendering parameters from sensor input data instead of having the game provide the view parameters themselves. In addition to handling the trivial case of ignoring sensor input, the generator function can incorporate additional information such as a head/neck positioning model that modified position based on orientation, or lists of other models to be positioned relative to the updated view.

If the game and rendering code are running in parallel, it is important that the parameter generation function does not reference any game state to avoid race conditions.

View bypass:
CPU1:ISSSSSSSSS------|
CPU2:                |IRRRRRRRRR------|
GPU :                |--GGGGGGGGGG----|
VID :                |                |VVVVVVVVVVVVVVVV|
                      .................. latency 16 – 32 milliseconds

The input is only sampled once per frame, but it is simultaneously used by both the simulation task and the rendering task. Some input processing work is now duplicated by the simulation task and the render task, but it is generally minimal.

The latency for parameters produced by the generator function is now reduced, but other interactions with the world, like muzzle flashes and physics responses, remain at the same latency as the standard model.

A modified form of view bypass could allow tile based GPUs to achieve similar view latencies to non-tiled GPUs, or allow non-tiled GPUs to achieve 100% utilization without pipeline bubbles by the following steps:

Inhibit the execution of GPU commands, forcing them to be buffered. OpenGL has only the deprecated display list functionality to approximate this, but a control extension could be formulated.

All calculations that depend on the view matrix must reference it independently from a buffer object, rather than from inline parameters or as a composite model-view-projection (MVP) matrix.

After all commands have been issued and the next frame has started, sample the user input, run it through the parameter generator, and put the resulting view matrix into the buffer object for referencing by the draw commands.

Kick off the draw command execution.

Tiler optimized view bypass:
CPU1:ISSSSSSSSS------|
CPU2:                |IRRRRRRRRRR-----|I
GPU :                |                |-GGGGGGGGGG-----|
VID :                |                |                |VVVVVVVVVVVVVVVV|
                                       .................. latency 16 – 32 milliseconds

Any view frustum culling that was performed to avoid drawing some models may be invalid if the new view matrix has changed substantially enough from what was used during the rendering task. This can be mitigated at some performance cost by using a larger frustum field of view for culling, and hardware clip planes based on the culling frustum limits can be used to guarantee a clean edge if necessary. Occlusion errors from culling, where a bright object is seen that should have been occluded by an object that was incorrectly culled, are very distracting, but a temporary clean encroaching of black at a screen edge during rapid rotation is almost unnoticeable.

Time warping

If you had perfect knowledge of how long the rendering of a frame would take, some additional amount of latency could be saved by late frame scheduling the entire rendering task, but this is not practical due to the wide variability in frame rendering times.

Late frame input sampled view bypass:
CPU1:ISSSSSSSSS------|
CPU2:                |----IRRRRRRRRR--|
GPU :                |------GGGGGGGGGG|
VID :                |                |VVVVVVVVVVVVVVVV|
                          .............. latency 12 – 28 milliseconds

However, a post processing task on the rendered image can be counted on to complete in a fairly predictable amount of time, and can be late scheduled more easily. Any pixel on the screen, along with the associated depth buffer value, can be converted back to a world space position, which can be re-transformed to a different screen space pixel location for a modified set of view parameters.

After drawing a frame with the best information at your disposal, possibly with bypassed view parameters, instead of displaying it directly, fetch the latest user input, generate updated view parameters, and calculate a transformation that warps the rendered image into a position that approximates where it would be with the updated parameters. Using that transform, warp the rendered image into an updated form on screen that reflects the new input. If there are two dimensional overlays present on the screen that need to remain fixed, they must be drawn or composited in after the warp operation, to prevent them from incorrectly moving as the view parameters change.

Late frame scheduled time warp:
CPU1:ISSSSSSSSS------|
CPU2:                |RRRRRRRRRR----IR|
GPU :                |-GGGGGGGGGG----G|
VID :                |                |VVVVVVVVVVVVVVVV|
                                    .... latency 2 – 18 milliseconds

If the difference between the view parameters at the time of the scene rendering and the time of the final warp is only a change in direction, the warped image can be almost exactly correct within the limits of the image filtering. Effects that are calculated relative to the screen, like depth based fog (versus distance based fog) and billboard sprites will be slightly different, but not in a manner that is objectionable.

If the warp involves translation as well as direction changes, geometric silhouette edges begin to introduce artifacts where internal parallax would have revealed surfaces not visible in the original rendering. A scene with no silhouette edges, like the inside of a box, can be warped significant amounts and display only changes in texture density, but translation warping realistic scenes will result in smears or gaps along edges. In many cases these are difficult to notice, and they always disappear when motion stops, but first person view hands and weapons are a prominent case. This can be mitigated by limiting the amount of translation warp, compressing or making constant the depth range of the scene being warped to limit the dynamic separation, or rendering the disconnected near field objects as a separate plane, to be composited in after the warp.

If an image is being warped to a destination with the same field of view, most warps will leave some corners or edges of the new image undefined, because none of the source pixels are warped to their locations. This can be mitigated by rendering a larger field of view than the destination requires; but simply leaving unrendered pixels black is surprisingly unobtrusive, especially in a wide field of view HMD.

A forward warp, where source pixels are deposited in their new positions, offers the best accuracy for arbitrary transformations. At the limit, the frame buffer and depth buffer could be treated as a height field, but millions of half pixel sized triangles would have a severe performance cost. Using a grid of triangles at some fraction of the depth buffer resolution can bring the cost down to a very low level, and the trivial case of treating the rendered image as a single quad avoids all silhouette artifacts at the expense of incorrect pixel positions under translation.

Reverse warping, where the pixel in the source rendering is estimated based on the position in the warped image, can be more convenient because it is implemented completely in a fragment shader. It can produce identical results for simple direction changes, but additional artifacts near geometric boundaries are introduced if per-pixel depth information is considered, unless considerable effort is expended to search a neighborhood for the best source pixel.

If desired, it is straightforward to incorporate motion blur in a reverse mapping by taking several samples along the line from the pixel being warped to the transformed position in the source image.

Reverse mapping also allows the possibility of modifying the warp through the video scanout. The view parameters can be predicted ahead in time to when the scanout will read the bottom row of pixels, which can be used to generate a second warp matrix. The warp to be applied can be interpolated between the two of them based on the pixel row being processed. This can correct for the “waggle” effect on a progressively scanned head mounted display, where the 16 millisecond difference in time between the display showing the top line and bottom line results in a perceived shearing of the world under rapid rotation on fast switching displays.

Continuously updated time warping

If the necessary feedback and scheduling mechanisms are available, instead of predicting what the warp transformation should be at the bottom of the frame and warping the entire screen at once, the warp to screen can be done incrementally while continuously updating the warp matrix as new input arrives.

Continuous time warp:
CPU1:ISSSSSSSSS------|
CPU2:                |RRRRRRRRRRR-----|
GPU :                |-GGGGGGGGGGGG---|
WARP:                |               W| W W W W W W W W|
VID :                |                |VVVVVVVVVVVVVVVV|
                                     ... latency 2 – 3 milliseconds for 500hz sensor updates

The ideal interface for doing this would be some form of “scanout shader” that would be called “just in time” for the video display. Several video game systems like the Atari 2600, Jaguar, and Nintendo DS have had buffers ranging from half a scan line to several scan lines that were filled up in this manner.

Without new hardware support, it is still possible to incrementally perform the warping directly to the front buffer being scanned for video, and not perform a swap buffers operation at all.

A CPU core could be dedicated to the task of warping scan lines at roughly the speed they are consumed by the video output, updating the time warp matrix each scan line to blend in the most recently arrived sensor information.

GPUs can perform the time warping operation much more efficiently than a conventional CPU can, but the GPU will be busy drawing the next frame during video scanout, and GPU drawing operations cannot currently be scheduled with high precision due to the difficulty of task switching the deep pipelines and extensive context state. However, modern GPUs are beginning to allow compute tasks to run in parallel with graphics operations, which may allow a fraction of a GPU to be dedicated to performing the warp operations as a shared parameter buffer is updated by the CPU.

Discussion

View bypass and time warping are complementary techniques that can be applied independently or together. Time warping can warp from a source image at an arbitrary view time / location to any other one, but artifacts from internal parallax and screen edge clamping are reduced by using the most recent source image possible, which view bypass rendering helps provide.

Actions that require simulation state changes, like flipping a switch or firing a weapon, still need to go through the full pipeline for 32 – 48 milliseconds of latency based on what scan line the result winds up displaying on the screen, and translational information may not be completely faithfully represented below the 16 – 32 milliseconds of the view bypass rendering, but the critical head orientation feedback can be provided in 2 – 18 milliseconds on a 60 hz display. In conjunction with low latency sensors and displays, this will generally be perceived as immediate. Continuous time warping opens up the possibility of latencies below 3 milliseconds, which may cross largely unexplored thresholds in human / computer interactivity.

Conventional computer interfaces are generally not as latency demanding as virtual reality, but sensitive users can tell the difference in mouse response down to the same 20 milliseconds or so, making it worthwhile to apply these techniques even in applications without a VR focus.

A particularly interesting application is in “cloud gaming”, where a simple client appliance or application forwards control information to a remote server, which streams back real time video of the game. This offers significant convenience benefits for users, but the inherent network and compression latencies makes it a lower quality experience for action oriented titles. View bypass and time warping can both be performed on the server, regaining a substantial fraction of the latency imposed by the network. If the cloud gaming client was made more sophisticated, time warping could be performed locally, which could theoretically reduce the latency to the same levels as local applications, but it would probably be prudent to restrict the total amount of time warping to perhaps 30 or 40 milliseconds to limit the distance from the source images.

Acknowledgements

Zenimax for allowing me to publish this openly.

Hillcrest Labs for inertial sensors and experimental firmware.

Emagin for access to OLED displays.

Oculus for a prototype Rift HMD.

Nvidia for an experimental driver with access to the current scan line number.

Why Lua?

Niklas Frykholm — Tue, 19 Feb 2013 08:23:00 +0000

A question that I get asked regularly is why we have chosen Lua as our engine scripting language. I guess as opposed to more well-known languages, such as JavaScript or C#. The short answer is that Lua is lighter and more elegant than both those languages. It is also faster than JavaScript and more dynamic than C#.

When we started Bitsquid, we set out four key design principles for the engine:

Simplicity. (A small, manageable codebase with a minimalistic, modular design.)
Flexibility. (A completely data-driven engine that is not tied to any particular game type.)
Dynamism. (Fast iteration times, with hot reload of everything on real target platforms.)
Speed. (Excellent multicore performance and cache-friendly data-oriented layouts.)

Whenever we design new systems for the engine, we always keep these four goals in mind. As we shall see below, Lua does very well on all four counts, which makes it a good fit for our engine.

Simplicity in Lua

As I grow older (and hopefully more experienced) I find myself appreciating simplicity more and more. My favorite scripting language has gone from “Swiss army chainsaw” Perl (I claim youthful ignorance!) to “kitchen-drawer-esque” Ruby, to minimalistic Lua.

Lua is really small for a programming language. The entire Lua syntax fits on a single page. In fact, here it is:

chunk ::= {stat [`;´]} [laststat [`;´]]
block ::= chunk
stat ::=  varlist `=´ explist | 
     functioncall | 
     do block end | 
     while exp do block end | 
     repeat block until exp | 
     if exp then block {elseif exp then block} [else block] end | 
     for Name `=´ exp `,´ exp [`,´ exp] do block end | 
     for namelist in explist do block end | 
     function funcname funcbody | 
     local function Name funcbody | 
     local namelist [`=´ explist] 
laststat ::= return [explist] | break
funcname ::= Name {`.´ Name} [`:´ Name]
varlist ::= var {`,´ var}
var ::=  Name | prefixexp `[´ exp `]´ | prefixexp `.´ Name 
namelist ::= Name {`,´ Name}
explist ::= {exp `,´} exp
exp ::=  nil | false | true | Number | String | `...´ | function | 
     prefixexp | tableconstructor | exp binop exp | unop exp 
prefixexp ::= var | functioncall | `(´ exp `)´
functioncall ::=  prefixexp args | prefixexp `:´ Name args 
args ::=  `(´ [explist] `)´ | tableconstructor | String 
function ::= function funcbody
funcbody ::= `(´ [parlist] `)´ block end
parlist ::= namelist [`,´ `...´] | `...´
tableconstructor ::= `{´ [fieldlist] `}´
fieldlist ::= field {fieldsep field} [fieldsep]
field ::= `[´ exp `]´ `=´ exp | Name `=´ exp | exp
fieldsep ::= `,´ | `;´
binop ::= `+´ | `-´ | `*´ | `/´ | `^´ | `%´ | `..´ | 
     `<´ | `<=´ | `>´ | `>=´ | `==´ | `~=´ | 
     and | or
unop ::= `-´ | not | `#´

The same minimalistic philosophy is applied across the entire language. From the standard libraries to the C interface to the actual language implementation. You can understand all of Lua by just understanding a few key concepts.

Lua’s simplicity and size does not mean that it lacks features. Rather it is just really well designed. It comes with a small set of orthogonal features that can be combined in lots of interesting ways. This gives the language a feeling of elegance, which is quite rare in the programming world. It is not a perfect language (perfect languages don’t exist), but it is a little gem that fits very well into its particular niche. In that way, Lua is similar to C (the original, not the C++ monstrosity) — it has a nice small set of features that fit very well together. (I suspect that Smalltalk and LISP also have this feeling of minimalistic elegance, but I haven’t done enough real-world programming in those languages to really be able to tell.)

As an example of how powerful Lua’s minimalism can be, consider this: Lua does not have a class or object system, but that doesn’t matter, because you can implement a class system in about 20 lines or so of Lua code. In fact, here is one:

function class(klass, super)
    if not klass then
        klass = {}
        
        local meta = {}
        meta.__call = function(self, ...)
            local object = {}
            setmetatable(object, klass)
            if object.init then object:init(...) end
            return object
        end
        setmetatable(klass, meta)
    end
    
    if super then
        for k,v in pairs(super) do
            klass[k] = v
        end
    end
    klass.__index = klass
    
    return klass
end

If you prefer prototype based languages — no problem — you can make a prototype object system in Lua too.

Smallness and simplicity makes everything easier. It makes Lua easier to learn, read, understand, port, master and optimize. A project such as LuaJIT — created by a single developer — would not have been possible in a more complicated language.

Flexibility in Lua

Lua is a fully featured language, and in the Bitsquid engine, Lua is not just used as an extension language, rather it has direct control over the gameplay loop. This means that you have complete control over the engine from Lua. You can create completely different games by just changing the Lua code. (Examples: First person medieval combat War of the Roses, top-down RTS Krater, beat-em-up platformer Showdown and hand-held puzzler Hamilton.)

Dynamism in Lua

Unlike C#, which only has limited support for Edit and Continue, Lua makes it possible to reload everything — the entire program — on all target platforms, including consoles, mobiles and tablets.

This means that gameplay programmers can work on the code, tweak constants, fix bugs and add features without having to restart the game. And they can do this while running on the real target hardware, so that they know exactly what performance they get, how the controls feel and how much memory they are using. This enables fast iterations which is the key to increasing productivity and improving quality in game development.

Speed of Lua

Measuring the performance of a language is always tricky, but by most accounts, LuaJIT 2 is one of the fastest dynamic language implementations in the world. It outperforms other dynamic languages on many benchmarks, often by a substantial margin.

On the platforms where JITting isn’t allowed, LuaJIT can be run in interpreter mode. The interpreter mode of LuaJIT is very competitive with other non-JITed language implementations.

Furthermore, Lua has a very simple C interoperability interface (simplified further by LuaJIT FFI). This means that in performance critical parts of the code it is really easy to drop into C and get maximum performance.

Lua’s weak points

As I said above, no language is perfect. The things I miss most when programming in Lua don’t have that much to do with the actual language, but rather with the ecosystem around it. C# has spoiled me with things like an integrated debugger, Intellisense, a very active StackOverflow community and the wonderfully helpful ReSharper. Lua has no “official” debugger, and not much in the way of autocompletion or refactoring tools.

Some people would argue that this shouldn’t be counted as an argument against Lua, since it doesn’t really concern the language Lua. I disagree. A language is not a singular, isolated thing. It is part of a bigger programming experience. When we judge a language we must take that entire experience into account: Can you find help in online forums? Are there any good free-to-use development tools? Is the user base fragmented? Can you easily create GUIs with native look-and-feel? Etc.

The lack of an official debugger is not a huge issue. Lua has an excellent debugging API that can be used to communicate with external debuggers. Using that API you can quite easily write your own debugger (we have) or integrate a debugger into your favorite text editor. Also, quite recently, the Decoda IDE was open sourced, which means there is now a good open source debugger available.

Getting autocompletion and refactoring to work well with Lua is trickier. Since Lua is dynamically typed the IDE doesn’t know the type of variables, parameters or return values. So it doesn’t know what methods to suggest. And when doing refactoring operations, it can’t distinguish between methods that have the same name, but operate on different types.

But I don’t think it necessarily has to be this way. An IDE could do type inference and try to guess the type of variables. For example, if a programmer started to write something like this:

local car = Car()
car:

the IDE could infer that the variable car was of type Car. It could then display suitable autocompletion information for the Car class.

Lua’s dynamic nature makes it tricky to write type inference code that is guaranteed to be 100 % correct. For example, a piece of Lua code could dynamically access the global _G table and change the math.sin() function so that returned a string instead of a number. But such examples are probably not that common in regular Lua code. Also, autocompletion backed by type inference could still be very useful to the end user even if it wasn’t always 100 % correct.

Type inference could be combined with explicit type hinting to cover the cases where the IDE was not able to make a correct guess (such as for functions exposed through the C API). Hinting could be implemented with a specially formatted comment that specified the type of a variable or a function:

-- @type Car -> number
function top_speed(car)
    ...
end

In the example above, the comment would indicate that top_speed is a function that takes a Car argument and returns a number.

Type hinting and type inference could also be used to detect “type errors” in Lua code. For example, if the IDE saw something like this:

local bike = Bicycle()
local s = top_speed(bike)

it could conclude that since bike is probably a Bicycle object and since top_speed expects a Car object, this call will probably result in a runtime error. It could indicate this with a squiggly red line in the source code.

I don’t know of any Lua IDE that really explores this possibility. I might try it for my next hack day.

This has also been posted to The Bitsquid blog.

Vexing puzzle design

Kyle-Kulyk — Sun, 10 Feb 2013 09:32:26 +0000

I enjoy a good puzzle. At my core, I look for patterns in pretty much everything around me and I think we all do to some extent. Looking for order in chaos is just something that we all do from the time we’re toddlers. That’s when toddlers aren’t creating chaos, as I’m sure other new parents can attest to. It’s no surprise that puzzle games are among the most popular games available for mobile devices. A good puzzle game will keep us captivated for as long as we find it challenging. I thought I’d take a moment and share my design process as a new game designer working on the puzzle mode for our imminent title, Vex Blocks.

When we started development of Vex Blocks, we set out to create a falling block style arcade game in the vein of Tetris that utilised a device’s rotation. The job of the player was to chain together blocks on the screen by matching colors, symbols or both and tracing out patterns with their fingers to connect the blocks. Random blocks would fall into the play area and the job was to clear as many as possible, rotating the device as necessary so blocks would fall into different arrangements. Once we had created the basic gameplay mechanics, we set about trying to think of how we could change the rules of the game to create different gameplay modes and a “nice to have feature if we have the time” was puzzle mode.

So, as development moved along I ultimately found myself faced with the job of creating various puzzles for our puzzle mode. I had never set out to create a puzzle before, but how hard could it be? Start simple, right?

I started by recreating my playing area in Photoshop and went about duplicating the various game pieces so I could simply drag and drop to create the puzzles before coding them into our game. My next step was to create something aesthetically pleasing before I even thought of how the puzzle would play. I’d drop in blocks to create geometric shapes and patterns, often drawing inspiration from simple icons as I only had a 5×8 grid to work with. Once I had a pattern on the screen that I was relatively happy with, I’d start thinking about how it would play.

Here’s where it really started to get fun. The point of the puzzle mode was to solve the puzzle, clearing all playing blocks from the screen in as few chains as possible, with an upper limit on the amount of chains you could use before the puzzle would reset. I’d have a look at the blocks in front of me and start tracing out the various options for chains. If it was too straightforward, then I’d start to throw in obstacles by swapping out blocks that couldn’t be readily chained together, or could only be part of a chain coming from one particular direction. Or, I’d start with a puzzle and then mimic a few phone rotations to see what I’d end up with. It was a bit like messing up a Rubik’s cube. As challenging as a Rubik’s cube is to solve, there’s a certain amount of satisfaction in taking a solved cube and mixing it up for another to solve. For a few puzzles, that’s exactly what it was like. Starting with a solved puzzle that was easy to chain together, then scrambling it. Mmmm…satisfying.

From a design perspective, starting simple was really the only way for this project to evolve. As I started to become comfortable designing simple puzzles, I’d gradually add in new game mechanics. What if I add a block that can’t be chained and has to be surrounded? What if I introduce blocks that stay fixed in one spot despite device orientation? What if we throw in blocks that explode if you don’t clear them quickly enough? What about using specific power-ups? Adding one new gameplay mechanic at a time and exploring that mechanic fully before moving onto the next, then adding them together provided a nice progression in terms of variety and difficulty. As I became more familiar with process, design started to shift away from the look of the puzzle and instead started with a particular challenge, and then I moulded the look around the puzzle.

Next up I assembled the puzzles in-game and turned them loose on our testers. I quickly discovered that what seems easy to me after working on the game full-time for nine months isn’t necessarily as easy for gamers who haven’t spent that type of time with the product. Test, test test. Who knew? There’s a fine line between challenging and “Nuts to this” with gamers. Thankfully, I’ve received some excellent feedback and what was originally a “nice to have feature if we have the time” has become a challenging addition to the title that extends the gameplay options while offering us the opportunity to release additional content if gamers like what they see.

Brewing Your Own Game Analytics Service

Colt McAnlis — Mon, 04 Feb 2013 15:29:51 +0000

In this post, I describe how to implement a game analytics system to collect and store game data. At the end of the post you’ll find links to the source code for my sample implementation.

Introduction

As a game developer, you must gather data about your game’s users, mine that data, and respond to it. The cloud being what it is today, there are multiple costs associated with collecting and using game data, including costs for transactions and storage.

There are of course tons and tons of services out there that do pretty much the same thing in terms of data collection. This is especially true in the mobile space, where it seems that a new games analytics VC-funded company pops up every day. However, these services often come with lots of questions regarding data ownership, costs, reporting structures, and so forth. At bottom, these services may or may not fit your needs.

As such, if you’re simply looking to understand more about a game analytics system, need more functionality, or just want to roll your own, let’s take a look at how to build your very own low-cost game analytics service from scratch.

The setup

To begin, we make the following assumptions:

The user will play your client, and you’ll submit events to a server for cataloging.
You have some client code that can make HTTP requests (we’ll use HTML5 in this article).
You have some semi-resident server-side compute resource with direct access to a data store. We’ll use Google App Engine (GAE) in this article.

In a naive implementation, we assume that the client does the bulk of the work, and pushes data up to the cloud in a regular fashion. For example, we can push an event every time a rat is killed. This setup results in a simple dataflow between our components:

As shown above, the client submits some packet of data to a server, which then pushes it to the GAE Datastore directly.

Let’s hypothetically say that our game has around 15,000 players a day, and each player kills about 2,000 rats – that’s a shocking 30 million events that we’ll be tracking. At this point, we need to take a hard look at what our cost structure is for storing and computing all that data. For instance, the latest pricing on Google App Engine charges about $0.10 per 100k writes to the Datastore, meaning you’d pay about $30 a day for 30 million writes. That’s a lot of money to throw around just to store in-game events for data-mining. I mean, if you have a dedicated data miner to track that information then the cost might be justified; otherwise, you’re going to need to find a more cost-effective solution.

In addition, your client may be sending huge amounts of data to the server (for instance if you’re tracking each mouse click in an RTS game). That’s lots of traffic and compute time that you’re churning just to collect some floating point numbers.

Client-side batching

In the naive implementation above, the dominating cost factor is the sheer number of writes into the Datastore. To lower the cost, we have to reduce the number of writes.

The first thing we should do is determine the importance of the data that we’re collecting and how often we need to use that data. For instance, mouse clicks in an RTS game may be semi-important, but not so important that we need that data instantly. As such, we could batch those mouse clicks and submit them at the end of the game. This deferred batch-submit strategy is great for reducing the amount of transfers from client to server (since we only get the data at the end of the game), but doesn’t really help us reduce the number of writes into the Datastore (assuming that we’d write each entry in the batch to the Datastore after receipt).

To reduce the number of writes, we turn to another offering in App Engine. The Blobstore API allows our application to create data objects (called blobs) that are much larger than objects allowed in the Datastore service. The original Blobstore API only allowed clients to submit blobs via HTTP request, but the new experimental Blobstore API allows writing directly to the storage system from server-side code.

Using Blobstore allows us to batch our events on the client side, and submit them in a single blob.

With this setup, we only submit a batch of events at the end of the game and then store them into the Blobstore, which drops our overall cost per day significantly.

There are, however, two issues with this setup. The first issue is client connectivity. Say a user plays 20 minutes and is then disconnected, dropping needed data that we may really want. This issue is even more apparent on mobile platforms, where users are on unreliable network connections and where you should expect random data loss during transmission.

Luckily clients can take advantage of persistent storage, which allows them to store batched data and attempt to resubmit the data at a later time. Having reliable network connections means submissions are more likely to succeed the first time, but any client-side batching system needs to have, at its core, the concept of cache-resubmit for any gathered data.

The second issue is that storing data in Blobstore limits our ability to do analysis on the data directly. Before we can mess with the data, we must read it from the Blobstore into a computational structure for usage. In other words, the query “give me all the users who’ve killed a rat today” requires us to read out the data from Blobstore into Datastore (or a similar container) before doing processing.

Server-side batching

In an ideal world we’d allow clients to message-spam our server as much as they want, so that we wouldn’t have to worry about clients dropping out randomly and taking their precious data with them. We can accommodate such spammy submissions by batching events at the server.

For those of you that are new to cloud computing systems like Google App Engine, it’s worth emphasizing that GAE modules aren’t always running – rather, they are instances that are spun up depending on request volume. More importantly, they can get spun down as well, depending on query volume and infrastructure service scheduling. That means there’s really no way to keep a resident in-memory copy of data.

This is where App Engine Backends come in. Backends are pseudo persistent, heavier-weight process that can hang around for longer durations. With backends, we can allow clients/GAE instances to communicate as normal, and cache/batch the requests into a backend before submitting to the Blobstore

The cost for this setup would be about $0.08/hour for the GAE backend, in addition to the size of the data that’s being stored in the Blobstore, as well as any additional front-end / back-end compute times.

With this setup, clients can be very spammy and intermittent, and the GAE instance effectively acts as a pass-through, simply handing data off to the backend.

You can also make the backend public-facing, allowing clients to submit data to it directly rather than going through the GAE instance. But be warned that this may create a vulnerability point, as spammy/rogue clients may have the ability to engage in Denial Of Service attacks against the backend. Combining the scalability of the GAE instance frontend with the longer-running backend is a good way to eliminate this vulnerability and still allow per-client scaling/throttling.

As the number of writes in our system increases, we’ll eventually hit an upper limit on the number of requests a backend can process. More specifically, depending on the size of our event structure, our 30 million events may exceed the capacity of the back end’s available RAM.

One way to address this issue is to add an upper storage limit and start flushing data. For example, once the backend caches say 10MB of data, it can flush that data to the Blobstore for storage. This technique is particularly helpful given that GAE backends are not truly persistent – they are processes that can be rescheduled on different physical machines, which can die or become unavailable for various reasons. As such, we run into similar connectivity issues as we do with clients, although at much lower frequency. Adding regular flushes can ensure regular storage pulses to safeguard our system from losing data. A downside to these regular flushes is that they can take a chunk of time, and generally, we’ll need to flush during an event submission (if the cache gets full, we’ll need to flush before adding the new event). As such, the GAE instance can easily timeout waiting for the backend to flush its data.

A more scalable method of dealing with the limited number of requests a backend can process is to simply increase the number of backends to a desired capacity, and use a hashing function in the GAE instance to evenly distribute events to the backends. Or rather, once our traffic increases to the point where it exceeds what one backend can handle, we can add additional backends (which obviously increases the cost).

Creating a balanced approach

One important point to note is that there are actually different types of data that you will want to collect for your game. Every game is different – there is no one best set of data for all games. Your game will have a range of statistics that you should collect and store with some variability. For instance:

There may be some statistics that are fine to gather locally and push up at periodic intervals; others, you’ll want to store immediately because they are so critical.
You may not need guaranteed delivery of every single event from every single game for every single player. You may just need “most” data or a representative amount of data. For example, you can log only data from a statistically relevant percentage of clients and then extrapolate results.
Not all events can/should come from clients. For MMO games, most of the event calculation and game state reckoning takes place on a game server instance, and as such, that instance should have access to submit events as well.

You should also adjust your data collection based on input loads. Tune your storage/flushing options based on where the bulk of your data is coming from. For any flushing point (client to server, or server to Blobstore) you should adjust when to flush based on duration and how much data is stored. Always remember that you’re trading RAM for IO operations – keep data in RAM longer and you’ll need more RAM; flush data more often and you’ll do more IO. Tweak the numbers constantly to find a good balance.

One final issue to consider is the cost associated with the size of your data in Blobstore. Since you’re charged per byte, it might be worth reducing the size of the stored data. Thankfully App Engine also has a solution for that through its ZIP API.

Our final implementation, shown in the figure below, includes data segmentation, multiple levels of batching, and data compression.

Additional considerations

True cost gut-check

There are of course many additional costs involved with cloud processing. For example, there’s the computational cost (= instance hours) of processing/handling every individual HTTP request from clients, as well as the bandwidth cost associated with the per-HTTP-request overhead. To estimate your true cost, the best thing to do is to build a mock system and run some valid traffic through it to ballpark your numbers.

Reducing long-term data costs

When you implement your game analytics system, you should plan for success and consider what you want to do in the long haul. For example, imagine that on a good day, you’ll be storing some 50 million in-game events. That adds up to a significant amount of data to keep around long-term. After a year of production, the likelihood that a single day’s worth of data will be useful drops significantly, so keeping it in the cloud is going to cost you money for data that’s not used. In that case, you should consider moving the data into a form that reduces your cost over time.

One solution that may make sense is to move data regularly from the cloud to a local box, where you can access the data for a longer period of time at a lower cost. Before archiving, you should cache important data elements so that future analysis can reference the results of the data without having to pull it all back out from deep freeze.

The source

You can find source code for each of the 3 tracking methods we’ve discussed here on my github page. With the App Engine SDK for Python, you can quickly upload and run the instances. Use the given HTML pages to test the system and see how things work.

Garbage Collection and Memory Allocation Sizes

Niklas Frykholm — Thu, 31 Jan 2013 15:21:52 +0000

As a performance conscious programmer in a soft-realtime environment I’ve never been too fond of garbage collection.

Incremental garbage collectors (like the one in Lua) make it tolerable (you get rid of the horrible garbage collection stalls), but there is still something unsettling about it. I keep looking at the garbage collection time in the profiler, and I can’t shake the feeling that all that time is wasted, because it doesn’t really do anything.

Of course that isn’t true. Garbage collection frees the programmers from a lot of busywork. And the time they gain can go into optimizing other systems, which leads to a net performance win.

It also simplifies some of the hairy ownership questions that arise when data is transferred between systems. Without garbage collection, those questions must be solved in some other way. Either by reference counting (error-prone) or by making local copies of the data to assume ownership (ugly and costly).

But still, there is that annoying performance hit.

I was pretty surprised to see that the developers Go, a language that looks well-designed and targets low-level programmers, decided to go with garbage collection rather than manual memory management. It seemed like a strange choice.

But recently I’ve started to see things differently.

One thing I’ve noticed as I delve deeper and deeper into data-oriented design is that I tend to allocate memory in much larger chunks than before. It’s a natural consequence of trying to keep things continuous in memory, treating resources as large memory blobs and managing arrays of similar objects together.

This has interesting consequences for garbage collection, because when the garbage collector only has to keep track of a small number of large chunks, rather than a large number of small chunks, it can perform a lot better.

Let’s look at a simple example in Lua. Say we want to write a class for managing bullets. In the non-data-oriented solution, we allocate each bullet as a separate object:

function Bullet:update(dt)
    self.position = self.position + self.velocity * dt
end

function Bullets:update(dt)
    for i,bullet in ipairs(self.bullets) do
        bullet:update(dt)
    end
end

In the data-oriented solution, we instead use two big arrays to store the position and velocity of all the bullets:

function Bullets:update(dt)
    for i=1,#self.pos do
        self.pos[i] = self.pos[i] + dt * self.vel[i]
    end
end

I tested these two solutions with a large number of bullets and got two interesting results:

The data-oriented solution runs 50 times faster.
The data-oriented solution only needs half as much time for garbage collection.

That the data-oriented solution runs so much faster shows what cache coherence can do for you. It is also a testament to how awesome LuaJIT is when you give it tight inner loops to work with.

Note that in this test, the Bullet code itself did not create any garbage. The speed-up comes from being faster at collecting the garbage created by other systems. And the reason for this is simply that with fewer, larger memory allocations, there is less stuff that the garbage collector has to trawl through. If we add in the benefit that the data-oriented solution will create fewer objects and generate less garbage, the benefits will be even greater.

So maybe the real culprit in isn’t garbage collection, but rather having many small memory allocations. And having many small memory allocations does not just hurt the garbage collector, it is bad for other reasons as well. It leads to bad cache usage, high overhead in the memory allocator, fragmentation and bad allocator performance. It also makes all kinds of memory problems harder to deal with: memory leaks, dangling pointers, tracking how much memory is used by each system, etc.

So it is not just garbage-collected languages like Lua that would benefit from allocating memory in larger chunks, but manually managed languages like C++ as well.

Recently, I’ve come to think that the best solution to memory management issues in C++ is to avoid the kitchen-sink global memory allocator as much as possible and instead let each subsystem take a much more hands-on approach to managing its own memory.

What I mean by this is that instead of having the sound system (for example) send lots of memory requests to the kitchen-sink memory manager, it would only request a few large memory blocks. Then, it would be the responsibility of the system to divide that up into smaller, more manageable pieces that it can make practical use of.

This approach has a number of advantages:

Since the system knows the usage patterns for its data, it can arrange the memory efficiently. A global memory allocator has no such knowledge.
It becomes much easier to track memory use by system. There will be a relatively small number of global memory allocations, each tagged by system. It becomes obvious how much memory each system is consuming.
Memory inside a system can be easily tracked, since the system knows what the memory means and can thus give useful information about it (such as the name of the object that owns it).
When a system shuts down it can quickly and efficiently free all of its memory.
Fragmentation problems are reduced.
It actively encourages good memory behavior. It makes it easier to do good things (achieve cache locality, etc) and harder to do bad things (lots of small memory allocations).
Buffer overflows will tend to overwrite data within the same system or cause page faults, which will make them easier to find.
Dangling pointer access will tend to cause page faults, which will make them easier to find.

I’m tempted to go so far as to only allow whole page allocations on the global level. I.e., a system would only be allowed to request memory from the global manager in chunks of whole system pages. Then it would be up to the system to divide that up into smaller pieces. For example, if we did the bullet example in C++, we might use one such chunk to hold our array of Bullet structs.

This has the advantage of completely eliminating external fragmentation. (Since everything is allocated in chunks of whole pages and they can be remapped by the memory manager.) We can still get address space fragmentation, but using a 64-bit address space should take care of that. And with this approach using 64-bit pointers is less expensive, because we have fewer individually allocated memory blocks and thus fewer pointers.

Instead we get internal fragmentation. If we allocate the bullet array as a multiple of the page size (say 4 K), we will on average have 2 K of wasted space at the end of the array (assuming the number of bullets is random).

But internal fragmentation is a much nicer problem to deal with than external fragmentation. When we have internal fragmentation, it is one particular system that is having trouble. We can go into that system and do all kinds of things to optimize how its handling memory and solve the problem. With external fragmentation, the problem is global. There is no particular system that owns it and no clear way to fix it other than to try lots of things that we hope might “improve” the fragmentation situation.

The same goes for out-of-memory problems. With this approach, it is very clear which system is using too much memory and easy to fix that by reducing the content or doing optimizations to that system.

Dealing with bugs and optimizations on a system-by-system simplifies things enormously. It is quite easy to get a good grasp of everything that happens in a particular system. Grasping everything happens in the entire engine is a superhuman task.

Another nice thing about this approach is that it is quite easy to introduce it on a system-by-system basis. All we have to do is to change one system at a time so that it allocates its memory using the page allocator, rather than the kitchen-sink allocator.

And if we have some messy systems left that are too hard to convert to this approach we can just let them keep using the kitchen-sink allocator. Or, even better, we can give them their own private heaps in memory that they allocate from the page allocator. Then they can make whatever mess they want there, without disturbing the other systems.

This has also been posted to The Bitsquid blog.

Implementing Voxel Cone Tracing

Simon Yeung — Thu, 31 Jan 2013 14:22:12 +0000

[Updated on 25-2-2013: added a paragraph about lowering the voxel update frequency for faster dynamic update. The demo also added a combo box for choosing the update frequency]

Introduction

In last year SIGGRAPH, Epic games presented their real time GI solution which based on voxel cone tracing. They showed some nice results which attract me to implement the technique and my implementation runs at around 22~30fps (updated every frame) at 1024×768 screen resolution using a 256x256x256 voxel volume on my GTX460 graphic card. The demo program can be downloaded here which requires a DX11 GPU to run.

With GI

Without GI

Overview

There are 5 major steps in voxel cone tracing:

0. Given a scene with directly lighting only
1. Voxelize the triangle meshes
2. Construct sparse voxel octree
3. Inject direct lighting into the octree
4. Filter the direct lighting to generate mip-map
5. Sample the mip-mapped values by cone tracing

Step 0

Step 1

Step 2

Step 3

Step 4

Step 5

The step 1 and 2 can be done only once for static geometry while step 3 to 5 need to be done every frame. The following sections will briefly describe the above steps, you may want to take a look at the original paper first as some of the details will not be repeated in the following sections.

Voxelization pass

The first step is to voxelize the scene. I first output all the voxels from triangle meshes into a big voxel fragment queue buffer. Each voxel fragment use 16 bytes:

The first 4 bytes store the position of that voxel inside the voxel volume which is at most 512, so 4 bytes is enough to store the XYZ coordinates.

Voxel fragments are created using the conservative rasterization described in this OpenGL Insights charter which requires only 1 geometry pass by enlarging the triangle a bit using the geometry shader. So I modify my shader generator to generate the shaders for creating voxel fragments based on the material.

Scene before voxelize

Scene after voxelize

Octree building pass

I use a similar data structure that described in the voxel cone tracing paper and giga voxel paper with a large octree node buffer storing the octree/voxel node data (with 8 node grouped into 1 tile which described in the paper) and a 3D texture storing the reflected radiance and alpha from the direct lighting at that voxel (I assume all the surface reflect light diffusely so only 1 3D texture is used which is different from the paper storing incoming radiance).

Each octree node is 28 bytes with the first 4 bytes storing the child tile index with another 24 bytes storing the neighbor node byte offset from the start of the octree node buffer:

The child node tile index is also used to index the 3D texture brick(more details can be found in voxel cone tracing paper about the texture brick) which store the reflected radiance of that octree node tile. And since I use 5 bits for store different bit flags for dynamic update, so my voxel volume can only be at most 512x512x512 large.

In the octree leaf node, it stores the voxel data directly which only use 16 bytes:

the first 4 bytes store the bit flag to indicate whether that voxel is created from static mesh so that it will not be overwrite by dynamic geometry. Also the 1 byte counter stored along with the voxel normal is used to perform averaging when different voxel fragments fall into the same voxel. The steps to perform atomic average can be found in the OpenGL Insight Chapter.

So given the voxel fragment queue output from the previous steps, we can build the octree using the steps described in OpenGL Insights Chapter and average the octree leaf node values when different voxels fall into the same node.

showing highest octree level

showing the octree with the voxels

Inject direct lighting pass

After building the voxelized scene, we need to add the lighting data into the data structure to calculate global illumination. First, we render the shadow map from the light’s point of view. Then for each pixel of the shadow map, we can re-construct the world position of the shadow map texel and then traverse down the octree data structure to calculate the reflected radiance(assume reflected diffusely) and write it to the 3D texture bricks.

In my engine, I use cascade shadow map with 4 cascades, the last cascade is used to perform the light injection, with the slope scale depth bias disabled, otherwise, the re-constructed world position may not located exactly inside the voxel. Also, it is better to stablize the shadow map (in the demo, the position of the directional light is computed from view camera) in order to avoid flicking during cone tracing step when the camera moves. But after stablizing the shadow map, not all the voxels are filled with lighting data at 512x512x512 voxel resolution as the shadow map is not fully utilized…

This artifact will also occur when the shadow map resolution is low. Consider the the figure below, we have a directional light injecting lighting to the scene (which have a wall and floor).

As we launch one thread for each shadow map texel to determine which voxel get light injected, we can only inject light to 3 voxels(the white square in the figure) in the above case. While if the shadow map resolution is high enough, those 5 vertical voxels should all be injected with light.

Filtering pass

In order to perform cone tracing step, we need to filter the lighting data at the leaf node of the octree. The voxel lighting data is filtered anisotropically along the positive and negative XYZ directions. For example, in a 2D case(which is easier to explain and very similar to the 3D case), for each node tile, we will have 5×5 voxels which will be filtered to 3×3 voxels in the upper mip level like this:

To filter the center voxel(texel E in the figure) in the node tile in upper mip level along the +X direction, we need to consider the 9 texels (texel g, h, i, l, m, n, q, r, s in the figure) in the lower level, The texels are divided into 4 groups as the figure below:

in each group, we filter along the +X direction, for example in the upper left group, we first alpha blend the value in the +X direction and then average them to get the value for that group.

After calculate the values for all the 4 groups, we can compute the filtered center voxel value by repeating the above steps for the 4 group values.

But for the corner values(e.g. texel a, b, f, g) in a node tile we cannot access all the 9 texels to compute a filtered value as those values are in the neighbor node tile. So the group value are partially computed and store in the 3D texture first, then we rely on the transfer step described in the cone tracing paper to complete the calculation. In other words, during the transfer steps, some of the filtered value is computed by first alpha blending the neighbor group values followed by averaging them, and other filtered value will be computed by averaging first followed by alpha blending. Although this calculation is not commutative, but this can reduce the number of dispatch passes to compute the filtered value with a similar result.

So we have 6 directional values for a filtered voxels. To get a sample from it for a particularly direction, we can sample it using the method like ambient cube with 3 texture read (the following code is simplified):

float3 sampleAnisotropic(float3 direction)
{
float3 nSquared = direction * direction;
uint3 isNegative = ( direction < 0.0 );
float3 filteredColor=
nSquared.x * anisotropicFilteredBrickValue[isNegative.x] +
nSquared.y * anisotropicFilteredBrickValue[isNegative.y+2] +
nSquared.z * anisotropicFilteredBrickValue[isNegative.z+4];
return filteredColor;
}

Voxel Cone Tracing pass

After doing the above steps, we finally can compute our global illumination. We first consider the simple case for ambient occlusion, which need to compute the AO integral. We approximate it by partitioning the integral with several cones:

Where each partition need to multiply with a weight W. In my implementation, 6 cones are traced with 60 degree over the hemi-sphere in the following direction (Y-axis as the up vector):

each with a weight W:

Then to calculate the visibility inside one cone, we take multiple samples from the filtered 3D texture bricks along the cone direction:

But, the remaining problem is to determine the sampling position. Since we manually filter the 3D texture bricks, the hardware quadrilinear interpolation will not work. So to avoid performing the quadrilinear interpolation manually, It is better to make sampling position located at each mip level, having voxel size equals to the cone width at that position. This position is calculated by assuming the shape of voxel is sphere rather than cube(which is easier to calculate) as follows:

Then the sampling location can be calculated given the cone origin, cone angle, trace direction and voxel radius with some simple geometry. And here is the voxel AO result:

AO result using 512x512x512 voxel volume

Next, for diffuse indirect illumination, the calculation is very similar to AO:

The indirect diffuse calculation is done using the same cones as the AO so that both the indirect diffuse and AO can be calculated at the same time:

Finally, we calculate the indirect specular, where cone is traced in the reflected view direction along the surface normal. The cone angle is depends on the glossiness, g, of the material, currently I use the following equation to calculate the angle:

The glossiness is limited to a range so that the cone angle will not be too narrow to avoid stepping through thin walls. Here is the result of indirect specular:

In my engine, I use a light pre-pass renderer, so for the opaque objects, I calculate the indirect lighting by rendering a full screen quad after the lighting pass which is then blend on top of the direct lighting buffer using the following blend state:

With the value of AO store in alpha channel. this can apply AO to both the direct and indirect lighting. Sometimes different AO intensity is need for the direct and indirect light, this blend state can be used instead:

which apply the alpha blending to only the direct lighting and the indirect AO is applied inside the shader.

When filtering mip map directionally from the leaf node, normal direction is not taken into account, which results in more light leaking for thin objects as below. And this artifact can be hidded slightly with the AO value calculated in cone tracing:

light leak through thin geometry

apply the AO can only hide the leaking a bit

Dynamic update

To make the indirect illumination calculation faster, 5 ways are used to speed up the calculation a bit.

First, when the scene is initialized, the first 4 steps that described in the overview section are performed for all the static geometry. Then in every frame, we re-calculate all the 5 steps for voxels that affected by dynamic objects. So, only dynamic objects will be voxelized every frame (while static voxels are already stored in the octree and we don’t overwrite those data which can be identified by bit flag stored in the node). Those dynamic voxel fragments are appended to the end of the octree node buffer which can be cleared easily. Also, we need to reset the static octree node neighbor offset which points to dynamic nodes, and those static nodes that affected by dynamic voxels in previous and current frame need to re-filter again. Those nodes are found by dispatching threads for all the nodes and queue those node index into another buffer. So only those with changed value will be re-calculated.

Second, for voxelizing the dynamic geometry, the InterlockedMax() function is used to compute both the diffuse and normal for the octree voxel which is faster than performing an atomic average. Note that using InterlockedMax() function for the normal will decrease the lighting quality if the voxel volume is at a small size(e.g. 256x256x256). You can see the artifact in the below figure:

Reflect light using an average voxel normal

Reflect light using InterlockedMax() voxel normal.

So I only use InterlockedMax() function for normal in the dynamic geometry and keeping the static geometry computing an average normal. That is why you can see the octree voxel data structure(in octree building pass section) store a 1 byte counter along with the normal but not other attributes.

Third, I perform a view-frustum culling when injecting the direct lighting into the octree because there is no point to filter the light that is far from the camera where we never sample it from the current point of view. An extended frustum is calculated from the current camera (refer to the figure below) for culling.

This frustum is simply moving the camera backward a bit with increased far plane using the same field of view. The extended distance is calculated by the maximum cone tracing distance (I limited the cone tracing distance in the demo, which lost the ability to sample from far objects like the sky) and the filtered voxel size.

Fourthly, I perform the cone tracing pass at half resolution of the screen resolution. However, this result in visible artifact when up-scaling to full resolution especially at the edge of the geometry (the strength of the indirect lighting in the following screen shots are increased to show the artifact more clearly).

up-sample with only bilinear filtering

So, I first decided to just fix those pixels by finding them with an edge detection filter using depth buffer.

perform an edge detection pas

And then perform cone tracing at those pixel again at full resolution:

perform the cone-tracing again in the edge pixel at full resolution

Some of the artifacts are gone, but the frame rate drops a lot again…

So, my second attempt is to just simply blur those pixels (averaging with the neighbor pixel). The quality of blurring is not as good as re-compute the cone tracing, but it is much faster.

up sample with blurring at edge

cone tracing at full resolution for reference

Lastly, The voxel world can be updated at a different frequency than the render frame rate. By assuming the light source/dynamic models are not moving very fast, we can take advantage of the temporary coherency and perform the voxelization at a lower frequency, say update at every 5 frames. We can trade the accuracy of the voxels for update speed, but this may result in the artifact shown below:

The voxel model is lag behind the triangle
mesh due to the lower update frequency

Conclusion

The advantage of voxel cone tracing is to compute the GI in real-time with both the dynamic lighting and geometry. Also the specular indirect lighting gives a very nice glossy effect. However, it uses lots of the processing power/memory and quality of lighting is not as good as the baked solution. In my implementation, I can only use 1 directional light to compute single bounce indirect lighting. And there are still room to improve in my implementation as fewer cones can be used for tracing, using the depth buffer for view frustum culling, better up-sampling when performing cone tracing and divide the voxels into several regions to handle a larger scene like Unreal Engine does. But there is never enough time to implement all that stuff… So I decided to release the demo at this stage first. In the demo, I have added some simple interface (for changing stuffs like the light direction, indirect lighting strength) for you to play around with. Hope you all enjoy the demo.

Finally, I would like to thanks Kevin Gadd and Luke Hutchinson for reviewing this article.

References

[1] The Technology Behind the “Unreal Engine 4 Elemental demo” http://www.unrealengine.com/files/misc/The_Technology_Behind_the_Elemental_Demo_16x9_(2).pdf

[2] Interactive Indirect Illumination Using Voxel Cone Tracing http://maverick.inria.fr/Publications/2011/CNSGE11b/GIVoxels-pg2011-authors.pdf

[3] Octree-Based Sparse Voxelization Using the GPU Hardware Rasterizer http://www.seas.upenn.edu/%7Epcozzi/OpenGLInsights/OpenGLInsights-SparseVoxelization.pdf

[4] GigaVoxels: A Voxel-Based Rendering Pipeline For Efficient Exploration Of Large And Detailed Scenes
http://maverick.inria.fr/Publications/2011/Cra11/

[5] GPU Gems 2: Conservative Rasterization http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter42.html
[6] Shading in Valve’s Source Engine http://www.valvesoftware.com/publications/2006/SIGGRAPH06_Course_ShadingInValvesSourceEngine.pdf
[7] Perpendicular Possibilities http://blog.selfshadow.com/2011/10/17/perp-vectors/

[8] A couple of notes about Z http://www.humus.name/index.php?ID=255

An update to Single Joystick

Alex Moore — Tue, 29 Jan 2013 09:23:34 +0000

Two weeks ago I posted an article showing progress on a new type of touch screen joystick, that allows you to move and rotate using a single touch. The response to that was really positive, and so I’ve continued working on it in the background. This is a short post updating on progress.

Most of the update is covered in this video, which I’ve recorded on a camera so that I could narrate:

To cover the main points done on the road to improving the system:

To alleviate the issue of not being able to see where your thumb is on the stick, I’ve added a mirror joystick in the centre of the screen that appears when you’re touching the main one.
I’ve added a nice orange dot to show you the direction you’re walking in. This was first done by Eric Raue on his own implementation of the original article (Eric also added a web player build of his version here).
Rotation arrows now appear when you touch the rotate gizmo. This seems to have been the turning point for ease of use: without these arrows people struggled to understand why they were rotating, with them they get it much easier.
The faster you rotate the slower you walk. This makes it possible to perform much tighter turns, which seems important as a lot of people don’t turn until they’re at the point they want to (unlike myself, who tends to use it more akin to driving a car).

This version has been focus tested quite a bit now, and is showing a lot of promise. Most people get it within a minute, a few take a bit longer. Eased into the mechanic though I think this is very good, especially when compared to the learning curve of dual sticks.

I’m going to keep working on this when I can, but it might be a little while before there’s another update. Which may well be about starting a kickstarter :)

Put On Your Game Face (revisited)

David Czarnecki — Tue, 29 Jan 2013 04:25:58 +0000

TL;DR

About a year ago, I wrote a blog post here about a weekly blog series on our company blog called “Game Face”. A blog post revisiting a blog post about blog posts? It’s blog posts all the way down. As a refresher, “Game Face” is “our weekly round-up of our internal and external open source work at Agora Games. Internal open source refers to our public projects that you can find over at our Agora Games GitHub account. External open source work refers to projects that we contribute to in off-hours and may or may not have anything to do with video games because we’re swell folks like that.” How’d we do in our open source efforts over the last year? Read on…

THE BRASS TACKS

In 2011, we open sourced 22 projects. In 2012, we open sourced 33 projects. Obviously not all of these projects are going to “stick”, but one man’s trash is another man’s treasure right? The projects aren’t trash, BTW. A few projects came out of internal company-wide Hack-A-Thon sessions that we run every 2 months, but a fair number of them are taken from production applications. In looking at the 2012 project breakdown, by language, we have Ruby (22), Python (3), CoffeeScript (3), JavaScript (2), C++ (2), PHP (1). 7 of our engineers and 21 external people contributed to the various projects that we open sourced in 2012. Contributions range from clarifications in documentation to major new features that are developed, documented and tested.

It is very easy for us to track contributions to all of our open source projects. All of the projects are setup with a service hook to notify our company’s “Open Source” chat room when events happen like code is committed or a pull request is opened or an issue is filed. Every week I create a new “shell” for the “Game Face” blog post after the last one is published. I set it to be published automatically at 9 AM on the following Friday. As contributions come in to our open source projects, next week’s blog post is updated.

I’m not sure if we’ll open source more projects than we did in 2012. 33 is a fair number of projects to have open sourced. We obviously want quality over quantity. I’m always pushing to see if there’s anything we can open source. One of the ideas being tossed around as a way to expand our open source efforts would be to hold a weekly “Office Hours” session where a few engineers hop onto IRC or a Google “hangout” for an hour or so and field questions on our projects.

FIN

We are happy to contribute back in any way we can to open source software, whether it be our own projects or the projects we use on a daily basis. Again, if your company does anything with open source, I’d love to know about it!

The Crowdfunding Experience

Alex Norton — Sun, 27 Jan 2013 04:25:07 +0000

So, just recently, I was asked to give a 90 min lecture in my home city regarding crowdfunding after managing to run a successful campaign myself in mid 2012. The lecture is pretty long, but I’ll include the video at the end in case anyone’s interested in sitting through the whole thing. If not, I was also asked to write a shortened version of it for people to read, so I thought I’d share it here.

Keep in mind this whole lecture purely relates to my own experience with crowdfunding and not to that of others, but I hope that any prospective crowdfunding applicants can get something useful out of it!

—

Hi, my name is Alex Norton, and in mid 2012 I ran a successful crowd-funding campaign for my procedural RPG, Malevolence: The Sword of Ahkranox, earning over 500% of my requested total. How did I do it? Well, I’ll tell you – but it’s not an exact science. The best I can do is to give you what advice I can.
First of all, why did I even look into crowd-funding? Many campaigns you see on places such as Kickstarter are looking to fund an entire project – sometimes asking for hundreds of thousands of dollars to do this. To me, that seems a bit much, particularly given that all that is offered up on these campaigns are some design drawings and an empassioned speech by a director of some sort. My tact was different. I already had a product, but a rough one. It worked, but wasn’t as pretty as it could be. To make your project pretty is often the part that takes up most of your money, so that’s what I looked to crowd-funding for. Polish is often what makes or breaks a product. No matter how impressive something is technically, without the right shine it just becomes hard to “sell” to an audience.
In addition to polish, crowd-funding my game gave me an insight into its market viability. Would it sell? Is the idea sound? How will the public react to it? All of these questions and more were answered purely based on the social response that my project got on the network, and on blog sites that talked about it. So that’s another angle for you to look at.
The first time I attempted crowd-funding, however, didn’t go so well. I made up a video for my product, slapped together a few screen- shots and gameplay videos and left it there for a month on IndieGoGo, expecting to find the account full of money at the end of it.
Needless to say, the excersise was fruitless. Not only did it get to barely an eighth of what I needed financially, it copped a fair amount of negative publicity, which put me to thinking.
Now please, don’t get me wrong, the fault for this lay with me entirely. IndieGoGo is a fantastic site (and available for use by people in Australia) but my strategy for running my campaign was entirely wrong. So I took the lessons I learned from it and made the move to Kickstarter via one of the American members of my team.
After doing significant research on other, more successful, crowd-funded projects, I came to the following conclusions:

- Communication is key! Talk to your pledgers and potential customers. Answer their questions when they have them and keep them engaged.

- It all comes down to how you sell your product, and how you sell yourself and your team as people.

- Be aware of your target audience and gear every little thing you do towards them and only them.

- Market your campaign. Send links to blogs, reviewers, journals, magazines and spread it across social networks. Get traffic to it.

- Be willing to put in the hours that it takes to make regular updates, answer all questions and keep the customer engaged at all times.

On top of this, I decided to change my tactics to include the following:

- Show the customer that the project is being made by people. Good quality, friendly, nice people. If they like you, they’ll be more
inclined to like what you’re selling.

- Don’t just show them why your product is special. Show them why it’s special to YOU, and why it should be special to THEM.

- When you’re selling your product, you should also be selling the people making it. It makes the customer feel a part of something,
rather than a simple ‘browse and buy’ scenario.

- Write lots of updates. At LEAST 3 per week. Show them you’re working at it. Show them how dedicated you are. Try and use video where
possible. People respond to video. Get your best speaker onto it.

- Talk to the pledgers, not just as someone who answers questions, but really engage them in conversation. Show them you’re real.

So after doing all that, I had an interesting campaign. It took work, to do all that, I won’t lie, and that’s something you have to be prepared to do. It’ll be more work than you think it will be, too. So, for the month that your campaign is running, prepare to turn away friends and family, to decline invitations and to tell everyone to leave you alone. If this project really means that much to you, then you need to be willing to dedicate your life to it for a month.
I did this myself, and the response was massive. My project was fully funded within the first week, and after that it just kept climbing and climbing. I had to move on to add stretch goals quite quickly, which was something I’ll admit I had not planned for, but when it comes to crowd-funding, you never know what might happen, so you have to be prepared for not only the worst, but the best, too!
Once it was all over and done with, I had the aftermath to deal with, which was another thing I wasn’t expecting. It’s not as simple as all that. To suddenly go from being a penniless developer to having all that cash sitting there… It’s a hard thing to be responsible and look after it properly, but you must. Don’t go nuts with it as some people have done in the past. Be responsible, be well-managed and you’ll reap the benefits of it.
Another thing you’ll have to deal with is the public hatred. Yes, it’s strange, but some people don’t like to see other people succeed, and if you happen to have a particularly successful campaign, some people will hate you for it. They will make you feel terrible, but trust me when I say that it’s best to just ignore them. If you’ve handled your campaign as I stated above then you’ll have a vast and loyal following behind not only your product, but you as well, and they’ll always make you feel welcome!

Not all campaigns work though, and it’s not always your own fault, or because your product is a bad one. Sometimes the planets just don’t want to align for you, and that’s alright, too. If that happens, it is definitely worth your time to analyze the wreckage and find out what could have been done better, so that you don’t make repeat mistakes next time. That information is PRICELESS, I cannot stress enough. A failure is only a failure if you fail to learn from it. Remember that. You can always try again.

So, what would some last-minute advice be to a potential crowd-funder? The following is the best I can offer:

- Just because you CAN crowd-fund, doesn’t necessarily mean you SHOULD. Make sure your product has GOTTEN somewhere first, otherwise jumping
into a crowd-funding situation can bring some pretty nasty PR down on your project.

- Don’t do it unless you’re prepared to deal with it. I cannot stress to you just how much work it is if you do it right, and that is
something that you really have to be ready for. PROPERLY ready for.

- Be sure to market your campaign EVERYWHERE. Don’t rely on people to just happen across it one day. Spread the word EVERYWHERE. Don’t be
shy about it!

So that’s all for now! This is a cut-down version of this topic, but for the full version, feel free to watch the video at the link below. I wish you all the best in your campaign and your projects! Good luck!

(In case the embedded video doesn’t show up, click here for the link to YouTube)

Surviving a Game Jam

Andrew Meade — Thu, 24 Jan 2013 23:12:00 +0000

So this being AltDevBlogADay, I’m sure that a lot of people reading this have done at least one game jam. I’m sure that a good deal of people also haven’t had the pleasure yet, so this little write up is for them. This isn’t really a full article, just a jot-down of some musings that are kind of rushed because I’m most busy when an article on it is most timely.

In a few days, thousands of people around the globe will gather at pre-registered locations, and attempt to make a game with a specific theme within 48 hours at the 2013 Global Game Jam. Game Jams are an amazing place to have fun, make new connections, and learn new skills. And of course, there’s the ever-valuable fact that you walk into a place, and 48 hours later you (hopefully) have a functioning game with YOUR name on the credits. Boom. 48 Hours. Game. Done.

So what can you expect from a one of these events? Well every locale is different, but typically you all meet at the prescribed location at the prescribed time, watch a video, and then get pseudo-randomly assigned to teams (unless you come with your own team). Once the 48 hours is up, an expo is held where everyone shares their games and talks about the process. Once that’s over, some people will continue to punish their bodies by the intense imbibing of spirits. That least bit I can’t recommend, but I do it anyway.

Anyways, the goal is to have fun. The goal is also to make a game, and it sucks when a team just can’t get it together in time. Unfortunately this will happen to some teams, so I’m here to kind of help keep that from happening. So without further ado, here is a small list of thoughts and hard-learned lessons to help you out on your maiden voyage.

Nourishment

I’m not huge on caffeine, but here’s where you’re going to see cases of red bull and pots of coffee all over the place. I personally like to stay hydrated with tons of water, naked juice to boost my energy, and snacks high in fiber and vitamins. I’ll also go through about 6-7 red bulls. That’s just me. You can also get by fine on Bawls, tasty cakes, and bugles, but don’t blame me if you’re feeling gross and your body starts producing neon waste after the 24 hour mark. I also advise that the team at least once physically remove themselves from their chairs, and go for a walk to a nearby eatery. First off, breaking bread together is just great teaming. Second, everyone gets a much needed break and some decent hot food. If you aren’t jamming next to a place to eat, consider ordering delivery and eating in a location where you aren’t working. I go to the Game Jam in Orlando, so we have the luxury of comfortably being outside without bundling up. I know some of you may not be that lucky.

Hygiene

Shower before you go! You probably won’t for two days. Nobody wants you to stink.

Rest

Try to get some rest. Some people can power through for 48 hours and do fine. If you’re one of those people, then good for you. I can be, depending on my mood. I recommend getting some sleep around the 30 hour mark. Maybe 4 or 5 hours. I knew a guy that refused to sleep and around hour 40 started wandering around aimlessly before falling asleep in the cupboard. Fun times. If you don’t sleep, take a break. When I do a Game jam, I get up and go outside for ten minutes every hour. Keeps my mind sharp, keeps my limbs moving, and keeps my energy up.

What should I bring?

- Your rig, obviously, with any spare stuff you can afford to lug. People are always forgetting cables, headphones, mouse pads – stupid little stuff.

- Drinks and snacks for you, plus room for it to get mooched on. Don’t be the guy that brings nothing, but don’t be the guy that’s a tightwad with the snacks when the less prepared run out. You’re all in this together, after all.

- Pillow, blanket, bedroll. You’ll likely be sleeping on the floor, so do what you can to make that a better experience for you.

- Change of unmentionables, fresh t-shirt, overshirt, and maybe a sweatshirt or hoodie if you want.

- Toothbrush, toothpaste, deodorant, meds, etc.

- Your talent!

Jobs

Well programmer, artist, and audio engineer seem pretty self explanatory, but the designer is a little harder to define. As much as it pains me to say, a designer’s talents aren’t optimal for a game jam, but they can still be super useful. As a designer, I do design work, but I also step into a producer role, a runner role, and an “oh my god we need three more artists and we have ten hours go find them” roll.

I wouldn’t advise for a team to take on more than one designer, unless you’re working in an engine like UDK or Unity, and all the programming is already done. Just remember designers, it’s not that we’re not useful, it’s that a Game jam isn’t a nurturing environment for us to really get going. Don’t believe me? That’s fine. Just do the game jam before you get on the comments section and call me an idiot.

Making the Game

You’ve got your team. You’ve got your red bull. The coordinator has announced the theme. Here’s some advice. Pick a simple game idea, with one simple mechanic, and get it working. Then figure out how to get your theme to work with that. At one GGJ we went through a list of simple games like pong, asteroids, and connect four. We ended up going with an on-rails platformer where the sole mechanic was jumping. To be fair, it was for the kinect, so you could maybe get a little more complex than that if you’re not putting together a kinect game in 48 hours.

Ideally you should have your game functional and running within a few hours, then spend the rest of the time making it your own and iterating. This is advice I was given for my first game jam by a very smart person. It is really a winning way to ensure you get a game done.

Well that’s it! If anyone out there has any more advice, add it in the comments! Good luck and happy jamming!

Aesop’s Games

Rich Skorski — Fri, 18 Jan 2013 09:00:00 +0000

My wife and I just had our first child. While our daughter was baking in my wife’s belly, we would talk about what kinds of video games she would like to play when she grew up or if she would wind up liking them at all. I thought long and hard about what games I would want her to play. Which games in my library would I leave on the table for her to discover, and at what ages? What I’ve been looking for are games that teach morals and lessons, particularly ones that do so in a way that cannot be done in other media. A game puts the audience in control, which brings a high level of teaching possibilities to the table.

Aesop was a person who may or may not have existed, but Aesop’s fables are well known. Even if you’ve never heard of them, you most certainly know one of the tales. The story of The Tortoise and the Hare and The Boy Who Cried Wolf are Aesop’s fables. They’re fun stories, and are great for children because they have good lessons behind them. The story is crucial because it drives the point home. You can tell a child “slow and steady wins the race” and they’ll think you’re a loony. Silly daddy, you’re supposed to go fast to win races! If you go on about the over confident hare and the determined tortoise, then you have planted the seed of knowledge and enlightened that youngster.

You can take a story with a good lesson, and smack that narrative into a game. There’s nothing wrong with that, but that’s not what I’m looking for. Instead, a game can teach through its mechanics and the choices presented to the player. This is like a lesson on steroids because the player needs to consciously make the correct decisions on their own in order to succeed (let’s pretend gamefaqs.com doesn’t exist for a moment). There is no better teaching tool than that! Think of the saying, “you can lead a horse to water but you can’t make him drink.” A horse must drink to complete a game, and if you design a game well then they will learn to drink on their own. I’m sure there’s a market for horse drinking games. Part of our art is in making that learning experience better. It’s no easy task, but this is what separates games from other art forms. This power cannot be found elsewhere.

But, with great power there must also come great responsibility!

Teaching the player how to think rather than what to think is the goal. Otherwise, you’re brainwashing, and that’s no good. There are many studies that try to prove games are great for psychological and social development, while others try to convince us that they desensitize us and encourage excessive aggression. It seems like Gamasutra.com has an article about new research every week or so. Studies that try to definitively mark games as good or bad are ridiculous, because it’s impossible to pin either side as the absolute conclusion. The power is in our hands to design a game that is healthy or harmful.

Labeling even a single game as one extreme or the other isn’t easy, either. There are few games I’ve played that have no health benefits. Those I’ve seen that are trash are found on Facebook. I don’t mean to knock Facebook games, and I have seen a number that are enjoyable and worth playing. The problem is that most of these games try to make everybody overly happy so that they play the game, and to do that they instill a false sense of accomplishment and never inflict penalties on the player. Even if they sit back and do next to nothing (they’ll probably have to hit a button), players will advance in the game. There’s nothing gained from that except wasted time. My wife calls these gerbil games because players bring their nose to the tube to get a drop of achievement water. Water is healthy, but a game is meant to have value in the way the water is obtained.

Some folks may mark violent or vulgar games as unhealthy, but I will not denounce these games. The target audience needs to acknowledges the same way medicine’s instructions and dosage are. Such games are not healthy for a young child, but nor is 1000mg of Tylenol*. They can be useful to adults as stress relief, or perhaps work as a sort of reverse physcology. As I’ve grown older, I have found myself thinking “I’m sure glad I don’t have to do this for reals” while playing war centric games. That would be an interesting research topic: can a violent games be designed to encourage people to be less violent? Regardless, I will let my little girl sticks with IRL games of cops and robbers and cute games of boyfriend+girlfriend+puppy for a good long while before letting her play adult games.

It’s not inherently bad to allow the player to make “evil” choices in a game, either. Most RPGs I’ve played recently had a good/evil meter or presented me with choices that were fairly black and white. Letting the player be a bad guy is ok. In fact, it’s the sign of a healthy game because it lets the player decide which path to take. If these games were to make it more advantageous to be a bad guy then it might be unhealthy, but the games I’ve been playing are pretty well balanced and sit in the grey area until the player starts to make decisions. That’s a good spot for games with moral meters to aim for, the neutral bull’s eye. If instead they were to lead the player to one side or the other, then they would be telling the player what to think. Having the world react reasonably to their decisions gives them the ability to decide how to proceed on their own. That makes a beautiful game.

But back to my task…I need to find a few games now that I want to store away so my daughter can play later. Here’s a few of the games I’ve picked out for my daughter, along with the lesson I think it teaches. Mind you that it’s going to be a few years before I let her play most of these.

Demon’s Souls – always keep your guard up

There were a few points early on in that game where I was reamed and killed with one or two shots from a sneaky demon because I was strolling around observing my surroundings with my shield and sword by my side. I learned quick. Any player who treads new territory without holding R1 is a fool. The lessons I learned in this game stayed with me while I played the sequel, Dark Souls, and I was better for it. This is generally good advice for somebody who is about to embark on their own adventures in the world: stay on your toes and don’t leave yourself vulnerable.

Pixel Junk Monsters – Time not dancing is time wasted

In this game, you build towers to kill dudes who are coming to eat your babies. If you stand in an area occupied by a tower, you dance to give that tower experience so it can be upgraded. If you’re sitting in the middle of nowhere, you’re wasting valuable dancing time that can be strengthening your towers (unless you’re dealing with the hive tower, but you shouldn’t be standing away from a tower for long). And who doesn’t love dancing? It’s fun, good exercise, and you don’t even have to be good at it for those two features to hold. If you are not enjoying what you’re doing, you’re wasting your time.

Resident Evil – Choose your battles

You have two shotgun shells, five 9mm bullets, and no green herbs. You walk up a set of stairs, see a zombie staring at a wall in a far corner of the room. There’s two other zombies doing the same in a hallway you need to travel, and you know that the door on the other end of that hallway is a door that leads outside. There are probably hunters outside, and they are mean sons of guns. That first zombie can stand there all the doo dah day. He’s not going to bother you. The other two zombies are slow, and you can weave through them no problem. Ignore them, because those two shells will likely be just enough to keep the hunter at bay while you dart through the garden. It’s just not worth taking on every challenge full force, and sometimes you just have to ignore them. Save your energy for the bigger challenges that really matter.

Mega Man – Practice makes perfect

I dare say that in every Mega Man level, something will kill you without warning on your first time through. The game is designed that way. There are some surprises that are meant to knock you down, and you just have to learn where they are and be ready for them next time. Each time you die, you try again and aim to get just a little bit further. Eventually, you build the muscle memory to get through the levels and you can almost perform them without thinking about it or maybe even without looking at the screen. Repeat something enough times, and you will become an expert.

Once Upon a Monster – ???

I don’t know if there’s a main lesson in this game, I haven’t played it yet. But I will get this game because it requires two players, a parent and a child, and puts them together in an ultra co-op mode. In many cooperative games, you can get through with another player without ever really interacting with them. You can just play alongside them (excluding the harder difficulty modes). But Once Upon A Monster naturally has you play together. It’s beautiful really. I can’t believe this game would fail to impress me, because Tim Schafer is dreamy and everything that comes from Double Fine is great.

Braid – ???

I have a tough time pinning a lesson on this one. I’m fascinated by this game because it tells the story through the game mechanics, and that’s just awesome. But, it’s also very abstract and that means its moral can change depending on how you interpret the game. I myself thought Tim was a young man trying to make his mark in the world, and the princess was a metaphor for The Big City. Others thought it was a nuclear bomb. Whatever it is, this game is a shining example of the power game designers wield, and how to use it for great justice. If my daughter is a gamer, she will play this game.

This is only the top of my list, there are many more. I hope these help show how both core game mechanics and tiny details can be used to teach a lesson.

* I am not a doctor. If a doctor tells you that your child needs this dosage of Tylenol, do not cite me as an argument against it.

A new type of touch screen joystick

Alex Moore — Tue, 15 Jan 2013 22:30:50 +0000

This post is going to be a little different to my usual posts, as it’s going to be more akin to how I write a document for a mechanic. The mechanic in question is a challenge I set myself: try and design a touch based joystick that, in a single stick, provides the functionality that usually requires two. I’ve decided to call this task “Single Joystick”.

Read on to see how I went about finding out if such a this is possible…

High Concept

For lots of types of games, touch screens are a fantastic input device. They currently don’t offer any sense of tactile response, though I’m sure that’s coming in time. Today, the games which are most successful on our phones and tablets are ones that use touch screens to their advantage. There is demand, however, to put games that have been very successful on PC and console straight onto our phones and tablets; by which I refer to first person and third person action games.

To date, this has often been done by drawing two virtual joysticks on the screen, one in each corner. These react in a similar fashion to how the physical joysticks do on a traditional console controller, with the left stick controlling XY movement and the right stick controlling rotation and looking up and down.

This approach has two main flaws:

The amount of screen space required for the joysticks
To hold the device so you can use both thumbs, you effectively remove the ability to use any other fingers at the same time.

In addition, such approaches:

Assumes that the user is familiar with twin joystick controls from console games, which requires a significant amount of skill and dexterity.
Often does not offer support for left handed players
Does not offer support for gamers with reduced mobility

I think these are all fairly well known, but it’s still an accepted approach because there doesn’t seem to be a better alternative.

Or is there?

Initial Concept

The initial concept is to combine moving and rotating into a single joystick. This is something that is only possible on a touch-based device, as the user can be very precise about exactly where they are touching the screen.

The joystick is made up of 3 core elements:

The move gizmo
The rotate gizmo
The nub graphic

Arranged as such:

The Move Gizmo

The move gizmo works the same as in an ordinary two-stick system:

The centre of the graphic is stored as (x,y).
The radius of the gizmo is known (stored as a variable, InnerRadius, to allow for tweaking)
When the user touches the screen, a check is done to ensure they are inside the move graphic.
If they are, then the touch location is stored as (x1, y1):

The amount to move the player by is simply calculated by:
- Position.x = (x1 – x) / InnerRadius
- Position.y = (y1 – y) / InnerRadius
This returns a value between 0 – 1, which gets applied directly to the player movement code per frame, where a multiplier is added as a scalar to allow for faster movement if required.
Because of the size of the stick, there is unlikely to be any need to apply an acceleration curve to this movement. There may be a need to implement a dead zone, to be decided in testing.

The Rotate Gizmo

The rotate gizmo fully encompasses the move gizmo, and has an internal size stored as the variable OuterRadius. The rotate gizmo is only active when the user’s touch position is between InnerRadius and OuterRadius.

To calculate the rotation amount, the gizmo works off angles rather than absolute location of the touch. This is very different to how rotate joysticks usually work.

At the point where the user’s touch enters the rotate gizmo, the position is stored and a vector is calculated from that position back to the centre of the joystick (x, y):

The user is now required to keep their touch within the rotation gizmo, and the position they move to creates an additional vector, current vector. The angle, θ, between this new vector and the original determines how much the user rotates per frame:

If the user rotates clockwise past the initial vector then they turn clockwise, and likewise turning anti-clockwise past the initial vector turns them that way.
There may need to be a maximum rotation speed, to be determined with testing.
If the user’s touch moves outside of the OuterRadius then the joystick deactivates and resets.
If they move their touch back inside the move gizmo, the rotation stops and movement resumes from where the updated touch resides.
Releasing the touch or moving outside of OuterRadius stops all movement and rotation, and resets the joystick to its initial phase.
If the user’s initial touch is within the rotate gizmo then the user can look around on the spot.

The key to Single Joystick is this:

If the user starts touching inside the move gizmo and then moves into the rotation gizmo, (x1,y1) remains at the point that they ‘pierced’ the rotation gizmo. This allows for circle strafing on a single touch.

Nub Graphic

This is just a simple graphic that moves to reflect the current touch position, assuming the user is inside the OuterRadius.

Vertical Look

The one downside of the Single Joystick is that it doesn’t have any support for controlling vertical look. There are several possible solutions to this by using additional touches or lock-ons, but for initial prototyping a simple height check slightly in front of the player camera (2m) is performed. If the check is lower than the player height then make the camera look up, and likewise look down if it is higher:

Testing the theory

Writing a game mechanic requires a leap of imagination and, for me, often involves a great degree of holding the input device, closing my eyes and moving my fingers trying to imagine myself playing the game.

This is great, and helps iron out any major flaws with a specification. But nothing beats actually playing it.

As I’m currently freelance, access to programmers is limited. However, access to game engines isn’t: especially with Unity available for free. I already had a licence for iOS deployment, and once registered as an Apple developer I could very quickly build projects and get them running on my iPad. Note: this is not to say the mechanic is only devised for Apple devices, but purely to say that’s what I had to hand and thus it made sense to use it.

So, here’s a prototype of this mechanic running. The stuttering is due to my capture method off my iPad, rather than framerate – as you’d hope / expect, it runs at a solid 60fps currently.

As you can probably see, the concept works to a decent degree but isn’t yet completely robust. I’ve been playing with this for about a week now, tweaking bits here and there. If you’ve read everything above, you’ll notice no mention of the icon that lights up when the touch is inside the rotate gizmo: that came about simply because you can’t see the nub (or, in fact, half of the background) when you’re actually touching it.

What next?

There’s quite a few more improvements to make, but I thought that this project had hit a point where I should write about it and see if anyone else has any feedback. I’ve tested it on a few people, with varying degrees of success – everyone needs to be told how it works, so at the ‘intuitive’ point from the brief it’s not quite there yet.

I’ll hopefully do a follow up post in a few weeks with improvements I’ve made. For now though, hopefully I’ve started to set designers everywhere a challenge: find a way to make touch screen joysticks better.

What Happens Next? (2013)

Alex Moore — Wed, 09 Jan 2013 14:08:59 +0000

In June 2011 I wrote an article called What Happens Next?, where I had a quick go at peering into a crystal ball and predicting the future of video games. Recently I did a talk for Social Media Cafe, discussing similar thoughts with 18 months worth of additional future taken into account. Parts of what I discussed in the original article still hold true, and I elaborated upon them in my talk. This article is an amalgamation of the original article, the talk, and a few more bits that I realised I could have discussed at the time.

Before reading on, please understand that my definition of the AAA gaming model is thus:

A game that you go to your favourite store to buy for £45 / $60, take home and play on a console or PC in the comfort of your own home. You expect that there’s a single player component that will last you at least 8 hours, if not quite a bit more, and a multiplayer component that will be polished and fun.

With that in mind:

Is AAA is dead?

Eighteen months or so ago, this wouldn’t have really even been considered. There were signs that the model was struggling, but sales were still plenty high enough on the big hits of the year and there were plenty of rumours that Sony and Microsoft were working away on the next consoles – a sure fire way to inject new life into the fire.

Today though, it seems to quite a few people’s minds. Sony and Microsoft haven’t unveiled any new hardware yet, and it’s costing more than ever to produce a current generation AAA game, which is against what has happened in the past. I’ve previously mentioned that making AAA games is hard, and nowhere is this more evident but in a simple table:

These are averages – naturally there are games that go against the grain, both up and down. If we take the middle-of-the-road for all of these, it took 19 man years to make a PlayStation 1 game, back in 1995. Fast forward to 2001 and it took 124 years to make a PlayStation 2 game. Today, it takes 420 man years. That’s 22 times longer than it took in 1995, yet games still cost £45 / $60 at retail. Thanks to mobile gaming, people are starting to become unwilling to pay £45, even though in real terms they’re cheaper than ever. In fact, people don’t even want to pay 69p – a lot of people expect entertainment for free. Currently, no one expects Assassin’s Creed III on their mobile phone for 69p, but, as the speed of mobile phones increases, so do customer expectations.

The number of copies you’d need to sell at 69p to make a profit on a game that’s taken 420 man years to make is astronomical, and highly unlikely to happen. Something has to change.

Technology will save us

The two most time consuming elements of making a AAA game are making the game engine and then creating the content to go in. A modern day game engine has a lot of boxes to tick:

Deferred Rendering
Texture and Geometry Streaming
Online Multiplayer Support
Parallax Mapping
Blended Animation System
Rigid Body Physics and IK Solutions
Motion Blur
Full Body Rig
SSAO (Screen Space Ambient Occlusion)
And on..

Not all of these boxes have to be ticked to make a game, but to get anywhere near the visual quality expected by the audience and gaming press your engine is going to have to do a lot. Technical accomplishments will only get you so far of course – your game needs solid art direction and high quality assets, so you’ll also have to spend a lot of money on high calibre artists, animators and sound designers. When so much money is being spent, it’s rare for a team to be afforded the ability to experiment too much with game mechanics: instead it’s safer to tread the well worn path that successful games have already laid.

As a games designer, this feels wrong to me. Thankfully, it feels that the tide is turning.

A lot of this comes down to the sheer power of hardware becoming available. Right now a mobile phone has more processing power than a top of the range desktop from 6 years ago. That is, officially, bonkers. And it’s only going to get even more so – in another couple of cycles mobiles phones will easily be as fast as current generation consoles, and they’ll continue to get faster.

Speeding up engine development

This power starts to give game makers a little bit of breathing space, because it begins to reduce the requirement to squeeze every last ounce of processing power out of the hardware to get the same results. That, in turn, gives you the ability to make more use of pre-written software libraries. In the past, these might have been avoided or rewritten to be tailored specifically to your exact needs and any additional code removed. As well as software libraries, CPUs and GPUs continue to advance and provide specific processing units for performing some of the heavy work on the chips themselves, such as physics, and this frees up valuable coding time down the line.

In addition to on-chip improvements, APIs, software libraries and even off-the-shelf game engines continue to advance at a good rate, as do protocols to bring about a level of standardisation between devices. In fact, these are the two areas that I think can bring about the biggest change in how game engines are made. This year looks like it’s going to be all about new hardware, with Ouya, a Steambox, an nVidia console, GameStick and a handful of others, and that’s before even considering that the Wii U is only just out and Microsoft and Sony might have something new this year too. That’s a lot of potential fragmentation, and I’ve not even mentioned mobile phones or tablets…

How about, instead of buying a specific device to play games on, you just run the platform of your choice on whatever you have to hand. Think of something like DirectX, but imagine it as a whole console layer. Imagine if you just wrote your game for Steam, and let the APIs, libraries and protocols do everything else to make sure that it runs on the device in your hand, from providing the input layer to rendering the graphics. This is a few years ahead, but I think it will happen. For instance, it wouldn’t matter how many buttons a device has, as long as it adheres to the interface protocol. That layer may be physical buttons, a touch screen, a motion sensor or something else entirely. The device would deal with how the user interacts with it, and push that information to the game as a series of standard inputs.

This might sound slightly ridiculous, but this is already how browsing the internet works. The websites you visit don’t know if you’re using a mouse, a touch screen or a TV remote. They don’t need to: the browser on your device deals with the input layer, and also does all the rendering of the data that gets sent back. Could this be a future for game engines too?

Asset Creation

Shorter term, asset creation is going to benefit greatly from more computing power. As with the engine, the requirement to optimise a model to within an inch of its life is reduced if there’s more processing power and memory available. Equally, as engines continue to become more sophisticated, they can offer perfectly acceptable optimisation of models on the fly. The requirement to create multiple LODs reduces and, soon, should disappear. This will save a lot of time.

In addition to this, the tools that create the content will continue to improve. It makes full financial sense: if you put resources into better tools, you need fewer people using them. If, for instance, six coders working on tools could reduce your art team by twenty, surely that’s the best thing to do?

The ability to scan objects in real time and generate models is becoming a reality. Faceshop for Photoshop can take a 2D image and create a 3D head from it. The results aren’t professional quality yet, but in time tools like this will get better and better.

Finally, the way we interact with computers is constantly evolving. There are a lot of UI developments just round the corner, some of which could help massively with content creation. Imagine sculpting a 3D model using a LEAP controller, or sketching out a 2D layout on PaperTab and having a tool turn that into 3D for you.

With all these developments, and many more, combined:

Gameplay becomes king

If development teams aren’t focused on ticking engine feature boxes, and art teams can produce high quality art in a fraction of the time currently required, then the focus will hopefully shift more onto just making great games. This has already been kickstarted by the mobile and independent scene, and it’s where AAA development is heading. But wait a minute… didn’t we say that AAA might be dead?

Well, yes and no. There will always be a market for high budget entertainment, and there will be developers that will cater for that. They may have to seek a different funding model, possibly a combination of a kickstarter campaign, a traditional publisher deal and in-app purchases. With development cycles being shorter and it being sold as purely a digital download, it should be possible to sell a game for less and still make a profit. See the Steam model.

Will technology have an impact on the type of gameplay? Motion controllers and Kinect opened the door to different types of gameplay interaction, and it’s hard not to assume that new interfaces will bring new gameplay experiences. New controllers are great, but inherently create fragmentation. If you look at mobile games, the ones that are successful aren’t necessarily breaking new gameplay ground. In fact, if anything, they’re relying quite heavily on paradigms that made games successful in the 1980s and 1990s. They are very highly polished, and very well defined – distilled almost down to a few core mechanics. Interestingly, too, is that mobile and independent games often offer a greater challenge than AAA games. Getting 3 stars on every level in Angry Birds is not easy at all, yet people who can’t finish the first mission in Halo will happily sit there for hours until they get them. I still haven’t managed to complete a game of Faster Than Light, but I’m having a lot of fun trying. I could list 50 such examples, but this article is already far longer than I originally intended.

The point I’m trying to make here is that there will be a meeting point of polished gameplay experience that looks as good as the top budget games of today: but for a fraction of the development cost. At the point that it doesn’t cost tens of millions to make Call of Duty, people will have room to take bigger risks with the gameplay.

Wrapping it up

Quite a bit has been made of Ingress, a location based game on Android. You go to a real location – predefined as an ‘energy source’ – and hack in. You’ve already decided side you’re on. Currently, I’ve heard (I haven’t tried it myself), it’s not very engaging. It’s currently in beta and an interesting experiment, and a sign of potential directions that some games will take. Coupled with Project Glass (aka, Google Glasses), it’s not hard to imagine that you could be playing games all the time, as you walk down the street or go shopping.

The technology to synchronise data automatically between devices is already with us, as is the ability to connect your mobile phone up to your TV – effectively turning it into a Wii U controller (iPhones achieve this via Apple TV, whereas other manufactures are looking to incorporate Intel’s WiDi protocols).

All of this means that we’ll be able to play games wherever we are, whenever we want. Whether or not that’s a good thing… I’m not sure because it doesn’t take into account why we play games. Partly we play them for a distraction, a way of turning the mundane into something more. But one of the biggest pleasures of any leisure activity is making time for it. You may be meant to be doing something else, but you’ve decided that for the next few hours you’re going to play a game. This is important, and isn’t going to go away.

The great thing is, there’ll be a huge amount of great games for you to play on whatever you’ve got to hand.

Networking Tips and Tricks Part 1

Andrew Meade — Wed, 09 Jan 2013 01:26:20 +0000

Wow, it’s been a long time since I’ve posted to AltDevBlogADay! Life has been massively crazy for the past six months or so, and I’m nearing the end of my time in school. In almost 30 days, I will be a college graduate! In the midst of our final project sprint, I was asked by a friend of mine if I ever thought about doing a blog post about networking.

Now, some of you may be wondering who I am to tell you about networking, to whit I like to reference Hitchhikers Guide to the Galaxy by saying “I’m just this guy, you know?”.

In my three years as a student, designer, contractor, and employee, I have picked up a few tips and tricks that may or may not appear to be common sense to you. Yet I consistently get people coming to me saying how outgoing I am, or how it’s insane how much I network, and I’m here to tell you a dirty secret. Networking, to me, is one of the hardest things I have ever done.

I’m actually a pretty huge introvert when left to my own devices, so the physical and mental act of getting out there and meeting people is just daunting. I share this with you because maybe you’re the same way, and feel like I’m being too casual with my advice, but the plain and simple fact is, Network and you will make friends and reap dividends. I have no advice for the introvert short of “get over it and do it”, because that’s how I pull it off. But enough about me, let’s go over some networking tips. This first part will be about your online presence, and some best practices for both online and in person, and then the next part will be about in person and online follow-ups!

Rule #1: Think Before You Post

So we all know that our online presence is a huge deal nowadays. Make one off-color comment that has the potential to be misconstrued, and you’re hosed. An actual example I saw a few months ago comes to mind.

Designer A shares a picture he thinks is funny on social media. Designer B thinks it’s tacky and wrong. Designer B loses respect for Designer A. Designer B will remember that if he is ever in a position to hire Designer A.

Over a simple laugh, Designer A is potentially out a contact, a positive relationship with an acquaintance, and a gig. Awful, yeah? Well it happens all the time.

It may be common knowledge, but I must say it. Think before you post.

Rule #2: Get Your Own Dot Com (And Maintain It!)

This is probably the hardest piece of advice I can give, because maintaining your site can be a pain in the butt. I could go on about your site for an entire post so for brevity I’ll try and boil it down into a few bullet points.

Make the URL easy to type and relevant. Your name is always a good bet, provided it isn’t far out in how it’s spelled.
Don’t put up your Works in progress. A recruiter may only look at your resume for 30 seconds. Do you want them to see polished content, or half finished content? Worse, do you want them to see half finished content that they don’t read as WIP, and assume that it’s your polished content? **shudder**
Put your contact info on every page. EVERY. PAGE. That’s name, number, email address. I know that some people like to break up their address as “my name at domain dot com” to thwart spammers, but come on. Again, people looking at your site have a lot of stuff to look at. Let them click on your email address, and have it come up in outlook. The less energy they exert to contact you, the better your chances.
Have a tab for your resume. In that tab put your resume down online in plain text, and then provide download links for your resume in both Word and PDF. Give them a choice in how they can view it.
Update it. All the time. Think your site is good enough? It isn’t. Tweak it. Tweak it again. Never stop being great.

Finally, there is an amazing keynote from GDC by Jacob Minkoff from Naughty Dog. It is hands down the best 50 minutes you will ever spend on your portfolio. Watch it, learn from it, and follow it.

Rule #3: Social Media is a Tool Chest.

A lot of people use social media in a blanket way. Maybe Programmer A uses every account they have to share lolcats. Maybe Programmer B uses every account to be political. In my opinion, that’s paying a disservice to the strengths and weaknesses of each type of social media. Below is a small list of my social accounts, and how I handle them.

Twitter

Twitter is how I make new friends. I try to be interesting, share relevant and fun “geek” links, and get into thoughtful and informative discussions. I can approach anyone I want on Twitter, and be seen. When I post to Twitter, I am posting to an audience of devs I don’t know, recruiters, colleagues, and friends. I am still myself. I still curse a bit. I am still opinionated, but I mold it for a certain audience. Everyone I meet at a convention also gets followed on Twitter, provided it’s on their business card.

Facebook

Facebook is my next tier of friends. In Facebook you get a glimpse into more of my private life, my photos, my politics, and my overall thoughts. In here lurk family, longtime friends since childhood, and colleagues. If I add you on Facebook it means that we have had numerous conversations, and we both seem to agree that we are rather friendly. Sometimes I will make an exception and add someone on Facebook cold, if they don’t have Twitter and I think they are an especially cool cat.

LinkedIn

Here I keep every single thing I’ve ever done professionally. I keep it current, and tweak it bi-weekly. I connect with anyone that has similar interest to me, and I likewise get the same connection requests from others. I am constantly expanding my network and meeting folks. I also hook my Twitter feed up to LinkedIn.

LinkedIn has a highly underused tool for making connections. I use it all the time. If I see someone I want to be connected with that’s out of my network, I’ll find a friend that is connected, and ask for a recommendation. Here’s the thing, LinkedIn is for one thing and one thing only. Networking. Don’t feel like you’re being ruthless by pointedly networking. The only caveat I can really add is make sure you don’t try and jump too far out of your area when asking for connections. For instance, I wouldn’t ask my buddies at Ubisoft for a connection with a CEO or something, because for one it’s not relevant, and for another it’s level-jumping, and I would be putting my friends in the awkward position of saying no, or feeling uncomfortable and doing it anyway. You never ever want to make anyone feel uncomfortable. Ever. With that in mind, I’ve gotten tons of great contacts just by asking for an introduction. Likewise, I am constantly recommending contacts (but more on that in a bit).

Klout

Klout is a great site that hooks up to your various social media and tells you, in a numerical way, how influential you are within your circle on a number of topics. It’s a great site to check out if you’re interested in seeing what sort of footprint you’re leaving on the internet. If your biggest sphere of influence is Vintage French Nude Lithographs, then maybe you need to reassess. Or not. That’s actually kind of awesome.

So that’s social media. You don’t have to follow my template, but remember that each site is a tool that can and must be used differently to be effective. Finally let’s look at a rule that bridges the gap of online and real life networking.

Rule #4: Don’t Be Selfish.

I mentioned earlier in the LinkedIn section about recommending contacts, and it’s so important that I thought I’d give it its own section. This is good for both online and in real life, so take heed.

If you are to become a crazy whirling dervish of networking, you’ve gotta spread the love. You have to share. If you don’t pay it forward, then it stops with you and doesn’t go any further. That’s no good for anyone, and is just plain selfish. When you see two people with common interests, introduce them! There is a classic method of introduction which is a bit formal, but I absolutely love. It goes like this:

*Talking with Artist A, when Artist B walks up*

Me: Oh, hey Artist B! Have you met Artist A?

Artist B: No I haven’t.

Me: Really? Well let me introduce you. This is Artist A. Artist A is an Environmental Artist at Treyarch. Artist A, this is Artist B. Artist B is a Concept Artist at Epic. Now, Artist A and I were just talking about (Blank), what do you think, Artist B?

Boom! Introduction! What you have done is repeated names at least twice, given relevant information, aligned interests, and included everyone in the conversation. Now gently steer the convo until you feel you are no longer needed, then excuse yourself. You have just connected two professionals, and potentially started a long and healthy acquaintanceship.

The thing is, you should want to get your friends gigs. Your friends should want to get you gigs. If you hear about something that isn’t up your alley, pass it on to a buddy. Furthermore, don’t be a mercenary cock. There’s an awesome studio that a few of my friends have been INTENSELY courting for the longest time. I would love to work there, but why would I try to? First off, I don’t have the relationship these guys have built. It would be a waste of my energy with so many places out there. Second, it’s just a dick move. Let them get the gig unobstructed. Getting a gig is hard enough without your friends pulling a fast one. Besides, once your friend gets in, he can try and help you out. Win/Win. Now, if that same studio comes to YOU and says they want to interview you, for the love of god don’t turn it down. All bets are off at that point.

Come back later for my next part, where I talk about networking in person, and how it folds back to the interwebs!

C/C++ Low Level Curriculum Part 10: User Defined Types

Alex Darby — Sat, 05 Jan 2013 15:25:05 +0000

Hello again peoples of the interweb. It has been quite a while since the last one (probably even longer than the gap between part 8 and part 9) so I thought I ought to pull my finger out and get the next post in the C/C++ Low Level Curriculum done.

In the previous posts we’ve covered the structural aspects of the language: flow control, functions, and so forth; and so now we move on to looking in detail at user defined types in C/C++ (i.e. struct, class, and associated keywords) which I naively expected to comprise the bulk of this potentially never ending series when I started it. D’oh!

Before we start, dear reader, I’m going to assume that you’re the kind of person / recently self aware google web trawling AI entity who likes to understand your jargon terms and so I will be including appropriate links (probably mostly wikipedia or other ADBAD articles) where appropriate.

You may also want to read the previous posts in this series (though I don’t think this one will particularly rely on older posts) so, in case you missed them, here are the back-links for preceding articles in the series (warning: reading these might take a while…) :

Data Types and Enums

We covered fundamental and intrinsic types in the second post in the series, which also touched on the enum keyword. I deliberately didn’t cover the use of the keywords struct or class in this post, but we did cover some facts about the behaviour of values defined using the enum keyword (i.e. that it was up to the compiler to decide what intrinsic type to use to represent each enumerated type you declare, based on the range required by its values).

Helpfully, the C++11 standard made some sweeping changes to the behaviour of enums; amongst which was the ability to specify the the fundamental type used to represent the values of each enum. Tasty.

Mentioning this welcome change is the extent of our discussion of enum, so let’s get on with starting to look at struct, class, andunion.

Thankyou Visual Studio Devteam

If you have been really paying attention to the older posts, you might remember that I mentioned some undocumented (and unsupported!) command line options for Microsoft’s Visual Studio C++ compiler which can be used to print out the memory layout of data types defined using the struct, class, orunion keywords.

These secret compiler options are /d1reportAllClassLayout which reports the layout of all classes in the current project, and its more user friendly sibling /d1reportSingleClassLayoutxxx (where xxx is a string used to do a substring match against classes that you wish to have reported).

I will be leaning pretty heavily on this compiler for the next few posts, so we may as well cover how to use it. It definitely works in VS2010 and VS2012; it even works with the Express versions. Woo!

Here’s where you type in the command line option in the property pages (n.b. this is the ‘single’ version and matches any class or struct with the string ‘Test’ in its name):

Output from /d1reportSingleClassLayout

So far so froody.

Now, it’s about time we looked at a code snippet defining a simple POD struct (POD types being the simplest cases of aggregate data types) and the output produced by /d1reportSingleClassLayout when we build it…

#include "stdafx.h"
 
struct STest
{
    int iA;
    int iB;
};
 
int main(int argc, char* argv[])
{
    return 0;
}

When we compile this with the fancy secret compiler switch, as expected we find an extra bit of information in amongst the usual Visual Studio compiler’s output:

1> class STest size(8):
1>   +---
1> 0 | iA
1> 4 | iB
1>   +---

Hopefully this should appear pretty much self explanatory to you, but in case it doesn’t – rest assured we’re about to look at it in a little more detail.

The first line contains the name of the class and its size in bytes – STest is a struct, but it is reported as a class – don’t worry about this for now.

The struct‘s name contains the string ‘Test’ which is the substring we specified to match against in the compiler option in order to get class layout information.

The rest of the information details the member-by-member memory layout of the struct organised by the name of the data members – the number at the start of the line is the memory offset in bytes of that member relative to the start of the struct.

The first thing to note is that the member variables are laid out in memory in the order specified in the class declaration.

A guarantee is given in both the C and C++ language specifications that memory address of each member will be higher than that of the one declared before it (see this post on Stack Overflow for more detail of the wording).

In the case of STest the first member iA is at an offset of 0 bytes from the start of the struct; and the second member iB is at an offset of 4 bytes from the start of the struct.

Importantly (by doing a little maths with the offsets and the size of the struct) this also tells us that the size taken up by iA is 4 bytes, and the size taken up by iB is 4 bytes – since sizeof(int) == 4 this matches up with what we would expect.

Accessing the members of a struct in assembly

We all knew this was coming, right?

Woo! I know you all live for hexadecimal numbers and assembler mnemonics.

As always, the main thing I want you to take away from this is not so much the understanding of the specific assembly code itself (though clearly it has its benefits…), but more of a generalised appreciation for the combinations of assembly instructions that ‘smell like’ the compiler accessing the members of a struct or class.

Getting used to the assembly level ‘smells’ of the various high level constructs in compiler generated assembly code will enable you to find your bearings much more quickly in code you see in the disassembly window, and – most importantly (assuming that you are lucky enough to have a valid callstack – and, like a sensible person, you have symbols for your release build) – you should quickly develop the ability to work out which bit of the high level code corresponds to the assembly you’re currently looking at. Win.

Here’s a code snippet that accesses the data members of the struct we just defined:

#include "stdafx.h"
 
struct STest
{
    int iA;
    int iB;
};
 
int main(int argc, char* argv[])
{
    STest sOnStack;
    sOnStack.iA = 1;
    sOnStack.iB = 2;
 
    STest* psOnHeap = new STest;
    psOnHeap->iA = 3;
    psOnHeap->iB = 4;
    delete psOnHeap;
 
    return 0;
}

Before we look at the disassembly we should explain a little about the snippet.

Two instances of STest are created:

sOnStack on the Stack – i.e. automatically allocated by the compiler as a local variable
psOnHeap on the Heap – i.e. dynamically allocated.

The reasons for doing this will become clear once we’ve inspected the assembly.

Aside: technically the area of dynamic memory managed by new and delete in C++ is called the Free Store, but almost everyone calls it the Heap. I’m pretty sure this is because the dynamic memory in C managed by malloc and free has colloquially and historically been known as “the Heap”, and a lot of C++ implementations define new and delete using malloc and free (and most if not all used to).

So here’s the disassembly generated by the VS2010 debug compiler:

    14: STest sOnStack;
    15: sOnStack.iA = 1;
00A01269 mov dword ptr [ebp-8],1 
    16: sOnStack.iB = 2;
00A01270 mov dword ptr [ebp-4],2 
    17: 
    18: STest* psOnHeap = new STest;
00A01277 push 8 
00A01279 call 00A010F5 
00A0127E add esp,4 
00A01281 mov dword ptr [ebp-54h],eax 
00A01284 mov eax,dword ptr [ebp-54h] 
00A01287 mov dword ptr [ebp-0Ch],eax 
    19: psOnHeap->iA = 3;
00A0128A mov eax,dword ptr [ebp-0Ch] 
00A0128D mov dword ptr [eax],3 
    20: psOnHeap->iB = 4; 
00A01293 mov eax,dword ptr [ebp-0Ch] 
00A01296 mov dword ptr [eax+4],4

Looking at lines 3 and 6 (and remembering what we learned in post 2 about how variables in memory are accessed in assembly); we can see that both sOnStack.iA and sOnStack.iB are being directly accessed by their memory addresses as offsets from ebp ([ebp-8] and [ebp-4] respectively).

Looking at lines 15-16 and lines 18-19, we can see that psOnHeap.iA and psOnHeap.iB are being accessed differently.

Since this is different to what we have seen before, let’s break it down a little:

For each of these assignments, first the pointer psOnHeap (i.e. memory address of the instance of STest created at line 7) is loaded into eax (line 15 and line 18), and…
… then the member is accessed via the memory address stored in eax (line 16 and line 19 – via [eax] and [eax+4] respectively).

In particular, note that when STest::iB is accessed (at address [eax+4] - line 19) an 4 byte offset is added, which is exactly the offset that the output from /d1reportSingleClassLayout gave us.

Hopefully it should now be pretty obvious why the instances of STest are accessed differently like this – and by extension why I showed code accessing an instance on the Stack and on the Heap (via a pointer):

When an instance of a user defined type is on the Stack, the compiler is in charge of where the instance is stored (relative to the stack frame); and so it can access its members by their direct offsets within the stack frame.
When an instance is stored in a memory location that is not known at compile time (e.g. accessed via a pointer) the compiler can’t do this and has to access it via offsets from the instance’s base address (i.e. the memory address the instance starts at).

NOTE: this is debug disassembly code, please do not attempt to infer anything about the relative efficiency of Stack vs. Heap memory from this! As far as I am aware, on every machine I’ve ever used Stack and Heap are both stored in the same memory and accessed via the same physical systems so in terms of theoretical minimum access speeds Stack == Heap.

What about class?

The short answer to this question is that there is no difference whatsoever between classes and structs at the implementational level of C++.

The long answer is that, in the C++ language, struct is actually a special case of class with one specific difference – for classes any unsupplied access specifier (i.e. public, protected, or private anywhere in the type declaration) will default to private, but for structs it will default to public.

That’s it. The only difference. Honest.

Access specifiers are ultimately just language level syntactic sugar to allow us to control the way our classes are used; under the hood struct and class are implemented the same way – even with regards to stuff like inheritance and virtual functions.

If you do a search and replace of struct for class in the snippet and add a public: to the top of each class declaration (so it compiles) you will get the exact same output from the class layout information and the same disassembly.

Union

As well as class and struct, there is another keyword that can define a type – the keyword union.

It’s not a frequently seen or used language feature, and so it’s all the more worthwhile discussing here because it can be very useful and its low frequency of use means that a lot of people don’t really know what it’s for, let alone how it works.

Let’s look at this with another example code snippet. This has had two new types added to it:

#include "stdafx.h"
 
struct STest
{
    int iA;
    int iB;
};
 
struct STestTwo
{
    int iC;
    int iD;
    int iE;
};
 
union UTestUnion
{
    STest sTest;
    STestTwo sTestTwo;
};
 
int main(int argc, char* argv[])
{
    UTestUnion* psOnHeap = new UTestUnion;
 
    psOnHeap->sTest.iA    = 1;
    psOnHeap->sTestTwo.iC = 2;
 
    psOnHeap->sTest.iB    = 3; 
    psOnHeap->sTestTwo.iD = 4;
 
    psOnHeap->sTestTwo.iE = 5;
 
    delete psOnHeap;
 
    return 0;
}

Compiling this with /d1reportSingleClassLayout we get the following output for the layouts:

1> class STest	size(8):
1>  	+---
1>   0	| iA
1>   4	| iB
1>  	+---
1>  
1> class STestTwo	size(12):
1>  	+---
1>   0	| iC
1>   4	| iD
1>   8	| iE
1>  	+---
1>
1> class UTestUnion	size(12):
1>     +---
1>   0 | STest sTest
1>   0 | STestTwo sTestTwo
1>     +---

The first thing to note is that UTestUnion is the same size as STestTwo. This is exactly as one would expect.

The second thing to note is that both UTestUnion::sTest and UTestUnion::sTestTwo have an offset of 0 bytes within UTestUnion. Again, exactly as one would expect.

So, why is this the case?

The keyword union allows you specify multiple layouts for a chunk of memory. When we declare the union of STest and STestTwo within UTestUnion, we declare our intent to be able to treat the memory of type UTestUnion as either an instance of STest or an instance of STestTwo at our discretion.

This means that, within the type UTestUnion, an instance of STest and an instance of STestTwo exist overlaid on each other. Since the union can be treated as either type, this means that it must necessarily have the same size as the larger of the two types.

Let’s back this up by looking at the disassembly:

    29: psOnHeap->sTest.iA = 1;
0122127C mov eax,dword ptr [ebp-4] 
0122127F mov dword ptr [eax],1 
    30: psOnHeap->sTestTwo.iC = 2;
01221285 mov eax,dword ptr [ebp-4] 
01221288 mov dword ptr [eax],2 
    31: 
    32: psOnHeap->sTest.iB = 3; 
0122128E mov eax,dword ptr [ebp-4] 
01221291 mov dword ptr [eax+4],3 
    33: psOnHeap->sTestTwo.iD = 4;
01221298 mov eax,dword ptr [ebp-4] 
0122129B mov dword ptr [eax+4],4 
    34: 
    35: psOnHeap->sTestTwo.iE = 5;
012212A2 mov eax,dword ptr [ebp-4] 
012212A5 mov dword ptr [eax+8],5

There it is, clear as day :)

In case you’re not seeing it, here’s a quick breakdown:

we can see that psOnHeap is stored at [ebp-4].
(line 2) UTestUnion::sTest::iA and (line 5) UTestUnion::sTestTwo::iC are both being accessed directly via the value loaded into eax from [ebp-4] - i.e. at an offset of 0 bytes; the same as their offset within their respective types as shown in the class layout information.
(line 9) UTestUnion::sTest::iB and (line 12) UTestUnion::sTestTwo::iD are both being accessed via [eax+4] at an offset of 4 bytes from the value loaded into eax from [ebp-4]. Again, the same as their offsets within their respective types as shown in the class layout information.
(line 16) UTestUnion::sTestTwo::iE is accessed via [eax+8] – an offset of 8 bytes as specified in the class layout information.

A more ‘real world’ example of the use of union might be a data structure used in a vector maths library similar to the one below:

class CTestVec4
{
    union 
    {
        struct 
        {
            float x;
            float y;
            float z;
            float w;
        };
        struct  
        {
            float vec[ 4 ];
        };
    };
};

The above code declares a vector structure whose data can be accessed either via its components or like an array – e.g. CTestVec4::z occupies the same memory as CTestVec4::vec[ 2 ].

The code looks like it should be illegal but it isn’t – leaving all the names out is entirely deliberate, this defines an “anonymous union” which makes the syntax for accessing the union ‘less cumbersome’ (i.e. basically just less typing).

If you weren’t sure how union worked, or indeed what it was for, now you know :)

Surely we’re due a spanner in the works about now?

We most certainly are! Well spotted.

There is an incredibly important low level aspect of the way memory is laid out within classes and structs that I have deliberately skimmed over until now.

Consider the following snippet containing an innocent looking struct declaration:

#include "stdafx.h"
 
struct STestSpanner
{
    int     iOne;
    double  fdTwo;
    char    chThree;
    int     iFour;
};
 
int main(int argc, char* argv[])
{
    return 0;
}

When we compile this we get the following output from the class layout information:

1>  class STestSpanner    size(24):
1>  	+---
1>   0	| iOne
1>    	|  (size=4)
1>   8	| fdTwo
1>  16	| chThree
1>    	|  (size=3)
1>  20	| iFour
1>  	+---

What witchcraft is this!?!

We’ve added a char and a double to our struct - requiring a total of an extra 9 bytes (on win32 sizeof(char) == 1 and sizeof(double) == 8).

However, the total size of the struct has increased by 16 bytes – in addition to the 9 bytes we know we asked for, we have also added an additional 7 bytes of invisible ‘alignment member‘ fields.

What is going on? Padding – that’s what.

Padding!?!

As long as the layout of the data members of a type meet the ordering requirement of the language standard (as covered earlier) they do not have to be immediately adjacent in memory.

The compiler is free, encouraged even, to insert additional padding into your structs / classes at its discretion.

Why might the compiler wish to adjust the layout like this?

The short answer is: to optimise for speed of memory access for the intrinsic types stored in that structure.

The longer answer goes something like this…

On each platform, the various intrinsic types have different sizes and typically also different memory alignment requirements.

In the best case, accessing an intrinsic type with an alignment requirement from an incorrectly aligned memory address will cause the memory access time to be slower than usual (on x86 the cost is typically relatively small, but can be an order of magnitude slower on other platforms); and in the worst case can actually cause the CPU to crash (yes, really – on some platforms unaligned access makes the CPU freak out).

There are three separate factors at play in determining the size of STestSpanner:

The logical number of bytes required by its constituent types – this is the minimum possible size of the type.
The ordering of the constituent types within the class declaration.
The individual alignment requirements of those constituent types.

The compiler honours both the ordering of the constituent types and their individual alignment requirements, and this interaction determines the amount of padding bytes that get added.

Since they affect each other and are closely related, the distinction between alignment, packing, and padding often causes confusion:

Alignment is a constraint on the start address of instances of a type in memory
Packing is a constraint on the alignment of adjacent members within the memory of a struct or class
Padding is the bytes added within a class or struct to maintain its packing constraints

This Stack Overflow question has a good a discussion of the implications of unaligned access on x86 for those of you who are interested.

Sorry, why the padding?

As we have covered before, compiler writers are wily.

You must be able to declare an array of any type that you define, and so the memory layout of a struct or class you define must not only maintain the alignment constraints of its constituent intrinsic types within an instance, but also across an array of instances laid out contiguously in memory.

The simplest way to ensure this is to make the internal structure of the type adhere to the largest alignment constraint of its constituent types – and this means that the size and internal packing of any structure you declare will usually end up being determined by the largest alignment requirement of its constituent types.

In the case of our struct this would be double which has a default alignment of 8 bytes (at least under the Visual Studio x86 complier, it varies with other compilers), and consequently so does the structure STest – hence the 7 extra bytes of padding to take the struct from the 17 bytes ( 2 ints (8bytes) + 1 double (8 bytes) + 1 char (1 byte) ) of data we asked it to contain up to 24 bytes – i.e. the next size that maintains the alignment of double.

You should find that, no matter how you shuffle the members of STestSpanner around, you still end up with a 24 byte structure that includes 7 bytes of padding.

On the plus side, if we needed to add 3 extra char and an extra int into STestSpanner we would get that storage space for free as long as we put them in the correct positions in the type declaration :)

But what about all the wasted bytes?

The compiler knows what it is doing, and 99% of the time you should not worry about the wasted space.

Get a cup of tea and a biscuit and make peace with it – it’s not wasted space, it’s space invested in making your memory access more efficient.

However, you should worry about it a little – because it is entirely possible to cause the compiler to introduce padding into a type that is actually a total waste of memory.

Consider this struct:

struct STestSpannerTwo
{
    int     iOne;
    double  fdTwo;
    int     iThree;
};

which produces this class layout:

1>  class STestSpanner	size(24):
1>     +---
1>   0 | iOne
1>     |  (size=4)
1>   8 | fdTwo
1>  16 | iThree
1>     |  (size=4)
1>     +---

A quick look at the class layout above informs us that there is 4 bytes of padding between STestSpannerTwo::iOne and STestSpannerTwo::fdTwo; and another 4 bytes of padding after STestSpannerTwo::iThree.

The x86 intrinsic 64 bit float format used to represent double is 8 bytes long and clearly has default alignment of 8-bytes under Visual Studio’s compiler.

The constraints of our type declaration combined with the 8 byte alignment constraint for double have resulted in:

4 padding bytes after STestSpannerTwo::iOne and before STestSpannerTwo::fdTwo to maintain the alignment within the memory of a single instance of STestSpannerTwo…
…and 4 padding bytes after STestSpannerTwo::iThree to maintain the 8 byte alignment across instances of STestSpannerTwo that are contiguous in memory (i.e. an array of STestSpannerTwo).

However, we can also see that STestSpannerTwo::iThree is a 4 byte int and so it will fit into the first block of padding; eliminating the need for the 8 padding bytes.

Re-ordering the members by hand like this will save 8 bytes off the total size of the struct, and so we can see that – in this case – we can save 33% of the memory used by the struct basically for free – don’t take my word for it, try it!

whilst this isn’t something you should lose sleep over, you should now be able to see the benefit in always taking a second to consider the most appropriate place to insert a new data member into an existing type ;)

…but what if I really need those padding bytes?

Unsurprisingly, this being C/C++, it is entirely possible to ask the compiler to change its default alignment and packing behaviour.

This is usually accomplished by use of command line compiler options and/or compiler specific commands that are inserted inline in your code.

In Visual Studio, for example, there is the /Zp compiler option, and another 2 ways to affect the alignment of data structures and the packing of their members with compiler commands in the code itself __declspec( align( x ) ) and #pragma pack (x)). There may also be others I’ve never seen or used, but a quick search on t’interwebs didn’t find them.

For example, using #pragma pack to tell the compiler to pack STestSpanner to a 1 byte boundary like this:

#pragma pack(1) 
struct STestSpanner
{
    int     iOne;
    double  fdTwo;
    char    chThree;
    int     iFour;
};

Gives this class layout output:

1>  class STestSpanner	size(17):
1>  	+---
1>   0	| iOne
1>   4	| fdTwo
1>  12	| chThree
1>  13	| iFour
1>  	+---

Be warned!

Adjusting the packing of a type can (and probably will) break the alignment constraints of its constituent types – in the example above there are no ‘wasted’ bytes; but STestSpanner no longer honours the alignment requirements of its constituent types and so will presumably take significantly longer to access than it would do had we not fiddled about with it.

This means that when you ask the compiler to change the packing of a type you need to be very careful.

My advice is that – in general – you shouldn’t mess with alignment or padding unless you have a very good reason to; altering padding and alignment can get you into a whole world of pain, especially with bigger projects and when using code libraries – here’s a link to a post on the Visual C++ Team Blog that goes into it in some detail.

The decisions to change alignment or padding will typically come down to essentially platform specific trade offs based on data derived from run time profiling – balancing issues like data sizes vs. memory resource constraints, worst case access times of individual parts of the data, and issues relating to system cache sizes and alignments.

Summary

In summary here are the main points I’d like you to take away from this post:

The undocumented compiler options /d1reportSingleClassLayout and /d1reportAllClassLayout are awesome, and can help you to understand the memory use implications of code you write, as well as being very useful debugging tools
We now know that, when an instance of a structure is accessed via pointer, its members are accessed via an offset from the instance’s base address in the assembly, and …
… that, logically, we can use this in the disassembly view to work out which member is being accessed
The difference between struct and class. i.e. that there is no low level representational difference.
What the keyword union does and how it works
What padding and alignment are and some of their implications

Next time we will look at how (simple) inheritance affects this picture…

Epilogue: Debugger Trick 17a

Here’s an alternative way to find out the offset of a member within a user defined data type, a way that you can happily use in the debugger rather than having to compile the code.

This method works with the vast majority of debuggers I’ve used in the last 5 years or so on both PC and console; and it relies on the fact that the standard C style cast syntax works in watch windows (try it! It’s awesome).

One upshot of this is that you can use casting to calculate the byte offset of any member of any user defined type (this is also valid, and very useful, C/C++ code):

That code looks horrible, not to mention dangerous, but what it’s doing is actually very simple and totally safe.

We have seen that, when using a pointer to an instance of a type, the compiler accesses members of that user defined type by adding an offset to the memory address the instance is stored at (its ‘base address’).

Note that the values we’re seeing in the watch window are identical to those seen in the output from /d1reportSingleClassLayout.

Here’s how this works:

(STest*) 0 - tells the debugger to treat 0 as the value of a pointer to an instance STest. If you’re thinking “but 0 is NULL!” – remember that 0 is only ‘NULL’ by convention (in fact, on some consoles, 0 is a valid memory address and can be accessed…). In any case, this code isn’t accessing the memory – it’s just asking the compiler to treat 0 as the value of a pointer.
&(((STest*)0)->iB) - tells the debugger to calculate the address of STest::iB. Again, since this is just calculating an address and not attempting to access it it is fine.

This is possibly my favourite thing I have ever found out about debuggers, and has come in incredibly useful over the years :)

Video Game Industry Predictions 2013

Alistair Doulin — Wed, 02 Jan 2013 01:20:34 +0000

Well, the world didn’t end (of course) so it’s time to look to the next twelve months and predict what we might see in 2013. Below are my predictions for the video game industry in 2013.

Virtual Reality

I’ve long been a fan of VR and have been disappointed with the consumer level hardware up to this point. That all changed in 2012 with the announcement of the Oculus Rift. With the backing of John Carmack, it was hard for the industry to ignore this new product. It looks like VR has finally turned a corner and (thanks to the booming smart phone market) the technology is available allowing a cheap (almost) consumer grade VR headset to be mass produced. Developers will have most of 2013 to prepare for the consumer grade headset and I see a lot of time and research going into these preparations. Rather than gamers peering through a small (24-70″) window into our worlds, they can now be completely immersed in them. The current fascination with (stereoscopic only) 3D is one thing, but true immersive head-tracked 3D will take our games to the next level. While there are only a handful of games that support the Oculus Rift at this stage, by the end of 2013 I predict this to be in the hundreds.

Mobile/Tablet

I predict a large number of people (particularly indie’s) will make it big on mobile in 2013. This is still the direction I plan to head in 2013. While some people have moved toward social/Facebook gaming in 2011-2012 I see some of them returning to mobile/tablet gaming in 2013. With Microsoft entering the tablet market (and with a better mobile offering) I see this market growing substantially in 2013. The “get rich quick” attitude many had towards Facebook games has died off a little and I’m seeing a number of indie developers I know pivoting back to mobile rather than pure Facebook/social. I see the mobile/tablet market maturing in 2013 as many developers learn from their mistakes in the past and embrace the benefits of the platform (touch interface, always connected, etc).

Real Social Games

With Zynga shutting down a bunch of their games, I see the need for truly social games. Facebook and other social networks give game developers amazing opportunity for connecting gamers to their friends and others around the world. Rather than abusing these connections I see a switch to meaningful engagement. Games like LetterPress show the simplicity of Facebook to connect users rather than allowing users to bug their friends into getting them in-game items for free. True social games will take this simple connection and use it to enrich their game design. Many older gamers grew up playing together in arcades or lugging their giant computers to each others houses for a chance to play together. I’d like to see this same experience enabled by social games rather than the cheap “social” behavior we see currently. Like it or not, many developers go where the money is and the fact Zynga and the like are now falling on hard times shows the need for more than a superficial social connection if these types of games are to succeed.

No New Apple Product

I don’t see Apple releasing any significantly new products in 2013. They will continue to evolve their existing products, however I see the pace of of their evolution picking up. The iPhone 5 was a disappointment to many of the die-hard Apple fans I know. 2013 will be a real turning point for Apple as they forge new territory without Jobs at the helm. I’m sure much of 2012 had already been planned out but decisions like the iPad Mini show a new course is being set. I’d love to see a new product as innovative as the iPad, but I don’t see that occurring in 2013. Perhaps in a year or two we might see Apple move into the VR market, but I think they will wait and see the response to the Oculus Rift before making a large move like that.

Last Generation of Consoles

It’s looking like the next XBox and Playstation will be released in late 2013. I predict this will be the final “pure console” generation we see. As mobile, tablet and PC converge I think it’s logical that the concept of the console will disappear. With the current console cycle being closer to 10 years than 5, we can expect the next generation to be at least as long. With the current speed of innovation in the tablet/mobile sector, it will only be a matter of years before a portable device is as powerful as the next generation of consoles. The only difference then is the input/output mechanisms. Microsoft has already shown their Surface is compatible with a XBox360 controller and I see this trend continuing.

In the mid-term future, I forsee one, or maybe a couple, of devices powering most of our personal computing. This device will connect to the output device of our choosing to power our experience. On the road we will use the device itself (which will be around the size of a phone and/or tablet, depending on your preference). At work the computing device wirelessly connects to your mouse, keyboard and monitor(s). Playing games and watching movies it will wirelessly connect to a large TV or projector.

Summary

What are your predictions for 2013? Do you agree or disagree with any of my points? What would you love to see in 2013?

Almost everything you need to know to enter mobile game development on 2013

Raul Aliaga Diaz — Wed, 02 Jan 2013 00:20:10 +0000

This is a list of useful resources to help you refine, define, execute and possibly reconsider your decision to enter mobile game development this year. The resources are a little more inclined to the business and game design parts of game development, except for a couple of ones linking other lists.

The executive summary version: It is a very attractive and fast growing market, but the odds are stacked against small, new and indie game developers. Successful games are usually paid, greatly innovative and highly polished casual titles -specially when working with a good publisher or a known brand- or free to play titles that operate as a service, seeing little revenue for a span of at least six months after launch, adding new content and/or further polishing, balancing and optimizing for retention and monetization. The former approach can be a logical step for studios that already have game development expertise, a team already in place and experience working with publishers. The latter is the approach that might yield the greater rewards, but requires a mid to long term vision, and also a tight and strong combination of resources and execution speed to actually pull it off.

What is mobile game development? We’ll consider it as any game focused development for smartphones or tablets running Android or iOS. What does this market look like? Let’s look at some numbers and insights:

[Source: eMarketer]

Looks promising, but it’s not all good:

So, how to make sense of the opportunity? Is it reasonable? Affordable? Worth pursuing? Let’s start with a list of general projections for gaming on 2013 and summaries for 2012:

This is a list of studies, articles and slides worth studying bit by bit to get a clearer picture:

[Source: Lostgarden]

Next, a collection of thoughts from established players in the industry:

Assuming you have a clearer picture now, and you are eager to learn more, what would be good resources to learn about all this?

Business links

[Source: Business Insider]

Game Development links

How does Rage of Bahamut monetizes? Analysing a Top Grossing Game: Rage of Bahamut and How Rage of Bahamut Monetizes.
Clash of Clans – the Winning Formula
Free 2 Play and the Four Currencies
7 Ways to Fail at Free-to-Play
The Free-To-Play Triangle [Game Economics]
The Principles of Game Monetization
The Design of Free-to-Play Games, Part 2 (Part 1)
5 tips for boosting your virtual economy
Which Types Of Mobile Games Are In The Engagement Sweet Spot?
The Ultimate Indie Game Developer Resource List (includes technical tools and software)
The Big List of Game Making Tools (includes technical tools and software)

[Source: What Games Are]

Going live links

[Source: GamesBrief]

Last but not least, Blogs, People and Websites worth following.

Blogs

Deconstructor of Fun: Monetization in Social Games, written by “the evil monetization guy @ Rovio”.
Extra Creditz: Not exactly a blog, but lots of fun and thought provoking videos about gaming, game design and game development in general.
Dr. Serkan Toto: Japan Mobile And Social Games Consulting.
Andrew Chen’s blog: Great insights about startups in general.
What Games Are: Great for game design and production.
Games Brief: Special focus on monetization.
Dave Sirlin’s blog: Great game design blog.
Gameconfs: a directory of game industry events.
Indiebits: Great tips for Indie Game Developers.
Game Design Concepts and Game Balance Concepts.
Raph Koster’s blog: The author of Theory of Fun.

People on Twitter: All of them usually share good insights and links about mobile game development

Devs: Aki Järvinen, Tadhg Kelly, Shawn Foust, Will Luton, Michail Katkoff, Amy Jo Kim, Daniel Cook, Mike Acton, Jesse Schell, Tami Baribeau, Raph Koster.
Executives: Shanti Bergel, Jussi Laakkonen, Lloyd Melnick, Kevin Dent, Ben Cousins, Charles Hudson, Nabeel Hyatt, Giordano Contestabile, Scott Dodson
Press and thought leaders: Christian Nutt, Stephen Totilo, Leigh Alexander, Adrian Crook, Dr. Serkan Toto, Nicholas Lovell, Epona Schweer, Allen Varney, Nicole Lazzaro.

[Source: @BenjaminCousins]

Sites: Good for keeping up with trends, news, good games and stuff

I hope all this serves you as a starting point to learn more about your upcoming adventure on mobile gaming this year. Enjoy!

Bonus Tracks

Lessons from Europe

Claire Blackshaw — Wed, 26 Dec 2012 12:40:31 +0000

Often posts from a personal perspective are a contrite, self-promotion exercise of polishing the trumpets. They can also be an honest expression sometimes lost in the NDA red tape culture.

It is important to know why you do things, what drives and motivates you, bringing happiness and a sense of satisfaction. Though most importantly to admit when you made a gigantic cock-up and own up to being an idiot.

In my last post I requested contribution to a survey on why we create. The results were watered down when, by casual. non-formal inquiry of two large studios, I realised the results are not representative. In the inquiry I reminded myself that most people do not read industry press or actively participate in what I will term the “creators community”. Instead the bulk of our co-workers are content in their enjoyable job, which they still see as a job. They are happy, good on them, and many are talented and motivated individuals. Individuals I enjoy working with and who make awesome games. Though for completeness the results of the online survey are presented below.

The reason for the lateness of this post and the original self examination which triggered this exploration was an honest peeling away of my own fate and career. Since leaving Mere-Mortals I have not published work of which I am proud, that is not to say I haven’t worked on amazing things but they just will not see the light of day for a range of reasons, or when they finally do surface I shall not be credited.

Prompted by this situation a good friend whose creative ability and integrity I trust offered me an chance at a large publisher studio which sounded exciting. Much more exciting to me was a chance to turn my currently long distance relationship of almost 3 years into a no-distance relationship. So I packed my bags and moved to Europe with a healthy relocation package in the offering.

When I arrived the situation on the ground was very different. On day one I found out the project had changed from a traditional boxed console product to a free-to-play title. No-one’s fault, things change.

Now the “sensible” thing to do would have been hold out in Europe in a well paid job, cash in the relocation package and enjoy my relationship. This while creating a style of game I have morale issue with and not being able to express myself in the hopes of a relocation or change of circumstance. This is what I like to call “existing while waiting to live”, I have made a similar mistake in the past and refused to repeat it and possible watch my relationship and happiness fall to pieces over it.

So I swallowed what looks to be over £2000 in loss, cancelled my contract 3 days in on moral grounds and returned to the UK. There my previous employer welcomed me back in the most awesome way, way to go Climax. The only part of which I regret being the fact that my relationship is once again long distance.

On my first weekend back in the UK I found myself in my old flat living out of a suitcase, sleeping on the floor next to my heater cuddled in a blanket making a game in 48 hours on my chromebook with my phone providing internet. I was happy. It really doesn’t take much to create and make games and that’s what I enjoy doing.

My experience in Europe has taught me many things. Firstly that free-to-play, while a nuanced topic, will not resolve nicely if we stand on the sidelines. As opponents to it we need to actively offer alternatives to help shape it and lobby for a future without exploitation. Non-participation will not work when companies make the kinds of profits they are currently making.

The industry is secondary to the products it creates. It is a process and machine which aids the creation of games, but games can be created without it. To serve the industry as an end rather than a means to an end is raising the process above the product. Do not glorify the industry for its own sake.

Finally on a personal note I am an entertainer and a creator. I will make games, plays, art, things, watchmacallits and thingymajigs because I enjoy it. I want to enhance the world in some small way through entertainment or education. Yes I have bills to pay, making money is not bad, free-to-play is not evil but the games come first and should be a positive force, not a commercial parasite to maximise profit. To this end I’ve committed to the One Game a Month project for 2013 and to advance the conversation around Free-to-Play with my personal voice I will be launching a project in the new year.

My name is Claire, I make things to feel better about the world and improve it. This is my goal and purpose.

Happy New Years

Four meditations on bad design decisions

Niklas Frykholm — Tue, 11 Dec 2012 15:25:12 +0000

I’ve recently been doing a major rewrite of one of our core engine systems, the graph that we use for our visual scripting language Flow. Taking it from something that looks like this:

To something that looks like this:

A major rewrite like this is always a humbling experience. When you have to rewrite your own code, every bad decision you made comes back to haunt you. And you don’t have anybody else to blame them on.

As if facing your own inadequacy wasn’t enough — rewriting an existing system is always harder than writing one from scratch. When you write a new system you start with a blank slate and can do whatever you want. When you rewrite, you are constrained by what the old system did — at least if you want to maintain any kind of backwards compatibility.

In addition, a new system can be written iteratively. You can start with a very small, simple system, release early and get feedback. Based on that feedback you can tweak the system. You don’t have to think about adding features until you have a good stable base.

When you are doing a rewrite you can’t release the new system until it is at least as good as the old one. Otherwise, your users will question why you have spent all that time working on a system that is worse than what you had before. And they will be right.

So a rewrite forces you away from the comfortable land of early releases and quick iterations and into the ugly old waterfall model.

With the power of hindsight, I’d like to reflect a bit on four design mistakes I made when I wrote the first version of the system that made this rewrite a lot harder than it could have been.

Don’t use strings for non-text things

Strings have one really good use — to hold pieces of text that either gets displayed to or inputted by the user. All other use of strings should be regarded as suspicious.

Strings are scary because they are both ambiguous and powerful. Does “a/b.txt” and “A//b.txt” represent the same path? Hard to tell. But maybe you can use case conversion, search and replace and some regular expression monstrosity to figure that out.

If you are doing that kind of string manipulation in any part of the code that is not directly related to user input or output, it is a clear warning sign that your code might be too “stringified”.

The most obvious example stringified code is the use of “stringly typed” data, for example, storing a date as the string “2012-12-09″. But the problem with strings can also manifest more subtle ways.

The original version of Flow used strings to identify connectors, both internally (as a representation of the connection) and visually (to show the name of the connector):

As a consequence, a Flow node couldn’t have two connectors with the same name, and a connector couldn’t be renamed (even visually) without breaking all existing connections.

In retrospect, rather than having a single Name property, it would be much better to have separate Id and DisplayName properties. The Id would be a GUID that uniquely identified the property, and the DisplayName would be a (localizable) name, suitable for displaying to the end user.

Using names/strings as identifiers has bitten me in other ways as well. In one system I knew that the names had to be unique (because that is how the script would refer to the objects) so I thought it would be safe to use them as identifiers. What I didn’t consider was that there could be situations when there temporarily were two objects that had the same name. For example, if the user had created a rock object, and wanted to create a rock_small object — as she was half-way through typing that name, there would suddenly be two objects named rock. This created problems for the system.

Lesson learned, I now avoid using strings as identifiers.

When in doubt, you should opt-out

Every system acquires features over time. That is good of course. Those features make the system more powerful and easier to work with.

But among the good features there are usually a few that don’t feel quite right. That don’t really fit into the design of the system. You can do them of course. You can do anything.

But usually it is best not to. Most of the time when I have added a feature that didn’t quite feel right, I have regretted it later. In retrospect it would have been better to try to find a different way of doing what the users wanted that was more natural to the ideas behind the system.

An example: Users of Flow wanted some way to specify the order in which events were triggered, when multiple connections are connected to the same Out connector. This is needed in some situations, for example you may want to make sure that a unit is spawned before it is used.

In the old version of Flow, this was implemented with a context menu on the connection where you could select if it should be a “Do First”, “Do Last” or “Do Normal” connection.

This solution never felt 100 % right to me. It was hard to find a good intuitive way to visually represent the “Do First” and “Do Last” connections, and as a result the Flow graphs became harder to understand.

In retrospect, it would have been much better to avoid this feature and wait until I had come up with the more elegant alternative: a sequence node that triggers each of its outputs sequentially:

Be explicit or you’ll miss it

Writing code where a lot of things happen implicitly feels great — to begin with. It is amazing how much you are able to do with just a few lines of code.

But in my experience, implicit code almost always ends up more costly in the long run. It is harder to understand, harder to debug and harder to change. It tends to lock you down in a “local minimum” that can be tricky to come out of.

In Flow, a lot of things are done implicitly. The definition of a Flow node is just a simple C# class:

[Category("Animation")]
public class AnimationEvent : Node
{
    public InVariableUnit Unit;
    public StringVariable Event;
    public InEvent In;
    public OutEvent Out;
}

Through reflection, Flow finds out the members in the class and their types and automatically generates Flow nodes for them. This process involves some ugly string processing (bad decision #1), such as stripping In and Variable from the type name to find the underlying type of members. Reflection is also used to serialize the graphs.

While it is nice to be able to express a node so concisely, there are also a lot of problematic consequences. For example, since the class names get serialized, we can’t change the names of classes or properties without breaking the ability to load old files. Also, we have to use some really ugly C# hacks to make sure that the reflection system always returns the members of a class in the order they are declared in the file (so that we can control the order of the connectors).

In retrospect, it would been much better to avoid all this clever reflection stuff and instead just define the node types in configuration files.

Avoid the road of complex code

There is some code that needs to be complex, because it is dealing with fundamentally tricky stuff (like computational geometry) or because it needs to run really, really fast. But in all other cases, complexity is just a cost.

If your code starts to feel complex and hard to keep track of, it is a sign that you are probably doing something wrong. And if you are not careful, you may lock yourself in, so that when you write the next version of the system, you have to recreate all that complex behavior in your new, simpler system. You have to deliberately make your code uglier.

The old version of Flow had a way of “folding” nodes. You could select a number of nodes, group them, and then “fold” the group, collapse it to a single node.

The system had a lot of really hairy code for dealing with this. The code takes a bunch of nodes and creates a new node from them, with connectors matching only the external connectors of the collapsed nodes. it also keeps track of the internal nodes and their connections so they can be recreated if the node is later “expanded”.

As you might imagine, this was complicated further by the need for connector names to be unique (see bad decision #1), which meant that some of the external connectors in the new node had to be renamed (since they could come from different internal nodes that had connectors with the same name). So a mapping table was needed to keep track of these renames. Obviously a bad idea, but once you have started down the path of wrongness, it can be hard to turn around.

The new version handles this a lot better. Collapse and expansion is just a visual feature. There are no new nodes created and no other strange things happening to the data, the visualizer just chooses to draw the data in a different way when it is collapsed. In retrospect, that is a much better choice.

That is all, four simple lessons
to guide your future coding sessions
now let your code be light and merry
until its time for Charon's ferry

This has also been posted to The Bitsquid blog.

Using the Internet for free-samples gaming

Colt McAnlis — Mon, 10 Dec 2012 15:52:46 +0000

Why Free Samples Work

Every week or so, I walk into my local Costco and talk to Meredith, who politely offers me a taste of whatever food she’s providing free samples of that day. Luckily for me, last week was chicken nugget week, and Meredith’s food cart was conveniently standing in front of a huge display of chicken nugget bags. Within 4 feet, I could try the free sample, and if I liked it, grab the bag to purchase.

Costco has used the free sample approach for as long as I can remember, because they understand the need to entice shoppers to new products. Shoppers generally walk into a store with a shopping list, and they are numb to discovery of new products that are not on their list. This lack of interest makes sense from a consumer standpoint: Why should someone invest money in a new product that they know nothing about, especially when their old chicken nuggets taste just fine? Seth Godin, a fellow bald guy, describes this principle in his TED talk about purple cows:

Consumers don’t care about you at all, they just don’t care. Part of the reason is that they have way more choices than they are used to, and way less time; and in a world where we have too many choices and too little time, the obvious thing to do is ignore stuff.

Game development today is dominantly focused on mobile, and with a total app footprint topping 1.4 million, it’s safe to say that most users have too many choices and too little time. This is why mobile app stores are so fervently fought over: They tell users what to pay attention to, and without that prompting, apps can easily languish.

The question for you as a game developer, then, is how to get noticed in these ecosystems and make your app rise above the noise. In this context, many developers have come to the realization that cost is an added barrier to entry. This realization has led to the development of freemium applications, which are now a dominant revenue stream. Even developers of premium applications produce free-trial equivalents in app stores, offering a familiar nod back to our shareware roots. The proof is there: Users are more willing to try something if it’s free.

As a game developer, then, you should be providing free samples of your game, not just in app stores, but to every human on the planet. For years, game developers have focused on making their apps available on as many platforms as possible. Digital distribution services do offer an opportunity to provide a free version of your content, but silo barriers still filter out many of the potential users that would otherwise seek to discover your content. Supporting multiple platforms also comes with an implicit resource cost, which at some point raises the issue of diminishing returns and Region of Interest.

The Ultimate Free-Candy Platform: The Internet

As a platform, the web provides an unbounded opportunity for game developers to reach out and engage with potential users. The viral nature of the Internet hasn’t diminished with time; in fact it has accelerated. Consider as an example the launch of the game 10000000. Eli Hodapp, the editor-in-chief of TouchArcade, quickly found and fell in love with the game. As was reported:

The speed of all this is remarkable: It only took a few hours for the community to begin talking about the game, which alerted Hodapp to its existence, and he gave it a shot and played through the day. His review went live that evening, less than 12 hours after the game had been released, and before anyone else in the industry was aware of it.

Hodapp then went on to quickly tell his friends about the app, even buying some a copy so they could try it.

That type of quick uptake and social championing confirms the results of a large research study that analyzed how mobile users discover new content. Not surprisingly, the two dominant sources through which users found out about new content were social channels and friends groups; on-mobile stores were in 3rd place in the study.

This trend has been confirmed by other studies as well. Google’s own Research Division produced a great case study showing that game players are more attached to the web for their discovery/purchasing options. They also retain the connection to the web post-purchase, looking to extend their experience through multiple channels.

As an industry, we’ve seen web-based success stories play out frequently. Minecraft started as a Java application played in a browser. That web presence let players experience and share the content quickly, and enabled the developer to raise over $700k in user-contributed funds to help finish the game (a task done way before Kickstarter was around).

The HTML5 game Bouncy Mouse illustrates another great example of how to use the free-candy web. Their case study shows that the web version of their game not only generates revenue, but also gives them the ability to leverage the reach and ease-of-discovery aspects of the web to help drive users to the mobile versions of the game.

Other games, like Air-Mech and Don’t Starve, have taken advantage of the web’s reach and ease-of-access to offer early beta builds of the games to a large user base, iterating and making improvements before doing bigger launches in other digital distribution locations.

And on a personal note, I quickly backed funding support for Desktop Dungeons after I found the game and played a lightweight version of it in HTML5. Which I highly suggest you do too.

How to Make Free Candy

Creating a web version of your game is not an easy engineering task, but it’s far from impossible. Modern web technologies are easily capable of running high quality games in a browser. With a little effort you can put those technologies to work for you.

Overcome the Language Hurdle

Converting C++ to HTML5 is one avenue that simple 2D games like Angry Birds, Fishing Joy, Cut The Rope, and Bouncy Mouse have followed. A straight port may be cumbersome, but you can fill in the gaps with a technologies like emscripten, which help maintain a game’s look-and-feel while providing decent performance.

For more complex games, you can use new technologies like Native Client that let you run your existing C++ code in a web page. Lara Croft and the Guardian of Light is one example of a full console game that runs right in Chrome—Native Client let the developer leverage their existing code and content in their migration to the web.

Unity developers have the option to export a web version of their game that users can play with a browser plugin. As a bonus, in the Chrome browser, Native Client eliminates the need for users to install anything.

New languages like Haxe provide an interesting middle ground. Haxe is a language that you can use to generate language-specific code for your desired platform, including C++, Objective-C, JavaScript, and Flash. With Haxe, you can write your entire game in one meta-language, and generate platform-specific builds as you see fit, each one optimized for performance and features post-Haxe-conversion.

Optimize for the Web

Once you’ve addressed the initial issues in creating a web build, you must make sure that your game translates over to a web experience properly. Specifically, make sure that you:

Use best practices for developing web games.
Take into account how to transfer digital assets.
Collect error feedback from users in the wild.

Once your game is live on the web, make sure that you allow users everywhere to experience it, and give them easy and incentivised means to share the game with their friends.

Most importantly, make sure that you take full advantage of your web presence by adding deep user-metric tracking to understand your users better, and implementing A/B testing to collect information about how users play your game. With that type of information, you can gain enough statistical data to determine which design elements in your game work, which ones don’t work, and which ones do the best job at funneling users from free-candy samples to premium purchases.

Start Making Candy!

The Internet is all about free content, social channels, and going viral. By using a few key pieces of technology, you can quickly generate a web-based build of your game to help drive user adoption and provide a base platform for marketing and viral social media. Give users some free candy, and they will happily buy the Kobe steak.

Marketing Outside the Box

Alex Norton — Wed, 05 Dec 2012 02:07:13 +0000

So, as an indie developer new to the scene, the thought of marketing my game has proven to be a daunting one. There are many avenues for marketing available through third parties, but indies rarely have the money for that sort of thing.
Many excellent blog posts before have gone into the various ways in which an indie can get their product out there on the cheap, but I want to talk about going a little bit deeper, and tell how you can use gamer psychology to your advantage to do a lot of your marketing for you.

But before we go into that, the first thing that an indie developer really needs to do is to have their product displayed in as many places as possible. When starting work on Malevolence I started out with just a simple BlogSpot account and did all of my updates on there. Guess how many people cared? None. And why should they? How could they even FIND the blog unless they were told about it?

Take a search engine like Google. It has a “crawler” that goes around the web, finding pages, and finding all of the links on those pages. It analyzes where the links link to, and finds which pages have lots of links heading to them. If a page has hundreds of other pages linking to it, and lots of visitors, then it must be important, right? Well, to make your game more visible, you need to have lots of people visiting your page – a bit of a catch 22 – but you can help it by having a presence in many different places. Oh, and good news everyone… The Internet has many, many viable, justifiable places where you can put your game on show.

For Malevolence, the first big step after BlogSpot was to create a profile on IndieDB/ModDB (creating a profile on one creates one on other other, too). They get quite a lot of traffic on there, and it will actually rank your game against others (a library of about 6000 games) in terms of how much traffic you get. It also rewards your dedication to the public by putting you on the front page whenever you write up a new update post. Given the way it works, and the attention that it gets, it’s very much worth taking the time to make your profile page attractive and thorough. That’s where a bit of the psychology comes in. I have noticed that gamers who browse ModDB for new indie games are a little bit like channel surfers. They flick through profiles quickly – bam bam bam – until they find one that appeals to them very quickly, and then they’ll stop and take a deeper look. So having lots of nice, bright screenshots, thorough videos and lots of clear, solid information about your game readily accessible for them will entice them to stay and take a proper look at your game.

Once you notice that you’re getting more traffic, you can make a move towards putting links on your IndieDB/ModDB profile to other networks that you’re operating on. Some examples of which include YouTube channels, Facebook Fan-pages, Twitter accounts, etc. Any and all are good. Just be sure to make the links subtle. Some nice clean icons, perhaps. Gamers, I have found, dislike feeling like they are being bombarded. Do some research, take some time, look at successful pages and the way they’ve done things. It pays off.

The social media avenue was the next step for me with Malevolence. It started with a YouTube channel where I would make a point to do screen-captures fairly regularly showing the progress of the game’s development. Many people either don’t enjoy or don’t have the time to sit down and read lengthy blog posts (he says in a lengthy blog post) and a nice, quick video can get the whole point across much faster and with more of a sense of interaction. Even taking the time to narrate the video can easily double or triple the amount of information translated in the same time frame. Things to think about there.

After that came the Facebook and Twitter. I actually expected these to be the most active of the network avenues for promoting the game, but in the end they fell into roles of keeping CURRENT fans up to date, and didn’t really do much to draw new fans. Though it seems obvious now, it took me a while to work out why this is. Most people on Facebook and Twitter these days use them on their smartphones, rather than their computer. While it IS possible to go browsing for new content via the smartphone interfaces, they are much more designed to just keep up with what their already follow. That’s not to say that you can’t still make use of these avenues though, as they offer quick and easy mini-update portals for you to address your current fan-base en-masse, and quickly spread links to any major blog updates you do elsewhere.

For the longest time, Steam was one of the holy grails of indie games “making it” and their new Greenlight process has taken that from some dark, incomprehensible process to something more tangible. As soon as it went live I create a profile for Malevolence on there thinking “wow, I might be in for a chance”, and things went fairly well for a time, but then started petering off quick quickly. Valve kept altering and adjusting the vote process for getting games approved and it was quite drastically affecting my game’s ability to get noticed on there. It’s been quite frustrating until it got me thinking… How WOULD you make an unbiased system for gauging a game’s popularity in the middle of a firmly established community of over 50 million users? The more I think about it, the more complex it becomes in my head. Sure, I have lots of problems with the way the Greenlight process works, however, since I can’t come up with a better one myself, I’m happy to leave it up to them to sort out. I’ve had many heated discussions with fellow developers about how the system should work… Mainly about the apparent pointlessness of having a “no vote” button (which I’m sure will create some lovely ‘discussions’ in the comments) however, as I said, I’m happy to leave the process up to Valve, as they would certainly know best. Of all the networks I’ve gotten Malevolence onto, that’s the only one that hasn’t really gone anywhere yet. As the owner of it I can watch the percentage of votes go up and down, and I can’t really make sense out of it, so I’ll just keep doing what I’m doing and hope for the best. The only advice I could give to other indies attempting it would be to make sure that all of your avenues (Facebook, IndieDB, dev blogs, etc) point to your Greenlight so that you can funnel as much of your traffic into it as possible.

HOWEVER, all of that only depends on how badly you want to get onto Steam. I’m aware that some people don’t. Some because they just don’t like how Steam operates, others because they’re making mobile games and it’s kind of useless to them. I, personally, am trying to get on there, but that doesn’t mean that everyone SHOULD.

But I digress. What about the psychology mentioned earlier? Well, that is where your marketing plan comes from. Just sticking your product on some blogs and social network sites isn’t really a marketing plan. It’s just a means to give your game a ‘presence’ on the Internet. A solid presence is a powerful thing, but it takes constant maintenance. It’s not something you can set and forget until you have a large, strong, dedicated community to do it for you. As an indie, that’s just not something that’s likely to happen straight up, so it’s something you’re going to have to do yourself. KEEP your blogs updated. RELEASE lots of screenshots. RECORD lots of videos. ANSWER lots of questions, and above all, MAKE SURE that your community is reminded regularly that you are still working on your game, lest they lose interest.

But what about expanding your community? That’s where your marketing plan comes into play. You’ve got your presence sorted out, you’re maintaining it well, but how do you bring new people in? That, in itself, depends on many different things:

TARGET AUDIENCE: What is the target audience for your game? You can’t just say “gamers” because not all gamers like tower defence games. Not all gamers like FPS games. Not all gamers like RPGs. And then, even within the types there are sub-types. In the RPG player genre there are gamers who like first person RPGs, but don’t necessarily like isometric RPGs or turn-based RPGs. So it’s a matter of specificity. Take the time to work out precisely what your target audience is with your game and market mainly towards them. There’s not much use in wasting energy trying to talk to people who aren’t interested in listening!

ANALYZE: Take the time to analyze the way your target audience acts. Do they talk amongst themselves? Do they operate vocal communities? Is theirs a community of sharing, or of bragging? As an overly abrupt stereotype, FPS gamers tend to have communities that are much more vocal and aggressive than communities of RPG games. I realise that’s a massive generalization, and it is most certainly not ALWAYS the case, but if you look into how the communities of your target audience operate, you can use it to your advantage when creating your marketing plan.

ACT AND REACT: As an indie, much of what you learn, you will learn on the fly. If you make a marketing faux pas, you will learn about it pretty quickly. Try not to be too stubbourn about things. While it can be trying at times, an indie game without fans/players, is nothing. It’s like owning a coffee shop that has no customers. Don’t be too hard-headed to change the way you operate if it’s something small and it keeps people happy. You need to be adaptive, you need to listen and you need to communicate.

So, with these things, you can create your marketing plan. As an example, my marketing plan for Malevolence was to make use of the RPG communities idiosyncrasies. I did my research, joining forums and following blogs, doing the whole Jane Goodall thing and “living with the apes” so to speak (not calling RPG players apes, by the way. Just using a metaphor) and I found that a common trait among them is that they are excited about indie RPG development and will quite often share links to interesting indie RPG projects that they find. BUT, at the same time, while there is a lot of link sharing going on, there isn’t much response to it. Many RPG forums will have an entire section just for sharing indie RPGs that people have found. These sections are often filled with lots of posts with links, but the posts won’t often have many responses. That doesn’t mean that people don’t follow the links, but it means that there is more excitement about sharing the links than there is about the games themselves. RPG fans seemed excited about the IDEA of new RPGs, which is cool.

I made a note of this and built my strategy around it. I planned to start my marketing push at the same time that Skyrim came out. Sounds crazy, right? Well, there was reason behind my madness. I started pushing the links to my various blogs, etc, on forums and other hubs while people were busy getting fussy over Skyrim. The community saw my links and shared them around, but, as expected, not much action happened other than the spreading of the links. Because of this, there ended up being many, many sites with references to this infinite indie RPG. As it spread, more people heard about it, but they didn’t focus on it because… Hey… Skyrim… But that was precisely my plan.

The excitement over every game, no matter how groundbreaking the game was, eventually wears off, and once the attention for Skyrim had started to dim, I was in the position where links to my game were ALL OVER the Internet, and a vast portion of the RPG community had heard something about it. Even if they couldn’t recognise it by name, when someone mentioned “Have you heard about Malevolence, that infinite indie RPG being made” more often than not they would be met with a reaction of “I think I heard something about that, yeah… Can’t remember where though.”

This is where the psychology of my marketing plan came in. Having another person bring it up makes them remember seeing it somewhere which inspires them to look for it again. They hit the search engines and start typing in the word “Malevolence” since they have no idea how to spell “Ahkranox” and, since I have made an effort to have a VERY strong web presence, they see this:

They barely have started typing the word Malevolence, and it shows up. Marketing plan complete. That being said, they may go to the site, look at it and hate it. I unfortunately can’t help you with that. Marketing is one thing, but your product still needs to stand on its own. I was lucky enough to have this strategy work quite well for my game, but sometimes even the most well thought out strategy will fail. Not having a strategy at all, however, will pretty much guarantee that you’ll fall short, so start early!

But that’s where I’ll leave off, I think. In conclusion, when marketing your game, you need to take the time to establish a web presence, develop a marketing plan based on research and analysis of your target audience and attention to detail when looking into the psychology of the way they act as a community. Stick with it. If you believe in your game strongly enough, you won’t have too much difficulty in translating that to your viewership!

Game Mentor Online

Lisa Brown — Mon, 03 Dec 2012 15:53:12 +0000

I have discovered that I have a passion for helping students and aspiring game developers learn more about the industry. Part of this is because I occasionally feel like I stumbled into this career, and feel compelled to educate young people about game development in hopes of inspiring other people who were like me (potentially saving them 8 years of figuring out what they want to do with their lives). So far, my favorite outlet for student outreach has been Game Mentor Online, an online service that matches up aspiring developers with industry professionals and facilitates one-on-one mentorships.

About the Program

Game Mentor Online is an initiative that was kicked off by Women in Games International. Originally concocted by Karen Clark and Sheri Graner Ray in 2006, its first form was a mailing list where mentors and mentees were able to link up and communicate. The program was able to raise funds with the help of the IGDA Women in Games SIG to convert to an online mentoring software solution, which is how it has appeared since 2011. Game Mentor Online is entirely volunteer-run and donation based.

We have over 100 proteges and more than 60 mentors from all over the world, and we’ve had more than 100 mentor-protege pairings since moving to the new system

-Karen Clark
GMO Admin

The main function of the site is as a sort of matchmaker, trying to pair up mentors and mentees, though there is a community area with member-posted articles and questions from proteges that any mentor can answer. As a freshly signed-up mentor, you first fill out a profile with your skills, discipline, experience and match preferences (such as how many proteges you would like to mentor at a time). It is the responsibility of the proteges to search through available mentors, though the system does suggest potential matches based on your profile information. A protege will then contact a potential mentor, and the administrators can set up the mentorship.

Once a match is made, a mentoring area is created on the site for the connection, where you and your protege can post messages to one another and set goals. At this point it is up to the two of you to decide how the mentorship will proceed.

My Experience

TRACED, a hacking-themed game, is my protege’s side project

My protege is an aspiring designer named Kaylin Norman. He and I plan to meet and chat about every 2 weeks, give or take depending on both of our schedules, where we dedicate at least an hour to chat over Google Talk. These sessions can range broadly in topics, but it can be anything from answering “Day in the Life of” questions, helping with his resume, giving feedback on his game projects, discussing design tips, and answering any question that Kaylin might have encountered in his most recent game design adventures.

How often and what form through which you mentor your protege is decided by both of you depending on your particular circumstances. Some mentor/protege pairs prefer to do regular Skype sessions while others carry on their mentorship primarily through email exchange. It’s very flexible, and the online site serves mostly to facilitate the mentor relationship and get it started in a natural direction.

What you bring to the table

Mentoring a student has given me a lot of insights of how much knowledge I have gained (and subsequently taken for granted) since working in the industry. There have been several times in a discussion when I would answer a question or make a suggestion of something that had become so commonplace to me that I assumed it to be common knowledge, only to find that it provided an amazing insight and learning point for my protege.

Besides your own experience, you also indirectly provide your protege the advantage of your professional network. A few times Kaylin has had a question or had a situation that I wasn’t sure I could give a lot of information on from my own experience, so instead I would take the question to my other friends and colleagues in the industry. I’d often return to my protege with responses from multiple industry veterans that he would find invaluable to his own development as a designer.

As far as what you can get out of mentoring relationship, I’ve found mentoring Kaylin to be incredibly rewarding. It’s made me very proud to see him grow as a designer, see him achieve academic goals and watch him work towards others. On days where my job has been trying or I’ve been frustrated by a design problem, chatting with Kaylin and seeing his enthusiasm about design and hearing how he is progressing proves to be quite renewing.

A Growing Program

Game Mentor Online in its current form has been around for 1-2 years, and one of their most admirable traits is how open they are to feedback about the program. Any time I’ve had a suggestion, the administrators have responded promptly with a plan on how they could incorporate the feedback. They are very passionate and eager to grow and improve the program.

As I mentioned before, Game Mentor Online is very good at facilitating the beginning of a mentoring connection by providing a formal structure through which the mentor and protege can find their footing with one another. But it seems to be common for pairs to move to other more standard means of communication once they establish a relationship. I think this is certainly a good thing, but it does mean that the population that actually interacts on the site appears to be kind of low. There are occasional articles, links, and questions posted in the community areas of the site, but the activity there is minimal.

I don’t think this trend is necessarily a bad thing. Afterall, if mentors and proteges are finding more natural ways of carrying out the mentoring relationship, that means the site has done its job of kicking off that connection. However, it would be good to grow the culture of mentors posting articles and proteges posting questions to create value for proteges on the site who have not yet been connected to a mentor.

Tips for Mentors

If you decide that you’d like to mentor an aspiring developer and sign up for Game Mentor Online, or any other mentoring program, here are a few tips I’ve picked up along the way:

Communication of expectations is critical. When you get started, make it very clear what kind of time you’ll be able to commit. Encourage your protege to share his or her expectations for what they hope to get out of the mentorship and be honest on how you think you can help fulfill those expectations.
Be really specific about times you are available. If you start a habit of chatting every two weeks, and then something comes up that may disrupt that pattern (like a conference or a big milestone), let your protege know early so they can expect the change. If you have to miss a session or expect to be late, inform your protege as soon as you can. Establishing and maintaining trust is really important for a good mentorship
Don’t hold back sharing experiences or stories because you feel they are mundane or routine. What your mentee may not know may surprise you, and you probably have a lot more to offer than you realize.
There is some element of luck of the draw as far as your compatibility with your protege. I was extremely lucky in that Kaylin and I forged a strong connection right off the bat, but it’s possible that you and your assigned mentee don’t click. I would advise not to get discouraged and to keep at it. The default length for a mentor connection is 4 months (though you can extend it for as long as you like if you and your protege are hitting it off) and the next protege you get paired with could be a completely new experience.
Meanwhile, the program has an FAQ for potential mentors that I would recommend reading for more information. I’d also suggest looking over the FAQ for proteges to get an idea of the frame of mind your mentee will be coming from when you get matched.

Other Opportunities

I love Game Mentor Online because it is easy and convenient to incorporate into my busy designer life, and because I feel very comfortable with remote communication. But there are other opportunities in our industry that I’d recommend checking out if you are interested in mentoring.

If you like to do your mentoring face-to-face, check out the IGDA mentoring program, where professionals are paired up with IGDA scholars at GDC, E3, and other industry events. You can go to talks with your scholar, meet up and discuss different talks that you’ve seen, answer their questions in person, and introduce them to a greater professional network.
MentorNet has a similar online structure to GMO, but is more generalized for engineering and tech careers and students. I found I didn’t have as prosperous a mentoring experience here, but part of that could be that I didn’t click with my mentee there in the same way as Game Mentor Online
Schools and educational institutes might have opportunities to connect you with a student that you may be able to help. Some schools have alumni networks for advising in career services, and sometimes it’s as much as emailing an old professor to say you’re available to answer any student questions about game development.
LinkedIn’s Video Game Careers group has a lot of student posters and is very active . It creates a lot of ways to give feedback to students by reviewing portfolios and giving feedback for their projects – a good way to advise and help many students in very specific ways versus a single relationship with one person over time.

If any readers know of other good mentoring opportunities, please let me know in the comments and I will add them here!

My Protege’s Perspective

I asked Kaylin if he would like to write a bit about his perspective on the program as a mentee. Here is what he had to say:

For the longest time, I’ve been teaching myself the aspects of game design from a young age, unsure of whether or not I was forging down the right path to reaching my goal of working in the games industry. Living within the sheltered life style that I did live in, television and the internet were the only mediums I had to explore the games industry, or at least what I had perceived to be the games industry at that time. With game schools posting commercials exploiting starry eyed aspiring developers from their funds by making them believe that the industry is this “easy place to carve out a name for yourself” along with the exclusivity of access to developers of any established company, from small studios like 5^th cell and Flowplay all the way up to 343 Industries and Bethesda, there was no one in my life to tell me how the game industry really functioned and what the experience was like, until that one defining moment when I got the message the hard way –from a veteran who was very disgruntled with my misguided passion.

Upon meeting Lisa through Game Mentor Online, I’ve been not only enlightened about the experience of working in the games industry, but also how to think and act like a game designer—a crucial yet often taken for granted thing when the majority of students already have a network with other designers in the industry. Because of Lisa and her connections, her willingness to answer my various questions, show me certain processes that I would otherwise have to learn through private schools and simply helping me when I’m in a hard spot, I’ve transformed from a shy sheltered boy to a confident designer. Her feedback was crucial to getting my side project—TRACED off of the ground, Without her overall input over the concept and her introducing me to the process of play-testing, concept documentation, etc. I would have never managed to find the three other teammates I have working on this project and keep them working on this project for a full year.

Game Mentor Online and my mentor/protégé relationship with Lisa have transformed my thinking! I would recommend it to all of my aspiring friends who want to also become game developers some day! I’d rather get it from the professionals rather than school commercials!

- Kaylin Norman

Conclusion

I know that our lives as professional developers are extremely busy, but we have so much that we can offer to the next generation of developers. Game Mentor Online’s flexibility and online focus is a great opportunity to connect with a student in a way that can fit into a very full game dev life. Even if you are new to the industry, you still have more to offer to an aspiring developer than you may realize. If you could spare as little as an hour a month to connect with an ambitious, excited, and curious protege, I suspect you may come away from the experience feeling inspired and renewed.

What they DON’T tell you about being a game developer

Alex Norton — Wed, 21 Nov 2012 01:14:24 +0000

So I’m in an interesting position. Malevolence, while not my first game by a long shot, is my first RELEASED game, and I’ve been lucky enough to have it gather a lot of attention (for an indie title) early on in its creation. From what people tell me, this does not normally happen. Normally, a developer will hit on gold after they’ve tested the waters with a few titles first, or had a hand in other game development, such as working for a AAA company.
Because of this unique perspective of having a relatively successful title (despite not yet being released) on my first ever attempt, I haven’t yet developed the pessimism that often comes with being an experienced indie game developer. This has led me to want to write this new thought piece, which goes into all of the things that they DON’T tell you about being a game developer. If you want the short and sweet version, feel free to skip to the end.

STAGE 1 – DELUSIONS
—————————
Going through university I had the same delusions as most people that I would get my qualifications, build a folio and get a job at a AAA game company. Shortly thereafter, fame and riches would ensue and I would live happily ever after, making games that I love, and having everything right with the world.
I finished university to find that all of my hard work would get me on a “consideration” list for a baseline, entry level QA job which would mostly consist of me being locked in a cubicle for 70+ hours a week doing some of the most repetitve, soul destroying work known to man.

A good example of what it is to be a tester, provided by Penny Arcade.

Sure, that would give me a foot in the door to have my true skills recognised later on, however, most game companies go through and trash their QA teams at the end of every project after smothering them in NDAs which make their soul now below to the company. So that’s a minefield in itself.
If, and I really mean IF, I was to be plucked from QA and given a position within the company, one of two scenarios would have happened:

EDIT: Since this seems to be causing some controversy, let me clarify that my thoughts listed below are not my opinion of ALL games companies. I am simply listing things that tend to commonly happen based on the experiences of the many developers I have spoken to, and these thoughts in no way represent what the industry is like as a whole.

Programmer Position – I would be sat down and made to do scripting work for 70+ hours a week on someone else’s proprietary engine, since using third party products “speeds development” and I would learn that there is such a thing as an “acceptible sacrifice of quality for speed” which would slowly destroy my integrity as a programmer. My passion for creating new features would slowly be vampirically sucked away, and anything I made in my spare time would be contractually owned by my employer which would sap my enthusiasm for innovation even further.

Artist Position – I might be lucky enough to get assigned work that interests me. I might be having a great time designing and modelling sci-fi assets for an amazing new shoot-em-up and enjoy watching them come to life within the engine. But one day the producer (who is computer illiterate and whose only gaming experience is with Bejewelled) would pop by and say “You know, my 10 year old niece is really into ponies right now. Change the game to be about ponies”. Because he is the money behind the masterpiece, I would now be forced to abandon all of my work and create ponies and handbags all day every day. Also, everything I make outside of work would be contractually owned by my employer, limiting my folio and preventing me from moving to another company.

STAGE 2 – DETERMINATION
———————————
So, having this realisation, I then turned my attention to the indie game dev scene, which was far more attractive as I was beholden to no-one, I could work on what I wanted and do it in any way that I pleased (so I thought) but there were certain demons there which no-one told me about, either.
I got myself a day job to take care of living expenses and turned my spare time towards my magnum opus… Malevolence: The Sword of Ahkranox – the game of my dreams that was everything that I wanted to see in a game. I was out to show the world what a game COULD be. New technologies, new gameplay mechanics and a dash of old-school to reel in the retro crowd. It was going to be amazing and the world would be changed forever.

I was young. I was naive. And no-one had warned me what was coming.

First off, I had been playing around with a new method of procedural generation to create infinite worlds, and my favourite gaming genre was always RPGs, so I figured “why not make an infinite procedural RPG?” it seemed a good idea at the time, so I started work on it and found that it worked. I started a blog about it, more for myself than anything. Things were going fairly well.

STAGE 3 – REVELATIONS
——————————
It was at this point that I started meeting a couple of other game devs. Before this point I didn’t really know any, and one with quite a long resume happened to start work at my day job. We got to talking and he introduced me to more people. They all seemed quite interested in my project and thought it was great that I was trying new things. This reaction gave me the motivation to make my development process a bit more public, so I started promoting Malevolence a bit more and getting more of a team together to work on the game’s shine.

Much to my amazement, it got quite a lot of attention (for an indie game being made by a nobody in the industry) and that made me happy, so I kept at the PR. But with positive attention comes negative attention, and it was then that I learned two important things:

People, when given the anonymity and audience of the internet, can be truly horrible.
You can read 1000 praising comments, but if just one of them is bad, it will ruin your whole day.

The fact that I was trying to do something new with my game was evidently a horrible crime to many people and I would get utterly horrible comments ranging from put-downs to persanal abuse that would get them arrested if said in person… Even one or two death threats. It’s a sad fact of life that people who are too scared to follow their own dreams will often try to talk you out of following yours. It’s easy for people to say “just ignore those comments” but that’s simply not possible.
Some people who disagreed with the game’s concept gave thorough and well-written justifications for their feelings, which was good to see, but they were few and far between.
I had decided early on to keep an open-doors development policy and be extremely communicative with the public. I even developed somewhat of a reputation for answering every question asked of me. Many people loved this, others took it as justification to send more abuse, because they knew someone was listening.

At first, I thought it was because of my new ideas and concepts that I was touting, but after meeting with other game developers I found out that it’s just what the gamer community is like. Full of angry, hateful, rude, abnoxious people who feel entitled to say anything they want to the developers who are making games for them. That’s not to say that they’re all like that. Far from it! Many, many people have been very supportive, communicative and encouraging throughout the entire process, and you have to really cling to people like that because, as a developer friend of mine once said “Those people hating on your game will always complain loudly. That’s just what they do. The fact is, though, that they’ll probably still buy your game.” and that’s what you need to focus on. It doesn’t matter if other people like your game. What matters is whether YOU like your game. If you love it, other people are bound to as well. Just look at how much hate has been poured upon Minecraft over the years, but Mojang have sold millions and millions of copies regardless, and you can tell that they’re super proud of their creation!

Being proud of what you’ve made is very important. Whenever something happens around your game that makes you feel proud, then you’ve gotta grab a hold of it and not let it go. I’ve had John Passfield sit me down and tell me that he believes that Malevolence has the makings of an epic game, I’ve been called a “visionary” by members of the Guild Wars 2 team, I’ve been recommended by RockPaperShotgun as a project to watch… These things make me glow with pride, whether I believe them or not, and whenever I’m getting slammed by ignorant haters, I remember these things to help get me through it, and that’s something you’ve just got to do to survive psychologically.

STAGE 4 – HINDSIGHT
————————–
So, looking back, I’ve realised there are a lot of things that people just don’t tell you about being an independant game developer:

A large, loud portion of the public will openly hate you regardless of what you do. Learn to live with it.
No-one will ever take your project as seriously as you, or fully realise what you’re going through.
Everyone will think they know better than you about your own project.
Getting noticed at all is incredibly difficult .
The odds of you making money out of it are slim.
If you want to succeed, you’ll likely have to sell out. Just how MUCH you sell out is up to you.
You have to develop a VERY thick skin.
Being open with the public isn’t neccesarily smiled upon 100% of the time.
You will meet many “game developers” but very few people who are actually developing games.
You need to have the ability to listen to all advice given to you. Remember that listening to advice doesn’t mean you have to take it. But listening can’t hurt and you never know what you might learn.

I’ve also learned lots of things to never do again which may help upcoming developers:

Don’t make an RPG as your first released game, nor any other kind of large-scale project. Start simple. Learn the lessons. Once you’re experienced, THEN you can work on something big.
Never announce your release date until you are 150% sure of it.
Never let yourself get so enveloped in your project that other parts of your life suffer.
Never engage the haters.
Get a test team and follow strict testing practices.
Have a thorough plan before you start working too hard on it, and then stick to that plan come hell or high water.
Think carefully about having a public development process. Depending on the target audience and the project itself, it may be better to develop it silently and only open things up to the public when you’re nearing completion.
Never let anyone cause you to stop being proud of your work. The moment you aren’t proud of it anymore, the moment people will stop respecting you for it. If you make the game, and no money comes of it, at least you’ll have work that you’re proud of.

Some of them are hard lessons to learn, but learn them well before you venture into the murky waters of independant game development. Consider yourself forewarned! Obviously, other people will have other bits of advice, or revelations of their own, so I’d love to hear them, too! Share them in the comments!

A Formal Language for Data Definitions

Niklas Frykholm — Sat, 17 Nov 2012 14:19:31 +0000

Lately, I’ve started to think again about the irritating problem that there is no formal language for describing binary data layouts (at least not that I know of). So when people attempt to describe a file format or a network protocol they have to resort to vague and nondescript things like:

Each section in the file starts with a header with the format:

4 bytes			header identifier
2 bytes			header length
0--20 bytes		extra data in header

The extra data is described below.

As anyone who has tried to decipher such descriptions can testify, they are not always clear-cut, which leads to a lot of unnecessary work when trying to coax data out of a document.

It is even worse when I create my own data formats (for our engine’s runtime data). I would like to document those format in a clear and unambiguous way, so that others can understand them. But since I have no standardized way of doing that, I too have to resort to ad-hoc methods.

This whole thing reminds me of the state of mathematics before formal algebraic notation was introduced. When you had to write things like: the sum of the square of these two numbers equals the square of the previous number. Formal notation can bring a lot of benefits (just look at what it has done for mathematics, music, and chess).

For data layouts, a formal definition language would allow us to write a tool that could open any binary file (that we had a data definition) for and display its content in a human readable way:

height = 128
width = 128
comment = "A funny cat animation"
frames = [
	{display_time = 0.1 image_data = [100 120 25 ...]}
	...
]

The tool could even allow us to edit the readable data and save it back out as a binary file.

A formal language would also allow debuggers to display more useful information. By writing data definition files, we could make the debugger understand all our types and display them nicely. And it would be a lot cleaner than the hackery that is autoexp.dat.

Just to toss something out there, here’s an idea of what a data definition might look like:

typdedef uint32_t StringHash;

struct Light
{
	StringHash	name;
	Vector3		color;
	float		falloff_start;
	float 		falloff_end;
};

struct Level
{
	uint32_t version;
	uint32_t num_lights;
	uoffset32_t light_data_offset;

	...

light_data_offset:
	Light lights[num_lights];
};

This is a C-inspired approach, with some additions. Array lengths can be parametrized on earlier data in the file and a labels can be used to generate offsets to different sections in the file..

I’m still tossing around ideas in my head about what the best way would be to make a language like this a reality. Some of the things I’m thinking about are:

Use Case

I don’t think it would do much good to just define a langauge. I want to couple it with something that makes it immediately useful. First, for my own motivation. Second, to provide a “reality check” to make sure that the choices I make for the language are the right ones. And third, as a reference implementation for anyone else who might want to make use of the language.

My current idea is to write a binary-to-JSON converter. I.e., a program that given a data definition file can automatically convert back and forth between a binary and a JSON-representation of that same data.

Syntax

The syntax in the example is very “C like”. The advantage of that is that it will automatically understand C structs if you just paste them into the data definition file, which reduces the work required to set up a file.

The disadvantage is that it can be confusing with a language that is very similar to C, but not exactly C. It is easy to make mistakes. Also, C++ (we probably want some kind of template support) is quite tricky to parse. If we want to add our own enhancements on top of that, we might just make a horrible mess.

So maybe it would be better to go for something completely different. Something Lisp-like perhaps. (Because: Yay, Lisp! But also: Ugh, Lisp.)

I’m still not 100 % decided, but I’m leaning towards a restricted variant of C. Something that retains the basic syntatic elements, but is easier to parse.

Completeness

Should this system be able to describe any possible binary format out there?

Completeness would be nice of course. It is kind of annoying to have gone through all the trouble of defining language and creating the tools and still not be able to handle all forms of binary data.

On the other hand, there are a lot of different formats out there and some of them have a complexity that is borderline insane. The only way to be able to describe everything is to have a data definition language that is Turing complete and procedural (in other words, a detailed list of the instructions required to pack and unpack the data).

But if we go down that route, we haven’t really raised the abstraction level. In that case, why even bothering with creating a new language. The format description could just be a list of the C instructions needed to unpack the data. That doesn’t feel like a step forward.

Perhaps some middle ground could be found. Maybe we could make language that was simple and readable for “normal” data, but still had the power to express more esoteric constructs. One approach would be to regard the “declarative statements” as syntactic sugar in a procedural language. With this approach, the declaration:

struct LightCollection
{
	unsigned num_lights;
	LightData lights[num_lights];
};

Would just be syntactic sugar for:

function unpack_light_collection(stream)
	local res = {}
	res.num_lights = unpack_unsigned(stream)
	res.lights = []
	for i=1,res.num_lights do
		res.lights[i] = unpack_light_data(stream)
	end
end

This would allow the declarative syntax to be used in most places, but we could drop out to full-featured Turing complete code whenever needed.

This has also been posted to The Bitsquid blog.

Playing with my kids helps me make better games

Kyle-Kulyk — Tue, 13 Nov 2012 01:33:01 +0000

A funny thing happened to me smack in the middle of my transition from the brokerage industry to the games industry. People tell you how everything changes when you become a parent. Friends of mine tried to explain the feeling, their eyes taking on a bit of a faraway look as if they were describing an unnatural love of unicorns or some sort of mythical being while I smiled and said “Oh yeah. Oh yeah.” I often joked that agents would slip into parent’s houses at night and pump them full of endorphins while they slept because it was the only way to describe the wonder I saw in those faces at the arrival of those little, pooping, screaming, sleep deprivation units. “Everything changes,” they’d tell me and I’d nod without a shred of comprehension. Then after years of difficulties it finally happened to my wife and I and I got it. I understood why so many I knew couldn’t really put the experience into words aside from the fact that everything changes and that it’s wonderful. I don’t even bother to describe the experience to people without children now, other than to offer a genuine smile and say “Hopefully, you’ll understand one day.”

I was never around children from the time I left home until nearly 20 years later when I had kids of my own. When I was faced with other people’s children, I often found the experience awkward and a bit uncomfortable. I had no idea how to relate to kids of any age or how to interact with them. Now with children of my own I can hardly remember a time where I didn’t know how to play with children, and in return my kids have opened my eyes to why we find certain things “fun”. I hope I can describe this idea in a way that could prove useful to aspiring developers.

Playing video games in my twenties and thirties I think I lost some of the understanding of why I found games fun to play when I was a kid. Video games to me were about roleplaying or they were about competition and if you had asked me why video games were fun even three years ago, I probably would have described some combination of those two factors but over the years I’d forgotten something. Perhaps not forgotten so much as overlooked. While roleplay and competition can be factors in why games are appealing long term I think what makes video games fun is much more fundamental to the way we learn. Watching my children grow and play has helped me remember what drew me to video games as a child and what still keeps me coming back now. It has to do with learning and the feeling of accomplishment when you finally master a challenging game.

From a very early age, babies love patterns. Nothing quite locks an infant’s gaze like faces and patterns. As they get older it doesn’t stop. We find patterns all around us all the time even when confronted with something that doesn’t seemingly have a pattern. We see shapes in clouds and we instantly look for some sort of familiar arrangement in a jumble of letters or numbers. I watched my son stare at a wooden puzzle, then progress to dumping the pieces and creating chaos only to then restore order. He would continue to play in this manner until eventually it’s no longer challenging to solve that particular puzzle and suddenly that toy is forgotten for good (or until his little sister picks up a piece). He moves onto the next challenge and that’s his day with the exception of naps and meal time.

To me, right there I see two fundamental pieces of what keeps us coming back to a good video game. One factor is some sort of pattern recognition mechanic and the other is a challenge. When I started looking at the video games I enjoyed as a kid and that I enjoyed now they all have, at their core, some sort of pattern recognition element and they all had increasing levels of difficulty. I’d play until I either mastered the game and it became too easy or until the difficulty became such that I grew frustrated and no longer found the experience entertaining. I see the same behaviours in the way my toddler plays. It’s fun unless the task is too difficult, and it’s fun until the task becomes too easy.

When I was a kid I remember spending quite a bit of time on Space Ace, among others games at my local arcade. Space Ace was a cartoon, laser disk based game along the lines of Dragon Slayer. A series of events would play out on the screen and a visual cue would signal the move to make with the timing becoming more challenging as the game progressed. Mastering a game like this in a time before strategy guides and the internet took trial and error, a good memory and a pocketful of quarters and I loved that game. That was, until I beat it. Shortly after I memorized the patterns, I moved onto the next game only occasionally popping in a quarter to feel important when throngs of kids who would gather when they’d see “that kid who can beat Space Ace” start a new game.

Whether it’s timing involved in arcade fighting games or if it’s strategy in an on-line shooter, when you break it down video games are all about recognizing patterns and using them within the confines of the game’s rules. It’s an understanding of game development that in retrospect I feel I poorly implemented in the first game my team released in our efforts to appeal to a wider audience. Each level of the game was unique, but the challenge of the game, the pattern required to win didn’t vary enough and looking back at the testing, our players enjoyed the game but the question we didn’t ask was “for how long will they enjoy it?” It’s a choice we made in the interest of appealing to a broader base, but I think this choice didn’t do us any favours and by the time we realized this and updated the title with different ways to play our window of opportunity had already closed. It’s something that seems so basic a notion in hindsight but hopefully by bringing this up I can encourage other new developers to take a look at their product differently.

Playing games with a two and a half year old also helped me rethink control schemes as well. My son loves to pick up a controller and ask “Sack-boy, Daddy?” but a Playstation 3 controller and LittleBigPlanet is a bit beyond him currently. However, I sat him down with Angry Birds – Star Wars and within seconds he was flinging birds at piggies and loving it. The same goes playing “Digit Chase” on the Playstation Vita, a quick demo that has users tap numbers on the screen in sequence. There’s something undeniably intuitive about touch screen input as illustrated by how quickly children take to them, but often mobile developers try to shoehorn controller type controls into their mobile games. I’m not saying there’s anything wrong with modern game controllers, but controls needs to be intuitive. That doesn’t mean they have to be toddler approved simple, I just think the basic controls should be straightforward. This was a lesson we learned developing our first game and reaffirmed by watching my son play. Just because you have variety of ways to control your game doesn’t mean you should just throw everything in because you can. It’s tempting to do. I know because I did it.

I can thank the time I spend playing with my little guy for bringing me back to the basics and helping understand why we find games fun. It’s not about simplifying the games themselves, but it’s recognizing that under everything we’re always searching for patterns and looking to challenge ourselves, because that’s how we learn. It’s not about making controls dead simple, but it couldn’t hurt to imagine a scenario where your game is being played by a gamer who’s never gamed before. Will your controls confuse or will they help the player become comfortable before becoming challenging? It’s easy to lose focus on basic game-play mechanics underneath everything else that makes up modern gaming, especially for experienced gamers. Watching children play and learn helped me realize this and I look forward to gaming with both my kids for years to come, and I look forward to what they have to teach me.

A Simple System to Patch your Game Content

Colt McAnlis — Mon, 12 Nov 2012 15:01:38 +0000

This article explains why it’s important to have your own patching system, and describes how to implement a simple patching system modeled after the Quake3 file-based patching process.

Introduction

Since the rise of PC games in the early 90s, game developers have needed ways to quickly issue fixes, updated builds, and new content to existing users – hence the rise of ‘game patch’ systems. Over time, this method of updating games has made its ways from PCs to consoles, and is now trickling into mobile development. It may take some effort to build and use a patching system for your game content, but once you’ve got such a system up and running, it’s a very powerful tool for your development studio.

Why have your own patching system?

For modern game developers, the most popular avenue to sell games is through one of many digital distribution services like Google Play, Steam, XBLA, and the Chrome Web Store. Besides marketing games to their users, these distribution services generally handle the lion’s share of transferring game content to customers on developers’ behalf.

For games that need to update frequently, however, the content hosting process from such distribution services can be problematic. For example, some of the services can introduce significant costs in patch creation, or delays in issuing updated builds to users. For multiplayer games, delays can let client builds get out of sync with server builds, with no way of triggering an update directly.

One opportunity to work around such hurdles is in embracing the ability to make your content available outside of distribution services. In general, building your game technology on top of your own patching system gives you a great deal of control as a developer, and generally provides options that give your company and products a great deal of flexibility to move between platforms.

Reach your users directly

In addition to solving the problems listed above, a patching system gives you the opportunity to reach your users directly. Nowadays every game platform is constantly connected to the interwebs, and keeping a long tail of customers happy means constantly listening to the community, fixing their issues, and furnishing new content to them. A patching system lets you market to existing users with new content, as well as news, updates, and notices relating to your game.

So, be your name Buxum or Bixby or Bray, your mountain of users is waiting, patch them to happiness, and be on your way!

Patching system overview

Patching systems generally have 3 components:

A build server that generates builds and patches (this server resides with the developer)

A content server from which to distribute builds and patches

A user client that can detect differences between the local and server versions of a game, retrieve assets, and update the local version

Figure 1: The components of a patching system

At its core, these are the three pillars of a patching system. You can create more fancy versions once you start getting into details, but such details tend to be game-specific and are beyond the scope of this article.

A simple patch-aware file system

The Quake3 source code contains an elementary example of a successful patching system. This simple system allows a patch to append new archives to the file system. The new archives contain all the assets that differ from previous patches/builds. Another way to describe this system is that new archives are overlaid on top of existing archives. When an asset is to be read in from disk, the file system traverses the archives and selects the newest version of the asset.

In this simple system, over time a user who installed the original game would have multiple archive files from each progressive patch, each archive containing an updated set of assets. In contrast, a user who acquired the game much later in its life span would not have a plethora of archive files on their disk, but rather a collapsed archive representing the proper state of the world as of the time of their installation.

The Quake3 model is hard to beat for simplicity, and offers a good starting point to address more complex topics as your patching system gets more sophisticated. The example patching system that we will implement is thus based on the Quake3 model, and has the following rules:

The majority of the content is archived.
Content in newer archives take precedence over content in older archives.
- Archived content is generally not patched, but rather replaced entirely.
We ignore binary patching altogether and instead include loose assets that are replaced entirely.

To restate, we assume that as far as assets go, you’ll have the lion’s share in a small number of archives, and that new content will be shipped out in the form of additional archives, the contents of which will wholly replace older content.

We thus assume that there will be a series of archive files on disk. At load time, we open the archives and merge their file lists into a global dictionary. When it’s time to read an asset, we consult the dictionary to determine what the newest version of the asset is, and which archive to pull the asset from.

Updating your build system for patching

Build systems are a bit like sacred rituals – each company tends to have its own flavor and guards its process heavily. I’m not going to cover the concepts of a build system (or tell you how to write one); rather I assume that you’ve got that under wraps.

To generate a patch for a build, your build system needs to generate a list of the files that are different from the prior build (for example, between build 299 and 300, 27 textures may have been updated, 3 models may have been deleted, and the rendering DLL may have been updated). Once you have the ability to generate this type of delta list, you need to combine the information into a patch definition, which is described below.

For our example patching system, any content that has changed or that has been added is included in the archive for a new patch. Finding new files is generally easy: Simply compare the file name listings between two folders to find what didn’t exist before. Finding existing files that have been modified can be trickier. For instance, simply testing the last-modified time of files may not work because of how your build system touches content. The fool-proof approach is to use a brute-force comparison between all the binary data in two build folders.

The ease with which your build system can compute these types of file set differences depends greatly on the language and tools of the build system. For example, if your build system is driven by C++, a binary data compare of a 40GB build would be a gnarly and less-than-ideal process. In contrast, if your build system is driven by Python, you can simply call dircmp, which gives you all the proper delta data between files in two directories.

Figure 2 below shows an example of build deltas. Blue indicates modified files, red indicates deleted files, and green indicates new files. In our example patching system, the new and modified files are included in archives, shown on the right.

Figure 2: Determining build deltas

Creating a patch definition

Once your build system can calculate what’s changed between two builds, the next step is to merge that data into some form that allows the client to consume it. Patch definitions are used to list key changes in builds over time, such that we can minimalistically update the client to the latest build.

We need a few pieces of information in each patch definition to help guide a client to the required actions to update itself. Here are a few examples of the type of information that should go into a patch definition:

build number – What build is this? A simple integer is easiest to track.

target region – If you distribute your game internationally, there may be restrictions on the types of patches/content you can ship to a specific region.

files to add – This list should include new archives, as well as specific loose files that need to be added to the local build.

files to remove – At times, in the course of builds, you’ll succeed in completely replacing an old build, or there may be some security/privacy risk with old data existing on the client disk. Having the ability to remove files from disk in these situations is useful.

files to binary update – For files that need direct, in-place binary patching, this can present a list of tuples, the content to be patched, and the patch file to use.

patch importance – Is this patch required before the user can play? Or can it be streamed in the background?

Determining what files to download

Once you can create per-build patch definitions, the next step is to allow the client to consume this information. The process is generally as follows:

The client queries the patch server, sending the local version and other metadata.
The patch server responds with some form of file information.
The client processes the information and begins requesting new patch data to update the local copy.

There are two primary ways (with lots of variants) to make the determination of what files to download – at the client level or at the server level. Your choice of implementation is highly dependent on the engineering resources that are available to you. The simplest approach in terms of server technology is to keep a text-based manifest file on the server that lists all the patches, their versions etc. This entire manifest file is passed to the client upon request, and the client is responsible for building up the series of file requests to update the local copy.

While simple to implement, this approach quickly runs into limitations. Significantly, this approach requires some advanced logic built into the client to parse the manifest file and generate the request list properly. If a large content-shift occurs (for example, you change the manifest file format), the client will likely need special processing to handle the changes, and may require a patch of the client itself before it’s able to update the content.

A much more complex but scalable solution is to keep all the version data on the patch server, listed as entries in a database. The client provides some simple metadata about the state of the local data (easily encodable in a URL) to the server. The server then computes the proper series of actions the client needs to perform in order to update itself, and transfers an ‘update script’ to the client for direct execution.

The main benefit of this server-based approach is that the computation of what the client needs to do in order to update the local state is all handled on the server. Thus, as the update logic changes, the client can remain neutral to those issues and simply react accordingly. This also allows the client to generally store less data needed for the update process (for instance, the client may only need to store its region and build number). The server can store the rest of the information needed to complete the update process, as well as provide the client with more advanced functionality, like grouping multiple patches into a single request update action.

Applying patches

Once the client has a clear roadmap of what’s required to update the local build, the next step is to actually update the data. For our simple system of downloading new archives, updating content is easy – we download the bits and write them to disk. Done. Let’s get tacos.

Eventually you will encounter a situation where you need to update the game client itself. This can be tricky if the game is running. To solve this problem, most PC games distribute a separate application that checks for patches and updates the local state, including the executable code. Typically these applications are easiest to generate as standalone applications that can patch and then launch the game itself.

For embedded environments, applying patches is a bit trickier. For example, on consoles the base data is shipped on DVD, so you generally have to write patches to the hard drive and check for content there first. Mobile platforms have a whole separate set of requirements that I won’t get into. Thankfully, most of those platforms contain APIs to help out with this process of applying patches, which makes things a bit easier.

Determining what files to delete

Over time a client can accumulate many patches, and in some cases, it may have lots of data that is no longer needed. For example, if all of the files in a given patch have been replaced by newer versions in subsequent patches, then the files in the original patch will never be used and are not needed. To keep the user’s machine from consuming disk space unnecessarily, it’s helpful to identify such instances and allow a patch to delete files from disk.

An archive that’s no longer needed is one whose assets are entirely replaced by newer archives. To test for this, your build system needs to be modified such that a target patch-archive has the ability to query newer builds to determine if it’s relevant any more; depending on how your build system is set up, adding this processing can be easy, or months of man-work, so make sure you take full stock of your system before trudging down the path of enlightenment.

Notes

Binary-level file patches

If you do a search for ‘patching game content’, your results will include numerous articles that describe how to minimally modify the on-disk contents of a file at a binary level. Typically this is done by computing the difference for a file and shipping only the difference to the client, resulting in a smaller transfer and a faster patching process. Unfortunately most of the research on patch generation revolves around how to patch executable files (see e.g. Courgette (source), bsdiff, and DCA). Very little (if any) research has been focused on patching binary assets like textures, models, and sounds.

Consequently the patching system described in this article focuses less on the traditional notion of a ‘patch’ and more on a process that allows us to distribute specific assets to clients so that we can easily update the clients to the latest build. With modern compression technologies this process can result in the transfer of fairly small files to users, and the mental overhead of maintaining this type of system is significantly lower. If however you’re one of the brave souls who needs to do the more traditional version of patching, here are some quick notes.

As a starter, I suggest taking a look at XDelta, which is a fairly straightforward command-line tool: You can run a simple command to create a patch, and another command to apply a patch to a file. The app is open-source so that you can build it into a custom part of your client executable. I haven’t seen XDelta produce amazing compression results, but it does the job fairly well in terms of patching. It’s also worth noting that XDelta doesn’t produce very small deltas for archive files, mainly because the predictive look-ahead model that it uses does not fare well with the contiguous file data laid out in archive files. You’ll quickly find that simply running XDelta on two archive files won’t produce the net savings that you’re looking to gain. As such, it may be more beneficial to generate patch files for each of your assets, and archive/compress those for transfer to the client.

If you’d like to roll your own patching system for arbitrary binary content, be warned that this is a tricky problem. While it’s easy to come up with a naive solution, you’ll soon realize that there can be multiple patch points to a single file, which usually throws simple solutions out the window. You’ll also realize that the delta between two files may not be linear – it may include deletions, insertions, and replacements, which is difficult to track. Make sure you’ve got manager approval before going down this route ;)

File processing and restricted systems

Newer operating systems place restrictions on what content can be executed, mostly requiring that applications be signed before they can be executed. Once an application is signed, the system can detect any change to the application, whether the change is introduced accidentally or by malicious code. How to sign your assets and properly distribute/patch them to the client on a given platform is left as an exercise to you. Each OS seems to have its own policies on that process, and your actions will be depend on the systems to which you ship your game.

Conclusion

It takes some effort to implement a patching system, but the benefits are well worth it. A patching system gives you increased flexibility and control over your game, lets you provide a better user experience, and gives you the opportunity to market new content to your users.

Source code

My github account has some very simple source code that implements the patching system described above. (Be warned, it uses Python.) The code includes a mock build system, a content server, and a client. You can use this system with the following commands:

python build/gen_patch.py
python server/httpd.py
python client/client.py

Command (1) generates a patch from two builds.
Command (2) starts a server from which to distribute patches.
Command (3) calculates differences in the local version (assuming the client already has the initial build) and pulls down a delta patch from the content server.

AltDev Student Summit – Speaker Change for Day 2

AltDevConf — Sun, 11 Nov 2012 13:03:42 +0000

Due to a last minute cancellation, the session “What is a Player Champion” given by Romana Ramzan will not be shown. Instead it will be replaced by “A Day in the Life of a Creative Director” given by Jurie Horneman of Mi’Pu’Mi Games:

When preparing for this talk I made a list of all the things I do on a regular basis as the creative director of Mipumi Games. I then realized that a lot of the things I do are not what one might associate with being a creative director. I will be talking about what it takes to work on the director level at a small game development company.

And here’s the bio for Jurie

Jurie Horneman is the creative director of Mipumi Games, based in Vienna, Austria. He has been making games since 1991 and has worked as a game designer, producer and programmer. He got into the games industry via the Atari ST demo scene.

We’re very grateful to Jurie for stepping in at the last minute. You can see his session at 1pm PST / 4pm EST / 9pm GMT today, Sunday 11th November.

We’ll be restarting the conference for Day 2 after a very successful first day in a few hours, at 11am PST / 2pm EST / 7pm GMT. Be sure not miss these great sessions live, and join in the conversation using #AltDevConf on Twitter.

Speaker Highlight – Anne Toole

AltDevConf — Thu, 08 Nov 2012 15:29:28 +0000

Our last speaker highlight is here, and the conference is only two days away, which is very exciting! Lastly, we’d like to introduce you to Anne Toole, and her session “Fitting Writers into the Development Process”:

Game developers are sometimes left scratching their heads on how to incorporate writers into the game development process. This talk will take attendees through the stages of development to identify how a writer can integrate with the team and add value to the game, whether it be narrative-driven or narrative-light.

And Anne’s bio is here:

Anne Toole is a writer and designer who focuses on bringing fun, meaningful, and relevant content to games. Her credits include the MMO “Wizard 101″ and “The Witcher”, an RPG which earned her a Writers Guild nomination. As a contract writer and designer, she has written games for the console, PC, and mobile in a variety of genres. Anne has also served on staff as the first head writer for the MMO “Stargate Worlds” and has written for NBC serial “Days of Our Lives”. Anne has spoken about games and entertainment for the inaugural Nokia OpenLab as well as at GDC Europe, GDC Online, South by Southwest, LOGIN Game Conference, and Comic-con International. She contributed chapters to the IGDA’s Professional Techniques for Video Game Writing and Writing for Video Game Genres books and serves on the board of the Los Angeles chapter of the IGDA. A citizen of the EU as well as the US, Anne holds a degree in Archaeology from Harvard.

It’s nearly time for the Student Summit and you’ve now met all the exciting speakers we have lined up. Check out information on the whole event and make sure that you sign up through Eventbrite!

Starting a new game project? Ask the hard questions first

Raul Aliaga Diaz — Thu, 08 Nov 2012 00:51:38 +0000

We have all been there. You wanted to start a new game project, and possibly have been dreaming of the possibilities for a long time, crafting stories, drawing sketches, imagining the dazzling effects on that particular epic moment of the game… then you start to talk to some friends about it, they give you feedback, and even might join you in the crazy journey of actually doing something about it.
Fast forward some weeks or months, and you’ve been pulling too many all-nighters, having lots of junk food and heated discussions. You might even have a playable prototype, several character models, animations, a carefully crafted storyline, a website with a logo and everything but… it just doesn’t feel right. It’s not coming together and everyone involved with the project is afraid to say something. What happened? What went wrong? How such an awesome idea became this huge mess?

Usually all game projects emerge from a simple statement that quickly pushes the mind to imagine the possibilities. Depending on your particular tastes, background and peers, these statements can be like: “Ace Attorney, but solving medical cases, like House M.D.!” (Ace House™), “Wario meets Braid!” (Wraid™) , “Starcraft but casual, on a phone!” (Casualcraft™). These ideas can be just fine as starting points, but somewhere down the line the hardest question is: Is this game something worth doing?.

When you work at a game studio and a new idea arises, that’s the first question it faces. And depending on the studio’s strengths, business strategy and past experiences, the definition of “worth” is very, very specific. It usually involves a quick set of constraints such as: time, budget, platforms, audience, team, among others. So for a particular studio that has developed Hidden Object Games and has done work for hire creating art, characters and stories for several other games, an idea like Ace House™ can be a very good fit, something they can quickly prototype and pitch to a publisher with convincing arguments to move it forward. However, in the case of a studio focused solely on casual puzzle games that has just one multi-purposed artist/designer and two programmers, it can be rather unfeasible, much more if all but one says: “What’s Ace Attorney? What’s House M.D.”?

Ok, you might say, “But I’m doing this on my own, so I can fly as free as I want!”. That’s not entirely true. If you want to gather a team behind an idea, all of the team members must agree that the project is worth doing, and even if you do it on your own, you must answer the question to yourself. Having less limitations can positively set you free, but take that freedom to find out your personal definition of worth, not to waste months on something that goes nowhere. Unless you can, like, literally burn money.

The Joker Game Studios

Why is the project worth doing? is the hardest question, and the one that must be answered with the most sincere honesty by everyone involved. The tricky part is that it is widely different for many people working on a game project out of their regular job or studies. It can be to start learning about game development, to improve a particular set of skills, to start an indie game studio, to beef up a portfolio, etc. It is O.K. to have different goals but they all must map to a mutually agreed level of time commitment, priorities and vision. But even if you figured this out, there are still other issues.

All creative projects can be formulated as a set of risks or uncertainties, and the problem with video game development -given its highly multidisciplinary nature- is that is very easy missing to tackle the key uncertainties, and start working on the “easy” parts instead.

So for example, for the Ace House™ project, it can be lots of fun to start imagining characters and doctors, nurses, patients and whatnot; there’s plenty of T.V. series about medical drama to draw inspiration from, and almost surely you can have a good time developing these characters, writing about them, or doing concept art of medical staff in the Ace Attorney style, but, What about the game? How do you precisely translate the mechanics from Ace Attorney to a medical drama? How is this different from a mere re-skin project? Which mechanics can be taken away? What mechanic can be unique given a medical setting? How can you ensure that Capcom won’t sue you? Are there any medic-like games already? How can we blend them? Is it possible? Is this fun at all? Is “fun” a reasonable expectation or should the experience be designed differently?

Let’s talk about Wraid™ now. If Konami pulled off “Parodius” doing a parody from “Gradius”, How cool would it be to do a parody of Braid using the characters from the Wario Ware franchise? Here you have a starting point for lots of laughs remembering playing Braid, and putting there Wario, Mona, Jimmy T. and the rest of the characters on the game, wacky backgrounds, special effects and everything. But: Is this reasonable? Let’s start with the fact that Konami owns the IP of Gradius so they can do whatever they want to it. Can you get away with making a parody of both Nintendo and Jonathan Blow’s IPs? Sure, sure, the possibilities can be awesome but let’s face it: It is not going to happen. What can be a valuable spin-off though? What if Wario Ware games have a time-manipulation mechanic? What if you take Wario’s mini games and shape them around an art style and setting akin to Braid? (Professor Layton? Anyone?) How can you take the “parody” concept to the next level and just make “references” to lots of IP but the game is something completely new in itself?

What about Casualcraft™? Starcraft can be said to have roughly two levels of enjoyment: as an e-sport, and whatever other pleasure the other people draw from it. If we want to make it casual, it should not be an e-sport, should it? If you’re a Starcraft fan and have experience doing stuff for smartphones, you might think “This should be easy, I can make a prototype quickly”, and given that a mouse interface can be reasonably translated to touch, you start coding, and get a lot of fun implementing gameplay features that pumps all your OOP knowledge and creative juices to the roof. But… what does exactly mean “Casual Starcraft”? How can a strategy game be casual? What is the specific thing different from the e-sport experience that we want to bring to a phone? Is it the graphics? Is the unit building-leveling? Is playing with other friends? Which one of those should we aim? Can still be an RTS? What about asynchronous gameplay? Can this be played without a keyboard? Can still be fast? Would it fit on a phone? People that play on a phone: would they play this game?

So, all these are tricky and uncomfortable questions, but they are meant to identify the sources of risk and figure out a way to address them. Maybe the ideas I presented here are plain bad, sure, but they are only for illustrational purposes. Since I started working in games, I’ve seen countless ideas from enthusiasts that are not really too far away from these examples anyway. The usual patterns I’ve seen are:

Not identifying the core valuable innovation, and failing to simplify the rest: It is hard to innovate, much harder to do several innovations at once. Also, people have troubles learning about your game having too much simultaneous innovations and can quickly get lost, rendering your game as something they simply “don’t get”. The key is to identify what’s the core innovation or value of your idea, the one single thing that if done right, can make your game shine and then adjust all the rest to known formulas. And by “key” innovation I mean something important, critical, not stuff like “I won’t use hearts as a health meter but rainbows!”. That can be cute, but it’s not necessarily a “key innovation”.

Putting known techniques and tools over the idea’s requirements: “I only do 3D modeling so it has to be 3D”, “I know how to use Unity so it has to be done in Unity”, “I only know RPG Maker so let’s make an RPG”. It is perfectly O.K. to stick to what you feel comfortable doing, but then choose a different idea. A game way too heavy on 3D might be awesome, but completely out of scope for a side project. Unity can be a great engine, but if all the other team members can work together on Flash on a game that it is completely agreed to live primarily on the web, it can’t hurt to learn Flash. RPG Maker is a great piece of software, but if you can’t really add new mechanics and will concentrate only in creating a story, why not just develop a story then? A comic book project is much more suitable. Why play your particular game when everyone that is into RPG’s surely has at least two awesome ones that they still can’t find the time to play them? Instead of crippling down the value or feasibility of your idea to your skills and resources, change the idea to something that fits.

Obsessing over a particular area of the game (tech, story, etc): This usually happens when the true reason to do the project is to learn. You’re learning how to code graphic effects, or how to effectively use Design Patterns to code gameplay, a new texturing technique, vehicle and machines modeling, a story communicated through all game assets and no words, etc. You can get a huge experience and knowledge doing this. But then it’s not a game meant to be shipped, it is a learning project, or an excuse to fulfill something you feel passionate about.

Failing to define constraints: The romantic idea of developing a game until “it feels right”. If Blizzard or Valve can do it, why can’t you? Well, because at some point, you’ll want to see something done and not feel that your time has gone to waste. The dirty little secret is that constraints almost all the time induce creativity instead of hinder it. So choose a set of constraints to start with, at least a time frame and something you would like to see done at particular milestones: Key concept, Prototype, Expanded Prototype, Game.

Refusing to change the idea: This is usually a sign of failing to realize sunken costs. “I’ve spent so much time on this idea, I must continue until I’m done!”. The ugly truth is that if you’re having serious doubts, those will still be there and will make you feel miserable until you address them, and the sooner you act, the better. It can be that all the time you spent is effectively not wasted, but only when you frame it as your learning source to do the right things.

So if you’re starting a new game project, or are in the middle of one, try asking the tough questions: Do you know why is worth doing? Do all people involved agree on that? Are you making satisfying progress?

Are you sure there isn’t a question about your project you are afraid to ask because you fear that it can render your idea unfeasible, invaluable or messy?

Don’t be frightened, go ahead. If it goes wrong, you will learn, you will improve and the next idea will get to be shaped much better.

Speaker Highlight – Ted Spence

AltDevConf — Wed, 07 Nov 2012 12:04:06 +0000

For readers of AltDevBlogADay, Ted Spence should be a familiar face. He joins the line up for the AltDev Student Summit with a session called “What Software Developers Should Expect From the Games Industry”:

Mr Spence, a twenty-year veteran of the games industry, will talk about his experience breaking into the games industry as a programmer, and how it differs from work in corporate software development. Mr. Spence joined the CD-ROM gaming wave in the early 1990s and repeatedly switched back and forth between game development and business software / analytics, eventually resulting in the startup EEDAR, an analyst firm that provided services to all of the top 20 publishers in the videogame industry.

For those who want to learn a bit more about Ted, here is a short bio

Mr. Spence is a prolific video game technology consultant and developer. In 2007, he joined EEDAR, a startup company providing analytics services to the videogame community. As CIO, he built all technology and infrastructure for the startup as it grew from a one person company to a metadata powerhouse, providing services to all of the top 20 videogame publishers and many of the top retailers.

Prior to EEDAR, Mr. Spence managed technology infrastructure for Knowledge Adventure, where he ran IT and Internet operations during its spinoff from Vivendi Universal. With a broad background and experience in a range of software and hardware technologies, Mr. Spence has developed a management methodology focusing on AGILE / SCRUM, and he writes about it regularly for gaming development websites AltDevBlogADay and Gamasutra.

Mr. Spence holds a Bachelor of Arts, Philosophy, from California State University, Fullerton, and lives in Oceanside with his wife and daughter.

It’s nearly time for the Student Summit and Ted, and all the other speakers, are working hard to create excellent content for you this weekend. Check out information on the whole event and make sure that you sign up through Eventbrite!

AltDev Student Showcase – The Judges

AltDevConf — Tue, 06 Nov 2012 12:08:26 +0000

We’ve announced the finalists for the AltDev Student Showcase already, but we keep talking about the expert judges who are going to be giving their feedback and deciding who will be going home with the Grand Prizes that Unity Technologies have put up. So, it’s time to introduce the judges!

We’ve got an exciting array of talent and a diverse spread of skills and experience lined up to tell our competitors exactly what they think of their projects. We’re going to be running two sessions of the Student Showcase, one on each day of the AltDev Student Summit, and we’ll be changing the panel completely between sessions.

On Saturday, the Session 1 of the Student Showcase will take place at 13:00 PST / 16:00 EST / 21:00 GMT / 06:00 JST. Our panelists for this one will be:

Kristen Bornemann – Development Director at ArenaNet
Phil Carlisle – AI Guy at Mindflock
Alex Darby – Technical Director at GamerCamp
Shawn Kirsch – Senior Production Services Technician at Sony Imageworks

The second session will be held on Sunday at 11:00 PST / 14:00 EST / 19:00 GMT / 04:00 JST and will be judged by

Mike Acton – Engine Director at Insomniac Games
Claire Blackshaw – Technical Designer at Climax Studios
Marcus Montgomery – Senior Designer at Backbone Entertainment
Pier Tortorelli – Associate Producer at Ubisoft Reflections

The AltDev Student Showcase is just one of the exciting things we have lined up as part of the AltDev Student Summit. Please check out the mini-site for more information and don’t forget to register to join us for this chance to learn how the pros assess projects.

AltDev Student Summit – Microtalks

AltDevConf — Mon, 05 Nov 2012 12:00:48 +0000

We offered speakers the opportunity to do some shorter sessions, following more of a microtalk format, and here they are.

George Kokoris – The Cutting Room Floor

Session – Game design schools usually spend a little time teaching students about scope, but it’s not always given the importance it really deserves. Scope can make or break any project, from the smallest 48-hour game jam entry to the most opulently funded AAA blockbuster. Scoping mistakes are one of the most common reasons that game projects fail, but they are also avoidable with a little perspective and determination.

Bio – George is an ex-cinematographer, a graduate of Full Sail University, and a game designer at Microsoft Studios. As a member of game development collective Colorcave, he’s contributed code and design to small-scale indie titles as well as to Microsoft’s AAA portfolio. He’s a stubborn advocate for rapid prototyping and iterative design, and will ramble at great length about both if given half a chance. George currently resides in the Pacific Northwest.

Travis Sanchez – The Blue Screen of Death

Session – Game crashing for everyone but you? It happens to all of us. Learn how professionals deal with bugs before shipping their game, and how you can improve your own beta or first deployment.

Bio – Travis is a graduate of New Mexico Tech with a BS in CS who worked for 4 years at High Impact Games shipping games across PSP, PS2, PS3, XBox 360, Wii and 3ds. Travis is primarily a programmer who has worked on optimising code and developing efficient multithreaded implementations for the XBox 360 and PS3, amongst many other things. Travis recently left the games industry to take a position at Google, where he is a Software Engineer

These great sessions and many more are taking place as part of the AltDev Student Summit, this coming weekend. Check out more information on the conference mini-site and make sure you sign up through Eventbrite to be kept up to date.

Speaker Highlight – Mitu Khandaker

AltDevConf — Sun, 04 Nov 2012 11:58:27 +0000

UK developer Mitu Khandaker is our next speaker highlight, in a session she has titled “A Day in the Life of a Solo Developer”.

Lessons from the lonely frontier of being a solo indie developer, from the one-woman team behind The Tiniest Shark Ltd, currently developing first commercial title, Redshirt. As a solo developer for the last 18 months (bonus content: while also trying to finish a PhD), Mitu chronicles the highs, the lows, and the cold, hard truths behind a day in the life of a new lone indie. and how you can prepare if you’re thinking of doing the same.

Mitu’s bio is right here for you to learn more about her:

Mitu Khandaker is an independent games developer at one-woman studio, The Tiniest Shark Ltd, currently developing its first commercial title, Redshirt. She is also a PhD researcher in games at the University of Portsmouth, UK. She occasionally writes for publications such as Continue Magazine, and was also named one of Develop Magazine’s Top 30 Under 30 upcoming game developers of 2012.

We’re really pleased to have Mitu and all our speakers involved in this great event, and we hope you’ll join us. You can read more information, including the full schedule, on the conference mini-site, and don’t forget to register through Eventbrite.

AltDev Student Summit – Breaking In Panel

AltDevConf — Sat, 03 Nov 2012 12:53:52 +0000

Knowing how to get your start in the industry can be tough. There’s no one fixed route to ensure that you get hired – if getting hired is even your aim. In this panel session, we’ve assembled some recent entrants into the world of professional video game development who have various points of view. They’ve approached the problem different ways already, and achieved different things, and during the panel they’ll be discussing what worked, what didn’t and sharing their experiences with the audience in order to help them to “Break In”.

Our four panelists are:

Becky Taylor (Stomp Games) – Becky has been PC-gaming since she was 2 and front-end web programming since she was 7. Becky is a very determined individual, getting published in the Davis Art Publication’s Communicating by Graphic Design series whilst still in highschool, and being named Fitchburg State University’s Interactive Media Student of the Year for 2012. Becky graduated with a Bachelor’s of Science in Communications Media with a concentration in Interactive Media. She currently works at Stomp Games as a Marketing Associate for Robot Rising.

Ben Holschuh (2K Games) – Ben is currently working as a Production Assistant at 2K Games based out of Novato, CA. While no producer-y things can be directly seen in the games, he has worked on both Borderlands 2 and Bioshock Infinite in his short 6 months with the company so far. In 2011 he traveled to the IGDA Summit and Casual Connect conferences as an IGDA Scholar; an experience that helped him get where he is today. He graduated from Ohio University with a degree in Digital Media, a minor in Japanese, and a burning passion for all things games.

Sarah Howarth (d3t) – Sarah is an electronics engineer and programmer, currently working at d3t Ltd. Her career began at Juiced Games as part of her high school work experience, and she hasn’t looked back as she has moved through Bizarre Creations and TT-Fusion, working on a number of titles such as James Bond: Bloodstone, Lego Pirates of the Caribbean and Lego Harry Potter: Years 5-7. At d3t, Sarah focuses on expanding the range of services offered by the company, as well as developing new approaches for platforms such as Playstation Mobile.

Tyler Coleman (Retora Game Studios) – Tyler is Founder and Creative Director of Retora Game Studios, an independent game studio in Phoenix, Arizona. In the past year, he has worked on multiple titles, with the first mobile release earlier this year. His passions include procedural generation, community involvement, game education, and Lua.

With panelists with such diverse views on how you can get your start in the games industry, this should be a session not to be missed. You can read more about the whole event at the conference mini-site, and please don’t forget to register to join us, November 10th and 11th from 11am PST / 2pm EST / 7pm GMT / 4am JST.

Speaker Highlight – Brandii Grace

AltDevConf — Fri, 02 Nov 2012 12:00:04 +0000

Next up is Brandii Grace with a session called “The Intern’s Secret Survival Guide”:

I hear the same tragic story over and over: You finally landed an internship at a major game company! You just know this is your “big break” and can’t wait to get to work! Problem is… no one has given you any work to do. In fact, no one seems to know what to do with you or have time to figure it out. Before you know it, the internship is over and all you’ve done is sit around catching-up on webcomics. Learn the secret tricks my students used to transform internships into full time positions!

And as always, here is where you can learn a little bit more about Brandii:

Brandii Grace is the Chief Creative Officer of Transform Entertainment – a pioneering company recently featured on Gamasutra for working to create AAA console games specifically targeting the growing women’s market. Brandii has nearly 10 years of design and development experience in the games industry working with a wide array of platforms, genres, and markets for games ranging from giant AAA MMO’s to award-winning indie titles. Brandii, who graduated from Western Washington University with a Bachelor’s in Computer Science, is an alumna of the IGDA Scholars program and received the 2012 IGDA MVP award. She has taught game design innovation, been published in respected game development textbooks, and helped start more than one collegiate game program.

Brandii’s session is just one part of what’s promising to be an excellent event. You can learn more at the conference mini-site, and we really hope that you will join us November 10th and 11th to learn more from these excellent speakers.

Bitsquid Foundation Library

Niklas Frykholm — Thu, 01 Nov 2012 21:42:31 +0000

Today I want to talk a bit about the Bitsquid Foundation Library that we recently released on Bitbucket (under the permissive MIT license).

It’s a minimalistic “foundation” library with things like memory management and collection classes. The idea is to have something that can be used as a reasonable starting-off point for other open source projects.

The library makes some interesting design choices that touches on topics that I have already talked about in this blog and that I think are worth elaborating on a bit further. It also serves an example on how these techniques can be used in “real world” code.

Separation of data and code

The foundation library implements the idea of separating data definitions and function implementation, that I talked about in this article.

Data is stored in structs with public members (prefixed with an underscore to indicate that you should not mess with them unless you know what you are doing) that are found in *_types.h files. Functions that operate on the data are written outside of the struct, in separate *.h files (and organized into namespaces).

For example, the data definition for the dynamic Array class is found in collection_types.h:

template struct Array
{
    Array(Allocator &a);
    ~Array();
    Array(const Array &other);
    Array &operator=(const Array &other);
    
    T &operator[](uint32_t i);
    const T &operator[](uint32_t i) const;

    Allocator *_allocator;
    uint32_t _size;
    uint32_t _capacity;
    T *_data;
};

The struct contains only the data used by the array and the operators which C++ forces us to implement as member functions.

The implementation of these functions, as well as the declaration and definition of all other functions that operate on the arrays are found in the array.h file. It contains things like:

namespace array
{
    template
    inline uint32_t size(const Array &a)
    {return a._size;}
    
    template
    inline bool any(const Array &a)
    {return a._size != 0;}
    
    template
    inline bool empty(const Array &a)
    {return a._size == 0;}
}

template 
inline Array::Array(Allocator &allocator) :
    _allocator(&allocator), _size(0), _capacity(0), _data(0)
{}

This way of arranging data and code fills two purposes.

First, it improves compile times by reducing header inclusion. Header files that want to make use of arrays only need to include collection_types.h, which just contains a few struct definitions. They don’t have to drag in array.h, with all its inline code.

Headers including other headers indiscriminately because they need their types is what leads to exploding compile times. By only including the minimal thing we need (the type definitions), compile times are minimized.

Second, and more importantly, this design allows the collection types to be freely extended. Is there anything you miss in the array interface? Perhaps you would like shift() and unshift() methods? Or binary_search()?

No problem. If you want them you can just add them, and you don’t even need to modify array.h. Just create your own file array_extensions.h or whatever, and add some new functions to the array namespace, that manipulate the data in the Array interface. The functions you create will be just as good as the functions I have created.

Note that this isn’t true for traditional class designs, where you have first-class citizens (methods) and second-class citizens (external functions).

The foundation library has some interesting examples of this. For example, the string_stream functions don’t operate on any special StringStream class, they just directly use an Array. Also, the hash and multi_hash interfaces both work on the same underlying Hash struct.

I believe that this design leads to simpler, more orthogonal code that is easier to extend and reuse.

Memory management

The library implements the allocator system mentioned in this article. There is an abstract Allocator interface, and implementations of that interface can provide different allocation strategies (e.g. ArenaAllocator, HeapAllocator, SlotAllocator, etc).

Since I want to keep the library platform independent, I haven’t implemented a PageAllocator. Instead, the MallocAllocator is used as the lowest allocator level. If you want to, you can easily add a PageAllocator for your target platform.

For the same reason, I haven’t added any critical section locking to the allocators, so they aren’t thread safe. (I’m thinking about adding an interface for that though, so that you can plug in a critical section implementation if needed.)

The system for temporary allocations is kind of interesting and deserves a bit further explanation.

Most games have a need for temporary memory. For example, you might need some temporary memory to hold the result of a computation until it is done, or to allow a function to return an array of results.

Allocating such memory using the ordinary allocation system (i.e., malloc) puts a lot of unnecessary stress on the allocators. It can also create fragmentation, when long lived allocations that need to stay resident in memory are mixed with short lived temporary allocations.

The foundation library has two allocators for dealing with such temporary allocations, the ScratchAllocator and the TempAllocator.

The ScratchAllocator services allocation requests using a fixed size ring buffer. An allocate pointer advances through the buffer as memory is allocated, and a corresponding free pointer advances as memory is freed. Memory can thus be allocated and deallocated with simple pointer arithmetic. No calls need to be made to the underlying memory management system.

If the scratch buffer is exhausted (the allocate pointer wraps around and catches up with the free pointer), the ScratchAllocator will revert to using the ordinary MallocAllocator to service requests. So it won’t crash or run out of memory. But it will run slower, so try to avoid this by making sure that your scratch buffer is large enough.

If you forget to free something allocated with the ScratchAllocator, or if you accidentally mix in a long-lived allocation among the short-lived ones, that allocation will block the free pointer from advancing, which will eventually exhaust your scratch buffer, so keep an eye out for such situations.

TempAllocator is a scoped allocator that automatically frees all its allocated memory when it is destroyed (meaning you don’t have to explicitly call deallocate(), you can just let the allocator fall out of scope). This means you can use it everywhere where you need a little extra memory in a function scope:

void test()
{
     TempAllocator1024 ta;
     Array message(ta);
     ...
}

The BYTES argument to TempAllocator specifies how much stack space the allocator should reserve. The TempAllocator contains char buffer[BYTES] that gets allocated on the stack together with the TempAllocator.

Allocation requests are first serviced from the stack buffer, then (if the stack buffer is exhausted) from the ScratchAllocator.

This means that TempAllocator gives you an allocator that can be used by all collection classes and will use the fastest allocation method possible (local stack memory, followed by scratch buffer memory, followed by malloc() if all else fails).

Minimalistic collection types

The collection classes in the library are distinctly anti-STL. Some of the important differences:

They use the allocation system described above (taking an Allocator as argument to the constructor). They can thus be used sensibly with different allocators (unlike STL types).
The use the data/function separation also described above, which means that the headers are cheap to include, and that you can extend them with your own functionality.
They use a minimalistic design. They assume that the stored data consists of plain-old-data objects (PODs). Constructors and destructors are not called for the stored objects and they are moved with raw memmove() operations rather than with copy constructors.

This simplifies the code and improves the performance (calling constructors and destructors is not free). It also saves us the headache of dealing with storing objects that must be constructed with Allocators.

Personally I like this minimalistic approach. If I want to keep non-POD data in a collection, I prefer to store it as pointers anyway, so I have control over when and how the data is constructed and destroyed. I don’t like those things happening “behind my back”. You may disagree of course, but in that case you are probably happy to use STL (or boost).

Another example of choosing minimalism is the Hash class. The hash uses a fixed key type which is a uint64_t. If you want to use a key that doesn’t fit into 64 bits, you have to hash it yourself before using it to access the data.

And more?

I’m planning to add some basic math code to the library, but haven’t gotten around to it yet.

Is there anything else you’d like to see in a library like this?

This has also been posted to the Bitsquid blog.