These days I just code with the knowledge I have,  guess how things might be and, when I get stuck, I jump into a reference manual or jump onto google and find the bit of information I’m missing. As a consequence, I only just found out a few days ago, purely by an accidental web link, that STL provided a bitset template.  I can see understand perfectly why I didn’t know this existed. It’s not that I haven’t had use for a bitset template in the past,  its just that its not something complex enough that I would ever needed to google for information on it.

The combined knowledge of the universe is now at my finger tips, kindly indexed by keywords. I know the answer to the ultimate question of life, the universe and everything is in cyberspace, if only I can find the right question.  But does this mean I don’t need to store information in my head anymore – just retask my brain to be a master of composing the correct keywords for a problem.  Is it really ok to be crippled if I have no internet connection.

And its not just knowledge that has been replaced by keyword access. When I was young, my pirate friends – not me, I would never – would trade games via mail (notice no ‘e’ prefix) on cassettes (young readers should google this if you don’t know what a cassette is). Now you just type in a keyword into Vuze on your Mac – again, not me, I don’t even know what Vuze is – and you get a list of every song, tv show, movie or game and within minutes its yours – well, not yours, but you have it.  Actually, even googling is too much effort.  Just highlight the word in your browser and Apture will find the information for you.

Admittedly, I appear to have a lot less time than I used to when I was young. I’ve done numerous experiments, including watching clocks in 70s videos, and it certainly appears that time hasn’t sped up, at least in no way I have been able to measure. Maybe its facebook, twitter, dayjob, family and nightjob – I certainly didn’t have all these when I was younger. I could just take a book into the bedroom for the weekend and emerge 3 days later a little bit smarter – yeah, it was a sad childhood really.

So are our new iBrains really a good thing?  Time to put on the opinion cap.  Yes, its awesome having access to blogs and white papers and answers to everyones questions,  but we need to temper that with discipline. The most common forum post these days is “read the manual”,  because most questions are clearly answered in the manual but no-one bothers reading now. Its easier and quicker to jump on a forum and ask if anyone else knows the answer – maybe someone who has actually read the manual.  (Actually, I have no idea what the most common forum post is, I totally made that up to support my argument. ).

So am I going to return to reading manuals front-to-back?  I was all ready to say “absolutely, I will make a point of always reading manuals properly from now on!”  But to be honest, I doubt it.  There’s a simple rule of the universe that all things follow the path of least resistance.  While humans are capable of using intellect to break this rule,  they generally don’t bother.  BTW, this is also a great rule to follow if your ever designing tools or interfaces that you want other people to use – make sure your stuff is the path of least resistance…

So what’s the morale to all this?

TL;DR

Learning from other code

Often, through my professional career, I’m forced to work in other people’s frameworks. Most often this is very frustrating as their way to solve a problem is in contrary to how I’d prefer the problem solved. However, its important to keep a cool head and try to understand the philosophy behind their code.  In most cases, there are a few things that do work well in the framework, and often, once you understand it all, you may even be swayed over by the advantages of their technique.

Learning from other languages

I recently had to code a simple utility in C#.  I don’t write utilities often and have very little experience in C#, so the process was a small learning curve.  However, by learning some of C#’s string handling interfaces, I was then inspired to add some new helper functions to my C++ codebase to make certain things easier to do, like having a simple Split function that divides strings up into arrays of smaller strings. Some languages are very cleanly designed and have some great features. You’ll often find you can duplicate those features to a reasonable extent in your own language of choice. So as well as having another language available to you, when you need it,  you also have absorbed interface design skills from that other language. It’s not just a case of every language having its use. It’s more that every language has found creative ways to solve common problems.

Learning from other engines

You will have seen other articles about the importance of home projects. Your “paid” projects are generally restrictive as to how much experimentation and personal growth you can pull out of them due to design, production and time constraints.  Personal projects are also grant a great opportunity to experiment with other engines. Some of the top engines out there have very mature and well thought out pipelines and programming methodologies. Producing a game prototype in various engines is a great way of gaining insight into how your own pipeline can be improved. Always remember that the intention here is not (necessarily) to find a new engine to replace your current engine. The purpose of doing a home project in a new engine is mainly to boost your knowledge by seeing how a problem can be solved in a very different way from how you would normally solve it. Until you have completed a substantial project in another engine, you never really know how these different techniques will cope with all the bits and pieces you need to tie together to make a deliverable game.

Taking the time to learn

Certainly, experience can be attained by just solving new and different problems, evolving your skillset to meet the new challenges. However, when you look at other people’s code, languages, and engines, what you are seeing is the result of those people’s already evolved skillsets. Understanding the decisions made there is a way to rapidly enhance your own ability.

It’s not always fun to have to learn how other people have done things, and generally we code at home because we enjoy it, so we avoid things that aren’t fun. So I like to look at it as “professional homework”. The pain of learning other peoples techniques is often because your brain is being challenged to solve problems in ways it is unfamiliar with. Like everything in life, the healthy things often taste terrible. Taking the time to do your homework will make your decisions much more well-rounded in future.

So what’s it good for?

How to set it up

  SOCKET m_listener;
  SOCKET m_connection;

  SOCKET listener = socket(PF_INET, SOCK_STREAM, 6);
  u_long nonBlocking = 1;
  ioctlsocket(listener, FIONBIO, &nonBlocking);

  sockaddr_in myAddr;
  myAddr.sin_family = AF_INET;
  myAddr.sin_port = htons(portNmbr);
  myAddr.sin_addr.s_addr = INADDR_ANY;
  bind(listener, (sockaddr*)&myAddr, sizeof(myAddr));
  listen(listener, 16);

  SOCKET m_connection;

  m_connection = accept(m_listener, 0, 0);
  if (m_connection != INVALID_SOCKET)
  {
    const int BUFSIZE = 8096;
    char buffer[BUFSIZE+1];
    int received = recv(connection, buffer, BUFSIZE, 0);
    if (received > 0)
    {
      buffer[received] = 0; // null terminate the buffer
      ProcessHTTPSpaces(buffer);  // spaces may come in a %20 so convert these back to spaces
      char* url = (char*)strchr(buffer, ' '); // string is "METHOD URL " + lots of other stuff
      if (url)
      {
        url++[0] = 0; // null terminate the method, increment pointer to url
        char* pSpaceChar = (char*)strchr(url, ' ');
        if( pSpaceChar ) pSpaceChar[0] = 0; // null terminate after the second space to ignore extra stuff
        if( url[0] == '/' ) url++;
        HandleRequest(buffer, url);
      }
      CLOSE_SOCKET(m_connection);
    }
  }

  http:\\127.0.0.1:1234\ExamineEntity?id=235422;verbose=true

  HTTPServer::Instance()->RegisterCallback(SZ("ExamineEntity"), SZ("Examine an entity"), &HTTPCallback_ExamineEntity, 0);

void HTTPServer::HandleRequest(const char* method, const char* url)
{
  // we only handle GET requests - may support POST in future
  if (!DMStringEqual(method, "GET"))
    return;

  // build the command and arg list
  DMString command = GrabCommand(url);

  DMHTTPArgList args;
  const char *pArg, *pValue;
  while (GrabValuePair(url, pArg, pValue))
  {
    args.push_back(HTTPArgument(pArg, pValue));
  }

  HTTPCallbackMap::iterator it = m_callbacks.find(command);
  if (it != m_callbacks.end())
  {
    it->second.callback(it->second.pUserData, command, args);
  }
  else
  {
    HTMLBuilder builder;
    builder.WriteLn("Unkown URL Command");
    SendResponse(builder.HTMLPage());
  }
}

void HTTPServer::SendResponse(const char* pData, const char *pDataType)
{
  const char *pHeader = "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\n\r\n";
  int dataSize = strlen(pData);
  send(m_connection, pHeader, strlen(pHeader), 0);
  send(m_connection, pData, dataSize, 0);
}

Conclusion

My latest contract has me working in flight simulator territory. This now introduces geocentric coordinates as well as lon/lat/alt co-ordinates on top of the usualy world space which is the core of the rendering pipeline. Geocentric co-ordinates are basically a typical cartesian system (X,Y,Z) but where (0,0,0) represents the centre of the planet and the Y axis points to the North Pole. Orientation also has several representations in the application. There is a geocentric representation which is the transform from the geocentric identity as well as the render world space orientation which is the more traditional “local to world” transform in 3d applications. All these transformations have the potential to make any code very confusing indeed, as I soon discovered coming onto the project. I spent the next few weeks researching the co-ordinate systems and coming up with some modifications to the code base to make it readable to myself and others.

Understand the co-ordinate systems you will be working in

The first thing you need to do is actually understand how the co-ordinate system works. For positional systems, this means understand exactly what (0,0,0) represents (assuming a 3 coordinate system of course). In some coordinate systems (0,0,0) might not represent a static position in space. Be aware of what direction positive is on each of the coordinates.

Some coordinates may represent curves through space. For instance, with lat/lon/alt coordinate systems, lattitude lines curved around the earth. This is significant if you are comparing distances between points in one space to differences between the same points in another space.

Understand the scale of the coordinates. Hopefully most cartesian systems will be in metres, but I’ve seen a few in centimetres. For lat/lon/alt, it is good to have a feel for how many metres a degree of longitude is. I see this as part of the classic mantra of “know your data”.

For orientation systems, understand exactly what it means to give one of your game objects an identity orientation in this coordinate system. Which direction will it face. I like to visualize the x,y,z vectors of a matrix at identity and see in which direction each axis represents with respect to an identity oriented game object. Understand what direction pitch, yaw and roll travel in. Understand what a cross product will give in this coordinate system. It only takes a few experiments to get data on all these and its generally not safe to assume that the maths libraries will do exactly as you expect if you didn’t write them yourself.

Understand what each coordinate systems strengths are

Each coordinate system is there for a reason, so learn what that reason is. If there is NO reason, then you would shouldn’t be using that coordinate system and complicating your code unnecessarily. For my project, all our coordinate systems were mandatory due to other software we were interfacing with.

Each coordinate system is particularly good at certain maths, and weak at others. Geocentric is a nice simple cartesian system in metres, but the concept of “up” varies with your location so heightmap patches are expensive to apply. Longitude/Lattitude/Altitude is convenient for heightmaps since “up” is represented by “altitude” and 0 is at the surface of the planet. It’s weaknesses are that it is in degrees (instead of metres) and the first 2 coordinates represent curved space making distance and angle calculations very inaccurate when the points are far apart.

Write simple explicit conversion routines for going between coordinate systems

Invent a unique string identifier for each coordinate system. This allows you to have consistent transformation functions between each space. While implicit transformations would be convenient and easy to implement, it is important for code readability to see clearly when data is transformed. For my project I have the 3 systems I need to consistently transform between. Geocentric (geo), Lon/Lat/Alt (LLA) and Local Open GL coordinates (Local). When I came onto the project, there was no consistent and easy way to transform between coordinates.   Some transformations used methods of a class, others used static functions and others were a composite of functions. Worse still, the naming of the methods had no obvious link to the concept that you where trying to transform from one space to another.

I’ve now wrapped all the various transformation methods with simple consistent helper functions.

vector3 PositionGeoToLocal( const vector3 &vectorGeo )
vectorLLA PositionGeoToLLA( const vector3 &vectorGeo )
vector3 PositionLLAToGeo( const vectorLLA &vectorLLA )
vector3 PositionLLAToLocal( const vectorLLA &vectorLLA )
vector3 PositionLocalToGeo( const vector3 &vectorLocal )
vectorLLA PositionLocalToLLA( const vector3 &vectorLocal )

quaternion OrientationGeoToLocal( const quaternion &orientGeo )
quaternion OrientationLocalToGeo( const quaternion &orientLocal )

Use a variable naming convention for your coordinates

Since you may different coordinate systems interleaved through a single block of maths, it can help with code readability to use a naming convention that identifies the coordinate system you are using. My prefered method is a small postfix on the variables, but a prefix would work just as well.

Having different classes for each representation helps the compiler spot any misuse of variables but it doesn’t help the clarity of the code.

Converting Velocities Safely

Velocities and directions are a little trickier to convert since they can depend on what position they are taken from. The simple trick here is to convert the current position, and calculate and convert the current position plus the velocity. Then if you take a difference between those positions, you get the converted velocity.

vectorLLA positionLLA = PositionGeoToLLA( positionGeo )
vectorLLA velocityLLA = PositionGeoToLLA( positionGeo + velocityGeo ) - positionLLA;

It important to realise here that applying the same velocity in LLA space will not give you the same results as applying it in a cartesian coordinate system.

Other interesting uses of curved space

Something I’ve done on a couple of projects now is to use a 2d coordinate system that represents curved space. When coordinates a transformd into their cartesian counterpart for rendering,  space is effectively warped giving you a more interesting world to move in. For instance, a simple old school 2d game can bend around 3d space without complicating the actual movement or physics maths since those are done as simple x,y maths in curved space.

On my current game, Magelore, I use a curved coordinate system to add some noise to what would have been a very square-edged tile system. All game code is just done in a simple x,y grid,  but when transforming from grid space to cartesian space, it all gets warped somewhat.

I figured binary serialization would be a good way to go as its fairly flexible and efficient.  If you haven’t used serializers before, there is a fair bit of information on it floating about. I won’t go deeply into serializing as it is outside the scope of this article. As a quick overview,  serializing a level involves iterating over all the live objects and having them write their variables to a stream.  To reload the level,  you can load the stream, and de-serialize the objects. Often the same code can work in both directions since it merely lists which variables need serializing.  Here is a quick example of serializing a door entity from my game.  Note the & symbol (which I stole from Boost syntax)  means the variable is read or written depending on the mode of the stream, so the same code works as a read as well as a write.

void Door::Serialize( DMSerializer &stream )
{
  parent::Serialize(stream);

  stream & m_isLocked & m_isOpen;
}

So this is all fine and dandy.   The big problem comes after you have published your game and you now want to add new content to a level in the next version update.  In my original plan, I would load a level file on the first visit to a level, but on subsequent visits, I would just load the serialized version.  However, that meant that once a player had seen a level,  it would never look at the level file again. If I added new NPCs or new towns, the player would never see them unless they started a clean profile.

This is a problem I’d never come across in my years of doing console titles.  Generally in updates we had to add new levels or extra profile information,  but never modify a level that already existed in a savegame.  I tried designing around the problem by saying that levels would always reset on entering them,  but that meant that destructible objects would reset each time which was too big an exploit.

After a few days of considering options, I settled on the following solution.  But before you get bored reading a lot of text and code,  here is a video showing the final result of updating a saved level.

Every entity gets a unique integer ID.   This is effectively a HANDLE for your entity.   Other entities can safely reference your entity by keeping a copy of this handle. This is useful for serialization since an entity may need to save a reference another entity. You can’t serialize a pointer, but you can serialize an integer. I was previously using weak pointers to entities, but after implementing unique ID handles, the weak pointers are no longer required and have been stripped out of the codebase.

All UIDs are registered in a table for a quick resolve. I use a simple array of entity pointers,  since I restrict the number of UIDs to 8192 so a 32k table can hold all known UIDs.  If an array is not viable, you could use a map or hashtable, but be aware the lookup time will be much worse.

UID -1 is considered an INVALID HANDLE.  This is important so you can tell if a UID no longer references any entity.
Split your UIDs into 3 ranges.  There 3 distinctive types of entity for this system and they each need to be isolated so you can tell what type of entity it is by looking at the UID.

1. Entities created in code that don’t appear in a level file,  such as the player.  These entities will be statically assigned a UID. For instance, my main character always gets UID 0.  These shall be called Global UIDs
2. Entites that exist in a level file. These will have their UID listed in the level file. The UID can never change and can never be resused in a future update. If an object is deleted from the level,  that UID should never be recycled.  These shall be called Static UIDs.
3. Dynamically created entities.  These do not exist in the level file but are spawned during the cause of gameplay. For instance, a spawner might spawn a giant rat. The spawner itself would be specified in the level file, so it is a type 2 entity.  The rat is spawned dyamically so it is a type 3 entity and will be assigned a dynamic UID from the range that has been reserved.  Code for allocating, freeing and resolving entities is posted below (edit – I have updated this code following responses to this article. Note that ths code is for demonstration purposes. You will want to add range checking assertions.).

u16 DMSceneManager::AllocDynamicUID( DMEntity *pNewEntity )
{
  u16 newUID = m_freeUIDs[ 0 ];
  m_freeUIDS[ 0 ] = m_freeUIDs[ --m_freeUIDCount ];
  m_entityArray[ newUID ] = pNewEntity;
  return newUID;
}

DMSceneManager::FreeDynamicUID( u16 uid )
{
  m_freeUIDs[ ++m_freeUIDCount ] = uid;
  m_entityArray[ uid ] = 0;
}

DMEntity *DMSceneManager::ResolveEntity( u16 uid )
{
  return m_entityArray[ uid ];
}

Saving a level

As we exit a level,  or just exit the app,  we will serialize the current level to disk.  I support multiple profile slots so each level is serialized to the file  ”<ProfileID>_<levelName>.sav”.   All entities of type 2 and 3 get serialized.  Entities of Type 0 get serialized to the players profile file since they are effectively global and not bound to any level and should only be restored on the initial profile load.

void DMSceneManager::Serialize(DMSerializer &stream)
{
  DMEntity *pEntity = m_pEntityList;
  while (pEntity)
  {
    u16 uid = pEntity->UID();
    if (IsUIDStatic(uid) || IsUIDDynamic(uid))
    {
      stream << UID();
      stream.StartBlock();
      stream << GetTemplateName();
      pEntity->Serialize(stream);
      stream.EndBlock();
    }
    pEntity = pEntity->m_pNext;
  }
  stream << INVALID_ENTITY_HANDLE;
}

Loading a level

As we enter a level,  we will throw out all entities of type 2 and 3 from above from the previous level.  They will have been serialized and will be restored if the player returns to the old level.  If we followed this approach correctly,  then before loading a level,  no entities of type 2 ad 3 should exist.   As we load the level file, we will be creating all the type 2 objects and giving them their UID numbers which will still match what they were last time the player was in this level.

Finally we deserialize the stream for this level. The UID is grabbed first and we check if an object exists with that UID.

• If the object exists,  then we serialize over the top.  This means serialize functions must cope with the object already being initialised.
• If the object doesn’t exist,  but is a type 2 UID,  just ignore it.  This means it was a level file object but that object has been removed from the level.
• If the object doesn’t exist, but is a type 3 UID,  then create it and deserialize it.

void DMSceneManager::Deserialize(DMSerializer &stream)
{
  u16 uniqueID;
  stream >> uniqueID;

  while (uniqueID != INVALID_ENTITY_HANDLE)
  {
    stream.StartBlock(DMString());

    DMString templateName;
    stream >> templateName;

    DMEntity *pEntity = ResolveEntity(uniqueID);
    if (pEntity)
    {
      // this entity was created in the level file - just just serialize over the top of its current state
      pEntity->Serialize(stream);
    }
    else if (IsUIDDynamic(uniqueID))
    {
      // dynamic entity - doesn't primed by the TileMap so we need to create it...
      pEntity = (DMEntity *)DMObjectFactory::Get()->CreateObject(templateName);
      pEntity->SetUID(uniqueID);
      pEntity->Serialize(stream);
    }

    stream.EndBlock();   // skips to the end of the block if entity didn't get deserialized

    // get next unique ID
    stream >> uniqueID;
  }
}

What does this system allow you to update?

Doing this allows you to add and remove objects to an already serialized level.  This is essential for adding new content such as quest givers or opening new areas with doors or towns.

You can also update anything in the entities that isn’t serialized. That is, any data you assign via the level file, but will not dynamically change  during gameplay.  This can be scripts or blocks of data defining behaviour, such as wander paths or spawn areas. As an example, I have npc’s that give a short speech when spoken to. The speech is embedded in the level file.  Since this text will not change during gameplay, it is not serialized and any modifications to the level file will not be override by saved games.

Another point I don’t delve into is that a serialized level has a version number that can be checked when de-serializing.  This means if you modify the variables in an entity,  you can still provide a fall back serialization routine that converts the old stream into some sensible values under the new system.  This is also relevant to being able to provide updates to a title,  but is a fairly standard technique so I haven’t gone into it in any detail here.

Final Questions

How do you address the issue of saving your levels?  Are you having to deal with the same issue, but have come up with a difference solution?  Please post about it in the comments below. I’d love to read about how others have tackled this problem.

Making your own engine can be a great learning experience,  and gives you complete control over any performance bottlenecks and the ability to fix any bugs and implement any obscure features your game might need.

I’m currently developing an RPG called Magelore.  I am using my own engine for this, which I initially wrote with Daniel Krenn for Venger,  released under the Wretched Games label.  It has been used in 4 released apps, with Magelore to be the 5th.  The engine is simple and contains the minimum set of features to streamline iOS app development without any bells and whistles.  While Venger was 3D, our subsequent games have been 2D as well as Magelore which is all 2D.

The most important point worth noting, is that if you don’t want to get bogged down writing an engine for months and never getting onto the game part,  it is best not to get too obsessed with optimising the engine.  A typical iOS 2D app is generally fill rate limited,  so spending too much of your precious time optimisations engine components isn’t going to pay off in the short term.   Physics and collisions are the other likely bottleneck.  Unless you’re doing your own physics system,  there is not much you can improve here from the engine side.  You simply need to adjust your design and gameplay code so as not to need so too many expensive operations per frame.

I’ll now go through the minimum elements you need for an engine in order to get a game onto the AppStore.  I’ll go into some of these modules in much greater detail in future blogs.  This will need to be a very brief overview or the blog would be massive.

Cross Platform

It can be a huge boon for development to make your engine cross platform, and really doesn’t cost a lot.  Our engine runs on PC also, which means we can use Visual Studio which as a much superior debugger to XCode. It also means we have access to mouse and keyboard for debugging shortcuts,  easy screen shots and frapping of videos.   The PC version has been set up so it will run in 320×480,  640×960 or 1024×768 modes in both portrait and landscape modes.  This way I can test what the game will look like on any device by changing a commandline parameter.  Since you’re using OpenGL,  to make something cross platform you mainly just need to isolate a few minor modules –  startup, sound, textures, input, and minor discrepancies between OpenGL and OpenGLES.   Our objective C OSX files are restricted to the AppDelegate, the View class, the OpenGLRenderer1 and 2 classes, SoundEngine and the Texture class.

C++ versus Objective C

My engine is completely C++.  The Objective C parts are restricted to the AppDelegate,  the EAGLView class and a small OpenGLRenderer1 and OpenGLRenderer2 class. Most importantly, this means the PC version shares 99% of the code with the iOS version.

File System

On most games,  you can get by with just a ReadFile and a WriteFile method. Most games do not require streaming files, and its generally more efficient to read them in one chunk and process them in memory. I always use a flat filesystem with my games. This means every resource is accessed by just a unique filename with no path. This removes a lot of possibilities for human error when managing your assets since they can be moved anywhere and still work, and can never get mixed up.  I’ve used that technique for console games with 10 gigs of assets, so there’s no scaling issue once you have worked out good file naming standards.

Asynchronous loading is an optional extra that will give you some power to polish UI transition between levels, but its not essential in most games.  You sound system will probably need streaming for music,  but I have used middleware for sound  (namely Apple’s SoundEngine.cpp/h)  which handles its own threading and soundbank streaming.

Data File Reader

You’ll want to data drive as much content as possible,  so you’ll need to pick a data file format and stick to that standard for all your data files. I use XML and have a smaller helper object that just manages construction and destruction of TinyXML documents.  Generally the runtime hit for loading and parsing your data will not be an issue,  but if it is it generally means you need to break up your XMLs into smaller parts so they don’t need to be loaded all at once.  Supporting Auto-Refresh is very useful for development here.  That is,  your class that reads an XML should continually check if the file has changed and reload it.  This is especially good for any system that needs rapid iteration like particle systems or combat balancing,  and saves you having to spend a lot of development time implementing an editor for those data files.

Maths

Good maths support is essential in any game. Vectors, trigonometry wrappers, random number generators, interpolators,  and Min, Max, Clamp templates should be there. In a 3d game you’ll need matrix and quaternion support as well.  Code for any complex routines, like quaternion maths,  are easily found on the net so no need to be a maths genius to put together a good library of maths helpers. Depending on your game design,  you will generally not need to worry about low-level optimisations of your maths library.   When choosing your vector classes, be careful not to fall into the optimising trap. The readability difference between a optimised vector library that efficiently pipelines a vector math unit,  versus a simple operator overloaded type-safe vector library is massive. However the speed improvement you might get for that payout may be insignificant as maths were not your bottleneck.

Containers

You need to provide a good set of stacks, lists, maps, hashtables. I’d recommend just using STL for this but keep an eye on your object file sizes.  A common trick to avoid code bloat is to use containers of pointers,  as generally compilers can share the same code for a  std::vector<classA *>  and  std::vector<classB *>.   The are some of other advantages to using a container of pointers, so its well worth looking into.

Render Framework

Of course, you’ll need to be able to render graphics and some sort of render manager class can help co-ordinate sprite systems, material systems, and models and provide an extendable framework for adding new types of rendering primitives.   I have taken a layered approach to rendering.  The scene is split into a lot of layers and each layer holds a bunch of Render Jobs.   Each layer is given a view object which determines its camera and projection matrix, thus determining if it is a 2D or 3D layer.  You can specify a sorting mode for each layer.  Each render job is added with a priority value.  In a 3D layer, that priority might be the render objects centre Z co-ordinate and you can specify that this layer gets sorted.   Other layers can be left unsorted and so the order you add render jobs becomes the order they render.  This can be useful for UI.  You can also set render targets and capture targets for each layer.   A render target specifies that this layer will render to a specific texture,  instead of the back buffer.  A capture target specifies that a copy of the backbuffer will be taken when the layer is finished and stored in a specific texture.   In practice,  capture targets are quite slow on iOS, so you’re better off sticking with render targets only.   Note that this is really only used for special effects and so you can happily skip this feature for a simple iOS game.

Render Jobs are an extendable callback driven rendering system.  Any engine or game module can implement their own render job.  Examples of render jobs I support are Sprites, Fonts and Primitive Rendering.  The concept is system. You tell the engine that you are creating a rendering job on a specific layer and priority and supply it with a callback function.   The engine returns a pointer to the some raw memory you can use for your job data. Once you have filled as much memory as you require you tell the engine that you are done.  At the appropriate time in the Draw pass it will invoke your callback function, passing it your render job data. You can then access OpenGL directly to implement whatever kind of rendering you need.

Materials and Textures Pipeline

You’ll certainly need textures.  Support for compressed textures (PVR) is also desirable but not essential .  In 2D artwork the compression artifacts can be quite noticable. They are really better suited for 3d object textures. I’ve restricted myself to just PNGs and PVRs so as to simplify the art requirements.  It’s fairly easy to support a range of textures through middleware, but it can be best to just stick to one format that does all your needs and not end up with massive bloats of code and inconsistent resources.   Materials are currently just a simple table of information detailing what texture they use, and what blend modes and render states to activate.

Fonts

You’ll need to be able to render fonts.  I use AngelCode’s excellent BMFont  utility to convert any font into a PNG with a text data file.  Font renderers will need to support all justification modes,  and it is very handy to have a render-in-box method that will word wrap text into a box area. This comes in handy for drawing requesters.  I also like to have support embedded commands like change text colours or jumping to a tab position so you can easily syntax highlight messages without having to split it into font draw calls.

Sprites

Sprites are your moving, animating elements in a 2D game,  though they can be used as billboards in a 3D game also. For sprites I use PSDs as my input source.  Each sprite animation frame comes in as a separate PSD layer and hotspots can also be defined through layers. Since I do all my art creation in photoshop, this is a convenient storage format. It may also be useful to support making a sprite out of lots of single png files – which would allow you to render something animated in

3dsMax and bring it quickly into the game.   I only support PSDs on the PC platform,  which then dumps out a binary sprite file with all sprites quad packed into large texture pages.  This sprite file is then used by the iOS device for its textures – which saves on load times and storage size on the iPhone, with the trade off that I need to run the game on the PC to generate data for the iOS version.

Particle System

At some point you’ll want some explosions or sparkles or some autonomous animating effect. For Magelore,  I use a simple envelope driven sprite manager with movement defined in an XML file. This gives me a very flexible system with minimal coding effort.  If your game is heavily focussed on retro pixel based effects where you need 100s or 1000s of points,  you may want to spend a lot of time here optimising for data cache access, vectorising the updates and reducing the dynamic memory accesses during the draws.  This can bog you down for a long time and needs many iterations of profiling and testing and very low-level knowledge of the memory pipeline for your hardware. However, for a lot of game styles you can get by with just a simple flexible generic system. Fillrate is probably going to be your biggest bottleneck with particle systems.

Primitive Renderer

Being able to dynamically generate and render raw textured uv’d polygons is always useful for prototyping new effects or one off effects.  I prefer to put a wrapper over OpenGL to hide any minor OpenGLES discrepancies. It also bad for someone to touch the raw render states as your material pipeline will be unaware of these changes to state.

Accelerometer Input

Many iOS games will need accelerometer input. This comes to your app via the accelerometer message so you can just provide your appDelegate as the delegate for that message,  and then wrap it up and send it to your C++ input module for everything else to access.  I put an adaptive smoothing over it so the values look smoother when rendered on screen but are still responsive to fast movement.

UI System

My UI System is, in most respects like a standard windowing input system – frames that have area on screen and children.  It deviates in a few areas to be more useful in a multitouch with your fingers environment like iOS devices.   Frames can be oversized and overlap. Touches will go to the frame they are most inside.  Frames can also register for multitouch,  which means while held, all future touches are captured by that frame.  An example use might be a zoom icon. When you touch zoom you can use the second finger to pinch the screen in to control the zoom.   I still expose the raw touch list to the game if they want to do something very freestyle that doesn’t map to the simple concept of buttons.

Serializer

One popular way to save games is via a serializer. Some important features for a serializer are:

• endian agnostic – always serialize bytes at a time so the endian of the platform doesn’t matter. Useful if you ever make a PS3 or X360 version of your game and want data compatibility.

• bit serializing – I like to include a method that lets you serialise an int to specified number of bits.  If you use the serializer for creating network packets, you’ll want to pack them as small as possible.

• version number – important for your future upgrade path so you can add new content to your objects.

• compression – supporting gzip compression is easy and fast.  If your posting large serializations to web servers,  any reduced bandwidth is a win.

• blocks – I support a StartBlock and EndBlock serializer command.  This inserts a size into the stream for you enabling you to skip whole blocks during deserialization if that object is no longer important to your game.

• ascii output – I include an ascii serializer  (write only)  for debugging purposes. When saving my games, I run the process a second time using an ascii serializer so if anything is going wrong I can easily see what data is causing it.

Scripting

Scripting language is essential for data driven games. I use a custom scripting language in my engine. When taking an off-the-shelf language like Lua or GameMonkey,  just be sure you’re not going to run into performance and memory bottlenecks. A lot of existing script languages can eat up a considerable portion of your CPU just doing garbage collection.  My script language was meant purely to automate simple tasks and has no dynamic memory usage. It is meant to be used as high level control I can embed scripts like “open door;  delay 3;  close door” into any system.   Note that some scripting languages are not designed to handle a command like “delay 3″ – one that stalls the script for 3 seconds – unless they are run as co-routines or threads,  which can add unwanted complexity.

Physics

I’ve not used any physics systems in any of our released games and Magelore is no exception.  Venger used a simple collision system for ray collides and custom 3d movement routines.  For a 3d physics system I’d recommend looking at Bullet or Newton Physics.  For a 2d physics system its hard to go past Box2D.  Writing your own requires very strong mathematics skills and I’ve seen many failed attempts by good programmers.

Sound

For iOS sound we’ve used SoundEngine.cpp which comes with Apples sample programs. It provides simple access to playing mono and stereo caf files, and for streaming m4a files.

Video Playback

Often useful. Fortunately this is extremely easy to implement on iOS through Apples MPMoviePlayerController class. You just create one of this and add its view on top of your current view. You later get a callback when the video is complete.

Debugging

It’s important to support a basic suite of Asserts, Errors and Logging. Lately I’ve taken to using HTTP debugging where you can listen on port 80 and deliver game information as HTML pages on HTTP GET requests. This is great for getting debug information from remote machines like someone QA’ing your game.  On my last project, which was very script heavy, I did a full debugger for my script system via the HTTP webpage. That turned out to be a great help to the designers,  and if they really got stuck, I could just browse to their X360 web page and see what was going wrong.

Scene Manager and Entity System

You’ll need some kind of object framework. Something to keep track of all your objects and call Update and Draw methods.  I’m just using a old style fat entity system for simplicity. My scene management is a simple list of entities. I can get away with this because I generally only have objects alive if they are near being visible, and I don’t have too many live objects for it to become a bottleneck.  Venger was a tunnel shooter and so we were able to just create and destroy all objects as you progressed through the tunnel,  meaning all live objects where valid for Update

I try to keep my hierarchies as flat as possible.  If you’re ever finding that your hierarchies are getting deep,  it’s time to switch to a component system.  However, components come with some baggage since you have extra intra-entity communication, dependency and ordering issues to deal with. So there is still some advantages to keeping with a fat entity system,  especially if you already have one working and the alternative would be to scrap it and start all over again ;)

All Done

Phew… Now that I have all that off my chest, I hope to write about more focussed topics in future blogs where I’ll be able to describe in better detail with pictures, code sample and videos how aspects of the game are implemented. Areas I have ear-marked for discussion include how content is generated,  applying updates to serialized levels,  script usage in xml content,  real-time lighting system, and so forth.

For information on the progress of Magelore, including pictures and video,  check out   http://magelore.blogspot.com