Scriptable Asset Loading

In the last few months, I’ve been spending a considerable amount of time fleshing out some tedious but necessary parts of my game. I realized that since I’m a one-man army, I need the ability to very quickly get all of my ideas out and into a playable form without a lot of process and layers of tools. Unfortunately, the only way to achieve a very seamless workflow is by specializing your tools, which means rolling my own level editor and game formats. These things are nice to have anyway, but I believe that the time I invest in these tools will pay off in even the very first game I make with them. I decided I needed a quick and easy way to import models and other game assets, a scripting language (I chose Lua) for data definition and eventually scripted events and possibly game rules, and a level editor that allows rapid building and playtesting of open 3D worlds.

I have a few requirements for my tools, as well as any third party tools that I use. They need to have an nonrestrictive license that allows me to use them without royalties when I attempt to release my game. In addition, the license cannot be parasitic, so that by using the software I will not be forced to release my source code (I’ll keep everything open source that I can). I am happy to document my process on this blog, and I include all the difficult/valuable code gems to help others in their endeavors; I just don’t yet want the responsibility of maintaining a reliable and active open source project. The tools will need to be cross platform (Windows, Mac OS X and Linux), since I’m a big fan of cross platform code, and I would like to prove to myself that it’s not prohibitive to produce quality cross-platform software. I’m also not a fanboy of any particular platform and have a love-hate relationship with basically all technology.

Asset Importer

For starters, I needed an asset importer. I have very basic requirements for my geometry, likely without even the need for texture coordinates, let alone more complicated materials. However, Because I’m using Bullet physics, and I’ve elected to use Blender as my modeling software, I would also like a format that stores Blender physics data. As far as I know, the only real standard format that stores this information is COLLADA, which has become a rather common intermediary format for development, however is not suitable for a production game due to the size of the files, the speed of the importers and the large amounts of unnecessary data. I’ve also found that a simple model loader that supports a number of formats including COLLADA is more rare than it should be. In my search, I discovered a tool called the Open Asset Import Library, or Assimp for short. This tool has been working out great so far. I created an Asset type in my engine with necessary hooks to add an asset just like any other geometric primitive I have defined by specifying a filename and a position on the screen (with options for scale, etc). The code to load an asset of any given type along with the code to parse the geometry into a list of vertices and faces for my own format definition is very straightforward:

m_pAiScene = m_pAiImporter->ReadFile(
        filename,
        aiProcess_CalcTangentSpace      |
        aiProcess_Triangulate           |
        aiProcess_JoinIdenticalVertices |
        aiProcess_SortByPType          
        //aiProcess_MakeLeftHanded            // Left-handed coords required by DX
        );

    if (m_pAiScene)
    {
        m_pVertices = reinterpret_cast<Vertex*>(malloc(m_pAiScene->mMeshes[0]->mNumVertices * sizeof(Vertex)));
        m_pIndices = reinterpret_cast<short*>(malloc(m_pAiScene->mMeshes[0]->mNumFaces * m_pAiScene->mMeshes[0]->mFaces[0].mNumIndices * sizeof(short)));

        for (UINT i = 0; i < m_pAiScene->mMeshes[0]->mNumVertices; i++)
        {
            Vertex vertex = {
                {
                    m_pAiScene->mMeshes[0]->mVertices[i].x,
                    m_pAiScene->mMeshes[0]->mVertices[i].y,
                    m_pAiScene->mMeshes[0]->mVertices[i].z
                },
                WHITE
            };
            m_pVertices[i] = vertex;
        }

        for (UINT i = 0; i < m_pAiScene->mMeshes[0]->mNumFaces; i++)
        {
            for (UINT j = 0; j < m_pAiScene->mMeshes[0]->mFaces[0].mNumIndices; j++)
            {
                short index = m_pAiScene->mMeshes[0]->mFaces[i].mIndices[j];
                m_pIndices[i*m_pAiScene->mMeshes[0]->mFaces[0].mNumIndices + j] = index;
            }
        }
    }

Lua Scripting

With the ability to import any model I wanted, I wanted to clean up my main.cpp which had simply become a file for defining the objects in my scene. I decided I wanted to have scripting support that at the very least supported data definition. Essentially, my game engine would load whatever assets and game objects the script requested, and proceeded to simulate “the game” by following a set of rules that are currently hardcoded into the engine. After a bit of research I decided upon Lua as my scripting language. Lua is fast, ‘easily’ embeddable into C++, and has higher level language conveniences that allow for quicker scripted events. Thankfully, since I’m a programmer, I don’t need something that’s strictly “human readable”, I just want something that is interpreted to prevent recompiles when changing objects in a scene, and reduces the amount of typing I have to do. I’ve also read about a great deal of success that other indie and major game studios have had with Lua as a scripting language, and since I’ve used it successfully in other projects I felt comfortable with the choice.

Meshing Lua with C++ however, was not as easy as I had hoped. I went through a number of different wrappers that claimed to make Lua binding easy. Simple Lua Binder aka SLB was my first choice, until I realized the limited support for wrapping C++ classes. And although it made calling from Lua into C++ very simple, calling from C++ into Lua was not made any easier by the tool. There was also the issue of support; it seemed like the project, while currently actively maintained, only has involvement from a single contributor which leaves it a rather fragile platform to build upon. After being scared away from SLB, I investigated Luabind, ToLua++, and SWIG. Luabind seems to be a very commonly used binding library, but it also seemed overkill for the basic data definition I wanted to do at the time. ToLua++ seemed like a good option, however I couldn’t seem to get it compiled and linking well on my windows machine, so without wasting too much time, I decided to drop it and try another solution. SWIG is interesting because it utilizes a pre-build step that simply outputs the native C/C++ code for binding to any particular language. The only changes that need to be made to generate bindings for various languages are configuration file changes. If I wanted to switch from Lua to Python, or even Ruby later I could do so with minimal effort. The downside is the overhead of an additional pre-build step. I wrote an API header that defines all C++ functions I wish to be visible to Lua, which is the only dependency the SWIG manifest file has. The other downside however is that SWIG still seemed to have poor support for calling from C++ into Lua (as it appears is typical among the binding solutions). Currently I have SWIG implemented and can load a camera, player, and all content into a scene using a Lua script.

The manifest file describes the binding code for SWIG to generate:

// lua_bindings.cpp is autogenerated from this file

%module game
%{
#include "API.h"
%}
%include "API.h"

The API.h file declares the functions that will be made accessible to Lua:

void SetPlayer(float x, float y, float z, float scale, float r, float g, float b, float a);
void SetCamera(float rotX, float rotY, float radius);
void AddEnemy(int numParticles, float x, float y, float z, float scale, float r, float g, float b, float a);
void AddAsset(const char* filename, float x, float y, float z);

This Lua script is used to define the scene:

game.SetPlayer(0, 0, -20, 3, 1, 1, 1, 1)

game.SetCamera(0, 3.14159265358979323/2, 100)

game.AddAsset("resources/ground.obj", 0, -36, 0)

game.AddEnemy(64, -50.0, 30.0, 12.0, 1.8, 0.4588235, 0.9058823, 0.1882352, 1);
game.AddEnemy(64, 50.0, 30.0, 12.0, 1.8, 0.2, 0.6588235, 0.8980392, 1);
game.AddEnemy(64, -50.0, -30.0, 12.0, 1.8, 0.2, 0.6588235, 0.8980392, 1);
game.AddEnemy(64, 50.0, -30.0, 12.0, 1.8, 0.4588235, 0.9058823, 0.1882352, 1);
game.AddEnemy(729, 0, 0, 12, 1.8, 0.9764705, 0.1490196, 0.4470588, 1)

Finally, the Lua script is invoked in C++ by:

    lua_State* L;
    L=lua_open();
    //luaopen_base(L);  // load basic libs (eg. print)
    luaL_openlibs(L);
    luaopen_game(L);    // load the wrappered module

    if (luaL_loadfile(L,".\\scripts\\scene.lua")==0) // load and run the file
    {
        printf("loaded .\\scripts\\scene.lua");
        lua_pcall(L,0,0,0);
    }
    else
    {
        printf("unable to load .\\scripts\\scene.lua");
    }

    lua_close(L);

However I’ve recently been reading Game Coding Complete, which I very highly recommend. McShaffry suggests LuaPlus, an augmentation of core Lua 5.1 which apparently has much better native binding support. While I haven’t had the time so far, I plan on evaluating LuaPlus, which could hopefully resolve some of the design issues I feel I may run into with other binding systems that seem to one-sidedly support Lua to C++ calls but not vice versa.