Monday, June 22, 2009
Time
I can say with some certainty that the approach I've taken of making the uber engine from start to finish is probably not the right one. I'm going on 9 months of this current engine iteration and although I think it's got good points it's not functional. Physics breaks and I'm redoing the camera system for the millionth time sigh...
Probably it's smarter to work with an existing engine and build in the points that you want to focus on. I think most people focus on gameplay. I suspect I will never get that far. Maybe one day.
If anyone was following this blog because they wanted to learn about engine development they should probably be discouraged reading this. Good thing nobody follows it.
Tuesday, May 19, 2009
Ping
This is a link to my current engine development. I think it's pretty cool but it obviously has a long way to go. Hopefully people can start to see how a game can be made with it.
http://www.wuala.com/GarethCox/hulkengine/hulkengine%20May%2019%202009.avi
Cheers
Monday, January 12, 2009
Engine Design
So how do I keep my engine simple? Well the basic answer is to write lots of little libraries that do one thing. Then take those little libraries and put them under unit test so that I have some moderate assurance of their functionality. I'll admit I use unit tests almost as documentation on how to use my libraries more so than for verification. This is mostly because I'm new to the unit test world and because I'm lazy. Whenever I discover a bug or think there might be a bug I usually write a unit test to look for it. Writing unit tests has changed the way I code and I believe mostly for the better. Point is you should use unit testing.
The libraries themselves are as self contained as possible. Some like the math and data libary I use in other libs but generally I prefer to duplicate code rather than add dependancies. That's right. I'll say it again. I prefer to duplicate code rather than add dependancy. This is a recent fad with me but I think it makes sense. I want each of my libraries to be self contained packets of functionality. If library A depends on B and B on C and so on then any time I write an application I have to link to every single thing. I suppose this isn't the end of the world but it's complicated and over time increasing dependancy between the two libs will make them uttery inseperable. Alternatively you can interface everything and have complexity and waste where none is needed. I think the phobia against code duplication is well founded but my desire for libs with zero or very limited dependancy is stronger.
Wrap each library in a namespace. This makes it very simple to know where code is coming from. Along with header paths and namespaces code becomes significantly simpler and easier to trace. Unfortunately namespaces tend to temp you to name things the same across projects. So I might have graphics::allocator interface and scene::allocator interface. Unfortunately this doesn't play nice with visual assist so despite the fact that with the namespace it's reduntant I would recomend still prefixing your classes. I don't but then I pay the price when visual assist fails. It's annoying and often I wish I had bit the bullet and prefixed classes. On a quick note get visual assist if you use visual studio. It's a time saver.
Consider what the API is for your library. You don't want to expose any more than is absolutely necessary. This is a great way to simplify your code. You can have some fantastical scene graph lib but if you create a clean API to use that library then you've really accomplished something. Try to export handles to objects rather than objects themselves when possible. So if you have a graphics library that loads textures don't expose the texture pointer external to the library. Expose a handle for tracking that texture and manage the texture internal to the lib. That said do not not use singletons or globals in your library. This will be a non-starter on many projects so make sure that you allow users to create and maintain your manager structures.
Ok well I'll continue this post later. This is tiring.
Gareth
Tuesday, December 30, 2008
The Project
I don't have coder OCD at work but in my personal projects I'm anal to the extreme. Admittedly this is a problem that kills productivity but it does lead to what I consider to be nice code/projects. I'll address my code OCD in a future blog. For now I'll visit what I consider to be good rules for project settings. I suspect that a real build engineer will laugh at some of what I say but I think most of this reasonable.
The first thing to know is what you are trying to accomplish. My goal in regards to my engine is always simplicity. I want it to be as easy to work with and understand as possible. I want to reduce the number of project setting. This makes it easier to add new projects or to copy to a new machine etc. I want to be able to support multiple configurations win32/x64/xbox easily. This way if my target platform changes I can switch with it easily.
I've found there are a couple of categories of components that make up my project. I believe each to be essential but you may find you only have a few. That said here is how my project is broken up.
- Libs
- 3rdParty
- Apps
- UnitTests
- Plugins
- Wrappers
- Resources
3rdParty directory is for all external libraries that are going to in general be unique to the game engine. So boost and directx would not go in here but some version of lua or antlr might. Really I think everything should go in here but I basically just put things in here that don't have installers. The reason to put external libs here is if you want to backup or do source control on your stuff it's easy to avoid grabbing this directory. It can also have benefits for include paths and linking similar to the libs directory. Not always though because there's no controlling how third party libs are layed out. Do not manually arrange third party libs. This should be obvious but basically once you change it you have to maintain your change.
Apps is the directory for all the applications you make using your engine. If you make one game or 100 test apps they all go here. The basic idea is you can setup most of the project settings once and then reuse when making a new app. Sometimes for one reason or another you may want to pull your app off into it's own solution. In that case put the .sln file for your new app at the same level as the .sln for the main project. Again this is so project settings stay the same.
UnitTest is the directory for unittesting individual libraries. I highly recommend unittesting libraries but that is another topic. Basically the advantage here is just to group all these together and similar to the apps directory to setup identical sets of project settings.
Plugins directory is just another grouping for items dependant on the libs directory. I try to keep these things seperate because I think it looks nice.
Wrappers is the directory for libs that have been wrapped for another language. If you want to expose a library for use in C# or python then you would put the code in here.
Resources is the directory for sharing common art assets across multiple projects. The idea is it's easy to setup common paths to art resources for different projects. In practice this is likely to change.
A long time ago I learned to setup project for use with make. Then I learned all the good stuff that is visual studio. Now I'm in the process of heading the other direction. I'm going to continue with visual studio for now but I tend to think I might use something different in the future.
I really had a lot more to say about project setup but I'm tired of writing about it. Which in my mind means 99% of people probably didn't read this far. If you take that number out of the 0 people that visit this blog then you see why I'm stoppoing.
Gareth
Saturday, December 27, 2008
Data Loading
In a previous version of the Hulkengine I created a system that would at runtime parse header files and generate essentially reflectance information. I then used this system to auto-magically load xml into a hierarchy of what were essentially data only classes. For the most part this worked well but it meant I had to include header files as part of the games assets bleck* and I had to either make a more robust parser or be careful of what I put in my headers. A better solution along these lines would be to generate C++ code from the headers and compile in special code for loading. This would load more optimally and remove the need for shipping with the headers.
I'm a big fan of parsers and code generation but I'm not sure of all the idiosyncrasies of this approach. I woul need to write an Antlr grammar. I'd also need to write a tool for code generation. In addition I'd need to create a build rule in visual studio and somehow get it to auto-add my auto-generated parsing code when necessary. I suppose I could add by hand but that doesn't have much sexiness does it? I actually think this is probably the best approach but well.... It seems kind-of a lot of work. Remember I'm doing this on my own time and dime so if I don't want to work on it I'm not going to. Well I suppose I could look online about existing code generators for XML schema files but mehh.
Instead I've decided for a much simpler and clunkier approach. Basically I'm going to force all my data classes to implement a serialization function. I'm a big fan of sexy coding and I realize this is not it. Gimme a sec though let me slap some paint on this jalopy.
Actually let me first point out all the horrible parts. All my data classes have to have a common base class. This means data files describing a db_vec4 in my math library will have to inherit from my data::base class in my data library. Wait it gets worse. This means my data only classes are all going to have a virtual table and procedures. I'll stop here and say I agonized over this all day but I couldn't come up with a better way. This is not to say a better way doesn't exist just I don't know it.
So here's what I came up with. First(well not really but for the sake of clarity) I created a data library. This has three core classes.
- base:
- factory:
- archive:
All of my data objects must inherit from data::base. I really am very proud of this naming convention. Basically the derived class must implement a getid() function for save and serialize() function for save/load. As a general rule although not enforced by the compiler I give each one a const string for their name and a static function to register with the factory for creation.
The factory exists to construct data-classes on load. When an array or a pointer is reach at load time the factory looks up the relevant id and returns the correct creation function. It's basically a map. Actually it is a map nough said.
Archives are my equivalent to a stream class. from the syntax it looks like you are adding entries to the archives when in reality it's more of a tool for navigating my save/load files. Rather than try to fully explain in English let me give an example and we'll go from there.
class db_vector4 : public data::base
{
public:
DECLARE_ARCHIVE(db_vector4);
float x;
float y;
float z;
float w;
};
DEFINE_ARCHIVE(db_vector4);
void db_vector4::serialize(data::archive& s)
{
s.begin("db_vector4")
.attribute("x", x)
.attribute("y", y)
.attribute("z", z)
.attribute("w", w)
.end();
}
Here's an example of a data class and its serialization function. Notice there are two macros there. One declares a few functions and the other defines them. Normally I'm anti-macro but I find it helpful in this case. The only thing interesting hidden by the macros is that I pass an allocator to the static creation function.
So there are a few benefits to this approach that I like. For one save/load is all in the same function. Another is that through the use of tabs I can make the serialization function look almost like the xml file it's going to save. Not really important but I find it clever. You don't actually need to use the classes with the archive. You can if you want to just write the values however you want. Also the the output format is an implementation of the archive so you could easily save csv or xml or binary if you wanted.
So let me give a few more examples.
void db_entity::serialize(data::archive& s)
{
s.begin("db_entity")
.attribute("name", name)
.array("components", m_components)
.end();
}
Here I'm save/loading an entity data class. basically an entity is a name along with a whole bunch of components. Notice the array function. This will on load iterate over children and factory construct the appropriate object for serialization.
void saveload(data::archive& b, options& in_options)
{
b.begin("profile")
.attribute("exportmesh", in_options.m_exportmesh)
.attribute("exportentities", in_options.m_exportentities)
.attribute("exportstage", in_options.m_exportstage)
.attribute("exportcameras", in_options.m_exportcameras)
.attribute("exportmodels", in_options.m_exportmodels)
.attribute("exportlights", in_options.m_exportlights)
.attribute("exportallmodels", in_options.m_exportallmodels)
.begin("mesh")
.attribute("channels", in_options.m_mesh.channels)
.attribute("influences", in_options.m_mesh.influences)
.attribute("exportpositions", in_options.m_mesh.bexportpositions)
.attribute("exportcolors", in_options.m_mesh.bexportcolors)
.attribute("exportnormals", in_options.m_mesh.bexportnormals)
.attribute("exportbinormals", in_options.m_mesh.bexportbinormals)
.attribute("exporttangents", in_options.m_mesh.bexporttangents)
.attribute("exporttexcoords", in_options.m_mesh.bexporttexcoords)
.attribute("exportboneweights", in_options.m_mesh.bexportboneweights)
.end()
.end();
}
This is the profile for my exporter notice how options is just a bunch of data it doesn't implement data::base.
Anyway the archive is the only remotely complicated class. Basically I create a different one for load and save and a different one for whatever format I'm serializing to. So for example I have and xmlloader and xmlsaver implemtation of archive. The loader has a reference to a factory.
Whenever I figure out how to post files up here I'll put up the source and then you can really start tearing it apart.
Well I've made my bed and now I will have to sleep in it. I suspect for the Hulkengine version 4 I'll revisit with some love for the auto-generation version but for now hacky serialization is what I'm going to stick with.
Loading data is fun;)
Friday, December 26, 2008
The Exporter

Now the first time I created a Max exporter I used the fast and loose approach. I thought who cares about the exporter it's the game engine that I really care about. Well I was wrong and I ran into big headaches. My first shortcut was to use classes from my game engine. This leads down the rabbit hole of all the problems with early version of my game engine but I'll just say I ended up having to link my exporter to directX, deal with auto-registration errors(a new peeve), and fight other non-exporter related bugs. All that said I did get a working exporter before. It just wasn't easy to use, wasn't robust, and took forever to add or remove features.
- meshes
- models
- profiles
- scenes
The Content Pipeline
- My artist works in Max
Max is where all my game content gets linked into the game. Max is my game editor. Now I'm not the first person or even game studio to try this. There are several drawbacks but the biggest being cost doesn't really apply to me. Basically because I'm planning on making small games even if I had a small team that would only be a few Max licenses. For big studios it's probably cheaper to create a game editor then have everyone use Max.
There are other problems beyond cost for using Max. The api is clunky and it's hard to create gui's for. You can use Maxscript for lots of things and I do but I'm not Maxscript guru and that means another language to support. I like the idea of using the right language for the problem but only when that language interests me and Maxscript definitely does not.
So how will this all work? What will I use max for exactly? Well obviously art assets. Artist will be able to create models/animations in Max click export and then run the game and view their newly created content. More interesting is the ability to attach game content in
max. So once an artist has laid out the art for a level a designer will load that level up and begin to attach game content.Game content will be attached using Maxscript widgets. Basically the designer will run the Hulkengine Maxscript and see a list of game specific components that he can attach to a Max Node. Then the designer can click export and all the art with game components will export and then he can see/play the game with his content.
The same is true for programmer or sound designer content. All content will go through Max and then into the game.Here's hoping this works well. Note I had a terrible time placing these images so that's my first blogger negative.