3 step parsed language

The work on the parser is going forward, and actually is working quite well. The parser will be a 3 step parsed language:

• Tokenize the source
• Parse statements
• Execute the AST

The first step is to be able to recognize the different parts of the source string into different pieces. For example "1+2" would be 3 tokens: [Number:"1", Operator:"+", Number:"2"]

Tokens are responsible of just having a content and knowing the type of data but don't yet know if it's valid to have a given type after another.

Parsing statement uses the tokens produces by the first step, and basically cascade the parsing from "expression", "and operator", "or operator" down to the "base statement". The order of the cascade actually defines the order of precedence of the different pieces, for example multiplications must be done before an addition: 3*2+4 must be done as (3*2)+4 and not 3*(2+4) as the result is not the same. In this step the parser can detect syntax errors, for example missing a parenthesis. At the end of this process we will have a AST (https://en.wikipedia.org/wiki/Abstract_syntax_tree) which can then be run.

Executing the AST is after that quite simple, each node knows how to execute its own operation and nothing more. Also if the AST is well built the execution of it should be fast. Ideally AST optimizations should be a step before the execution, and an AST could ideally be transformed into a binary code directly run by the CPU (that would produce a JIT if done on run-time).

Now you may wonder but why do I need to write all that, and yes it's not a small piece of software? Yes there is available parsers like the Javascript function "eval" or use 3rd party libs which would implement a scripting language for you. Well the answer is pretty simple: eval would be easy to use and cheap for the developer (me). However it would have a major issue: security, any valid code would be able to run, for example even nasty operations. 3rd party libs may actually much bigger than what we would ideally want and may be harder to integrate or debug in case of issues.

Stay tuned to see my parser first in action!

Further thinking about stats and skills

We are still thinking about how we want to handle the customization. On one side the option to give full customization is tempting, as it would really allows game owner/designers to change the rules as they want. On the other side it will for sure increase the difficulty for new comers to approach the system as they may need to write some code to change the rules.

This can be of course mitigated by the fact we offer some default content, documentation and tutorials but it may scare some people.

Does it then make sense to take this road or would it be better to offer mainly a form which need to be tweaked?

Let's take a possible example: our "Life" or "HP" stat:

// Example of the script behind the HP stat.
// Function returning the currently maximum allowed value.
function MaxValue()
{
    return API.GetStat('Level')*20;
}
// Function returning the currently minimum allowed value.
function MinValue()
{
    return 0;
}
// Function run every time the value is modified
function ValueChanged(newValue)
{
    if(newValue < 1)
    {
        API.Teleport(0,0,0,0);
        API.SetStat('HP',1);
    }
    if(newValue > MaxValue())
    {
        API.SetStat('HP',MaxValue());
    }
    UI.UpdateStatBar();
}

In this example the API and UI are functions offered by the engine. While the functions defined in the stat code will be called from the engine at some points. As you see this could open the door to multiple features like a "berserk" skill which could trigger once you reach 10% of life.

To implement this, I will first have to develop the full parser / runtime of this language, however if I do it now, it will be possible to use it later on for plug-ins development which would mean we would have a single language used in multiple places.

Also having such language could be used on the objects themselves for example a potion could restore a stat or kill you.

Again it's a question of balancing flexibility to user friendliness (easy to use) and here I'm a bit unsure which road we should take.

Stats points, skills, make the engine flexible

To make the engine an engine and not just a fixed  game which would offer little flexibility the engine should let you define the stats points and the skills.

As we want to let the game owners create the games they want, being vampires, gangs or medieval, the engine must let you create whatever stats points you want as well as define any skill you want.

Stats would be like your current value on something, being the money, your life, or experience. Stats could recover themselves have triggers when reach some value (you die if you reach 0 life for example) or have a maximum level (energy for example). All that's is just matter of letting you define a name, and some fields. But what if a stat maximum depends of a "level" or depends on what you wear? Here the things starts to be a bit more complex and require some sort of logic built on the stats which must be defined by the game owner.

Of course we must offer standard stats with a standard logic such that new comers don't spend 10 years learning our tool but can start with something and later on tweak it.

To build all that, it will require also tools to view / modify those information, and store / retrieve them from the database. The rules must be defined by the owner and stored per game, and for each player a set of current stats must be stored as well.

All this require quiet some work which will not directly be visible, but it's a must to make the engine an engine.

Initially I plan to offer really few default stats, and we may increase the default offered later on:

• HP / Life
• Money
• Experience
• Level

As skill, I want to offer those one:

• Attack
• Defense

As you see it's really really limited, but if we manage to create a good system, adding further one in the default setup will be a piece of cake.

Hopefully we will not already need a full scripting language to support those stats / skills but it may end up nearly the same, with complex formula and actions to trigger when the values changes.

Coordinate transformations

Any graphical game will at some point transform coordinate from one kind to another. For example, for a grid map, you will need to transform the map to screen coordinate, which at first is really easy: screenX=mapX*tileWidth and the same for the Y coordinate.

On the other side, what if you want to center the screen around your player? Already there there is a bit more transformations, with some offsetX and Y for the top left corner for example. What if your maps are split in different areas? Like an area is a 100x100 like in my case, as soon as you goes out of this area you need to change the area index and restart the X,Y coordinate on the map.

You slowly see how I'm heading? The more features the more complexity for your coordinate system. And guess what? What you do in one direction you will most likely need to have in the other, what if I click on the screen and need to know on which cell of my map I'm? And here you are with the reverse of the previous calculation.

Therefore, it's smarter and certainly safer to have those transformation stored in two functions, and then always call those function. That will allows to debug only once and then be assured it will always work (or at least so it should be).

Example of my screen to map coordinate transformation:

public ScreenToMap(x: number, y: number): RenderScreenCoordinate
{
    var pos = $("#gameCanvas").position();
    var tileWidth = game.World.tileSetDefinition.background.width;
    var tileHeight = game.World.tileSetDefinition.background.height;
    var orx = Math.abs(this.offsetX) % tileWidth * (this.offsetX < 0 ? -1 : 1);
    var ory = Math.abs(this.offsetY) % tileHeight * (this.offsetY < 0 ? -1 : 1);
    var x = (x - pos.left) + this.offsetX;
    var y = (y - pos.top) + this.offsetY;
    var ox = x % tileWidth;
    var oy = y % tileHeight;
    x = Math.floor(x / tileWidth);
    y = Math.floor(y / tileHeight);
    var cx = this.areaX + Math.floor(x / this.world.areaWidth);
    var cy = this.areaY + Math.floor(y / this.world.areaHeight);
    var tx = x;
    var ty = y;
    if (tx < 0)
        tx = (this.world.areaWidth - 1) - (Math.abs(tx + 1) % this.world.areaWidth);
    else        tx %= this.world.areaWidth;
    if (ty < 0)
        ty = (this.world.areaHeight - 1) - (Math.abs(ty + 1) % this.world.areaHeight);
    else        ty %= this.world.areaHeight
    var rx = tx + (cx - this.areaX) * (this.world.areaWidth - ((cx - this.areaX) < 0 ? 1 : 0));
    var ry = ty + (cy - this.areaY) * (this.world.areaHeight - ((cy - this.areaY) < 0 ? 1 : 0));
    return { TileX: tx, TileY: ty, AreaX: cx, AreaY: cy, RelativeX: rx, RelativeY: ry, OffsetX: ox, OffsetY: oy };
}

As you see, not really a 2 line function. Yes some comments would also help to read what's going on, but that's more for an example than anything else.

Path solving

Path solving in game programming is something you will mostly use at some point. This kind of algorithm let you find ideally the shortest / quickest way between 2 points, either by using connections between points or cells on a grid map.

The most well known and the best one to use in most case is called A*.

A* is nothing else than a code which will try all possible routes and use the shortest one when it find it. At start it will goes in all the possible directions, add those as new starting point, and repeat. As optimizations you could sort the list of path to try by using the nearest one to the goal first. Of course don't test twice the same node or cell of the map.

Path solving may have some drawbacks: it may take quite some time to solve, so you may want to avoid testing it every single time. Second, if you work at a pixel level it may end up doing WAY too many checks, maybe use a grid on top to make things faster could help you. If blocking elements move you may need to rework you path, to avoid having to do it all the time, you may do it only when you get blocked while traveling the path.

As possible example of a running A* algorithm:
http://bgrins.github.io/javascript-astar/demo/

The same kind of algorithm will work in mazes or with weight on the cost of travel (for example swimming could be slower than walking).

On my own I have a couple of more issues to solve, like for example as my maps are split in "areas" crossing an area change the way you need to handle the path.

Another issue is that maybe I don't want that you can solve a maze simply by clicking the goal, as it would spoil the game. To solve this, I may simply introduce a limit on the number of steps my path solver will try to go through.

Having a good / smart path solver will actually increase your game experience a lot therefore don't spend too little time on it.

Async callback nightmare

One of the main complain I can have with node.js is its way to handle the "asynchronous" calls. Basically some calls take time to run, for example querying a database or connecting to a remote host. Instead of blocking the single thread on which your node.js code runs, those function will call you back once the operation is completed.

At a first thought you may think: great, I don't have anymore blocking calls and at the same time I don't need to deal with multi-threading and possible locks / semaphores.

Indeed this model where every part of your code runs within one thread and you are called when there is something for you is great to solve multi-threading issues. It is also great that you don't have to deal with shared variables or hoping some code is not interrupted in some nasty areas.

Yet, many tasks do require a cascade of events like:

1. Connect to a database
2. Execute a select
3. For each elements of the result update a value
4. Close the connection & free up
5. Return the values

As you see, there is no way you can run on parallel these tasks, and you really need the result of the previous step to do the next one.

In node.js such code could be written like that (it's more pseudo-code than a real API):

db.connect(function (err,conn)
{
   conn.executeQuery("select * from users",function (err2, results)
   {
       for(var i=0;i < results.length;i++)
       {
         conn.executeQuery("update users set gold=gold+10 where id="+results[i].id,function(err3,results2)
        {
        });
     });
   });
});

Ooo wait! There is a bug! You can't call executeQuery to update within the loop, as the queries will be sent all in parallel and may actually be an issue if you have a limit of the number of queries to run at the same time. Would be better to run then one after the other. Also this is by far not readable if you end up in the 10th callback function.

So how can we solve that?

There is some work around found on the net, for example: https://github.com/yortus/asyncawait

I didn't tested them yet, but it should solve exactly this kind of figure where you need to do the operations in sequence (and believe me it's more frequent than to run them in parallel). This library don't actually block node.js after your await call, instead it will call you back transparently once the function you await completed.

As said that needs all to be tested to further understand how that works and if it works well, but hopefully I can clean up the mess created by an actually poorly thought framework. Why am I so aggressive against node.js? Because those problems should be solved at a language level and not via some 3rd party library. And if so many developers have the same issues as me it means that should be actually be a problem solved at the root.

Bugs... stupid bugs...

When you start developing you fight mainly with the language and the framework. While your knowledge grow you still fight but usually you fight for an algorithm.... or stupid bugs which resists you.

Today I fought about 3 hours to find a single little number which was wrong:

area.actors.splice(i, 0);

Instead of

area.actors.splice(i, 1);

Result? The first code do... nothing, while the second actually removes an item from an array.

You could think that his kind of bugs can be solved in no time, sadly the bigger the code base the harder it is to find such bugs. I had to isolate the area which produced this error, try to make sure it wasn't somewhere else, and at the same time I discovered a few other things which didn't really made sense either. As said... 3 hours for this.

Anyhow now I have rats walking around my world:

The movement of the rat is mainly random, it doesn't head yet toward the player. However I implemented collisions with background tiles, and for example water is non walk-able.

I will have to implement collisions with objects as well, yet for this part I need to think how I want it. Will it be a circular area round the ground position? Or could I choose the collision shape? All open questions.

I will need also to create monster spawner, which will let the game owner place monster where he/she want and not simply randomly scattered around the map.