Input Handling with Entity-Component-System Architecture

The Problem

The Entity-Component-System architecture forces us to process the game data in a somehow functional fashion: The Entity Database is processed sequentially by each System (ignore the possibility of parallelism for some moment), each makes some modifications to the Database by creating/removing/modifying the Entities and Components. The output from a previous System becomes the input of the next one, basically compositing the update functions.

In the Systems, we define the logic of how an Entity transforms its states into the next frame. We can define this process as a pure function:

constexpr auto SystemA::update(EntityDatabase db) -> EntityDatabase;

If we want another system to continue processing on the Database, just pass it on:

constexpr auto SystemA::update(EntityDatabase db) -> EntityDatabase;
constexpr auto SystemB::update(EntityDatabase db) -> EntityDatabase;

constexpr auto updateEntities(EntityDatabase db)
{
    SystemA sys_a;
    SystemB sys_b;
    
    return sys_b.update(sys_a.update(db));
}

constexpr auto updateEntities2(EntityDatabase db)
{
    SystemA sys_a;
    SystemB sys_b;

    db = sys_a.update(db);
  	db = sys_b.update(db);
    
    return db;
}

So far so good. However, we are missing an important detail: How does the player interact with the world? In other word: How does the player input affect the Entity Database?

We hope that the Systems can remain stateless, that is they operate only based on the data seen, without keeping internal booking or holding any resources like textures and buffers. So that we can insert and purge Systems at any given time without crashing the game. This is an important characteristic for implementing the efficient data serialization of the game state.

Since the update functions are also supposed to be pure, we can also expect that given the initial state of the Database and the sequence of input, which is the sequence of the elapsed time of each frame (not shown in the code above), the frames can be perfectly reconstructed. If this could be done, it would definitely be a great tool for debugging the game logic, potentially also the rendering problems. The bugs around them are notoriously difficult to reproduce.

Let’s revise the update function:

constexpr auto SystemA::update(
    EntityDatabase db, float dt) -> EntityDatabase;
constexpr auto SystemB::update(
    EntityDatabase db, float dt) -> EntityDatabase;

constexpr auto updateEntitiesWithTime(EntityDatabase db, float dt)
{
    SystemA sys_a;
    SystemB sys_b;
    
    return sys_b.update(sys_a.update(db, dt), dt);
}

constexpr auto replay(
    EntityDatabase init_db, std::vector<float> elapsed_time)
{
    SystemA sys_a;
    SystemB sys_b;

    for(auto dt : elapsed_time)
    {
        init_db = sys_b.update(sys_a.update(db, dt), dt);
    }

    return init_db;
}

Dealing with I/O in the Systems apparently breaks this dream because the update functions would not be pure anymore. The affected System would now depend on the sequence of input in addition to the timing sequence. But is there really any difference between the timing sequence and the input sequence? Well, actually, not so much. They are both information feed from the outside that has nothing related to the data inside the Database. But the timing sequence is so important to the update of the Entities for things like animations and physics, we simply cannot leave it out when a reconstruction of the game state is needed.

In fact, we could do exactly the same thing for the input sequence: record all the input events in a list and feed it to the Systems during the replay. The problem is: is doing so a good design?

Let’s consider the use case for Systems and the input system. For Systems, we set them up so that they automatically manipulates the Entities according to some rules we set. After the Entities are put into the game world, we just let the Systems deal with them. These tasks can include updating animation, simulating rigidbody motions, collision detection, calculating AI route and update strategy parameters, moving around the NPCs, compiling the rendering command list (wait, what?), and so on.

Yes, they are stuff we do in the update(dt) function in OOP.

On the other hand, when user input is received, we want some explicit control over some Entities. For example, if the player presses W, the game commands the game character to load a movement vector, use the proper animation, set up the sound effect that sounds right, etc. There is nothing related to the core game logics: if the player doesn’t do anything, the game should also run well – only better.

Apparently a game without user interaction is useless. However, we see that the source of the input sequence has very different characteristics from the timing sequence: it is not essential to the game execution and is arbitrary. But without it, we cannot reconstruct the game state, either.

Typical Input Handling Methods

Usually, there are three ways of handling input: polling, checking the realtime status, and callbacks.

Polling

In a polling scheme, the interested party actively asks the input system whether there are incoming new input events. If there is any, they are processed and usually removed from the internal queue of the input system. Usually, each event comes with a timestamp stating its time of occurrence. This is simple and used by the operating system such as Windows to push events to the application. However, when the game logics is complex, such as if we have several layers of GUI, the game scene at the bottom, and an overlay debugging window, who knows who is the right person to remove an event from the queue and how many times an event should be processed? The order of execution? They are all implicitly stated in the code. It’s not impossible to implement, but hard to maintain, obviously.

Realtime Input

Instead, we can check for the status of each key and value of every axis during each frame. If W is pressed, make the character walk forward. If Shift is also pressed, let it run!

But how about checking for input sequences? There might not be a simple way to do so.

Callbacks

Instead of letting the game logic ask the input system for input events, now they just sit around and wait for the dinner to come to the table. Whenever an event comes, the input system notifies the interested parties of the input event and they decide what to do on their own. This is usually done by registering some callback functions, or listener classes. This approach also has the problem of deciding who to invalidate the event and who to process it, and perhaps slower than polling because every listener has to be notified regardless of whether they are interested in the current event.

A more serious problem is while polling the queue and reading realtime input have their problems, they play well with serialization: you can just record the events in the order of occurrence and replay them later by feeding them into the Systems. With callbacks present, it becomes much harder because anything can be sealed in the little closure passed as a callback, potentially this pointer, references to other objects, and so on.

Handling Input with ECS

First, let’s review some of the design objectives of the ECS:

1. Systems don’t rely on a particular state of the game to operate. It can be added and removed between any frames.
2. They also don’t own any runtime resources. All the resources are referred to using handles and stored in Components.
3. Given the same state of an Entity Database, the same System always produces the same output.

The motivations for them are:

• Efficient and reliable serialization of the game state for networking, game saves, and replay.
• Ease of debugging and prototyping.

The first question is whether should the input be handled in any System.

Like said before, since handling input usually means explicitly refer to some Entities, the first objective will be broken. Therefore the answer is no. In addition to this problem, it also makes the scheduling of Systems harder because of the extra dependencies on the order of handling input events. The Systems handling the input also cannot be parallelism due to the race condition involved. This scheme also makes reusing the Systems much harder since the input handling is usually specific to games, while general game update logic is not. It also violates the third objective since the operation now depends on extra information.

So, well, where to handle the input?

Just like in most game engines, we isolate the input() function from the update() function. The input handler doesn’t know that how would the Entities be updated later by Systems. Neither do Systems know that how the Entities were changed externally by the input handler after the previous frame.

Suppose that input() is a function receives the input and changes some Entities. We want explicit control over the order of input handling logics of different game components. We also want reproducible game states… Wait, why does this sound so similar to the Systems?

An Imaginative Game

Imagine there is another Entity Database whose sole purpose is for storing input events. In the meantime, suppose there are several Systems:

• SystemInputEventPump: Pumps input events from the operating system and creates Entities with ComponentKeyboardInputEvent or ComponentMouseInputEvent or similar, each represents an input event with a timestamp.
• SystemInputDebugGui: If the mouse cursor is active, any input event will be redirected to the debug GUI system. The key presses are captured per-frame, the mouse movements are accumulated, while the text input is recorded in the order of occurrence. If the mouse cursor is inactive, the input is ignored. However, if the player presses some designated key such as F1, the cursor is activated.
• SystemKeyHoldTimer: Record the time of holding for the pressed keys.
• SystemInputGameLogic: Controls the game character according to some custom key mapping.

Let’s categorize them as Input Systems to differentiate from those Systems used in updating game entities, or we can name them as Update Systems.

It is obvious that both event-based input and realtime input are involved in Input Systems. The realtime input can be derived from the input events: during the iteration of input events, the iterator can act as the current state of the input devices: incrementing the iterator causes the effect of the current event to be applied to the state of the device; decrementing reverses that effect. What if the Input System just wants the key states at the end of this frame? Just use the end() iterator. For reconstructing the game state and replay, it is always possible to record all the events, just be careful with the occasion of exposing them to the Systems. So this is not a big problem.

A significant difference in this use of the Entity Database comparing the use for Update Systems is, the Entities are visited in an undefined order for Update Systems since the Entity Database View only filters the Entities but not sorts them. For Input Systems, the input events must be iterated in the order of their occurrence timestamps.

Another difference is that while Update Systems should not have any connection with the outside world in order to achieve the first design objective, an Input System must have some means to contact the outside: the input events will convert to changes to the game world after all, while the Input Systems resident in a separate world, which is another Entity Database specifically created for handling input events.

Handling Input for Thirdparty GUI

Consider the case of SystemInputDebugGui. Suppose we use a thirdparty library to handle the GUI. The GUI system will have a context object which stores the states of all the windows we create and so on. This Input System simply cannot do anything meaningful without reaching out to that context object. This does not play well with any of our design objectives.

What can we do?

Whatever Entity Database we talk about, they don’t live in a vacuum. We have something like a Script which controls the creation of Entity Database, the task graph of Systems, and so on. It is the god of the game and knows everything. Because it creates everything and knows everything, it can just create the Input System and supply the reference to the GUI context. Since the Script outlives the Input System for sure and the context is fully controlled by the Script, we can guarantee that the Input System always holds a valid reference to the GUI context. Basically, we have made a compromise by allowing the System to have some connections to the outside, to be some more dependent, in a strictly controlled environment.

Custom GUI System

Consider another case where we implement our very own game GUI system by creating Entities with ComponentClickaleRegion, ComponentGuiButtion, and so on. They resident in the normal Entity Database and has nothing to do with the input system. They rely on some Systems to link them with the user input.

If the Systems want to process these Entities, they must be present in the task graph for processing the same Entity Database as well. However, we don’t have access to the input events here!

Probably we can make another compromise here by allowing the Update Systems to have read-only access to the state of input devices after input handling, basically the end() iterator. However, this could seriously limit what could be done as a GUI system.

GUI Events and Callbacks

Usually, a GUI system provides feedback regards the status of a given element. In immediate mode GUI, the clicked status of a button is immediately checked upon the invocation of the creation method of the button so that the program can act correspondingly. In deferred mode GUI, the program registers a callback for particular events. When the event happens, the callback is called. With a GUI system implement using ECS, we have nothing like these because functions cannot be put into Components and inventing a System for every possibility of event handling is basically insane. We will have an infinite number of different Components just because we have different behaviors for every button.

Who create those buttons? It is the Script. It also creates initial Entities and put them into the Entity Database so that Systems keep updating them. It also manipulates some Entities on occasions in order to achieve specific game logic. Therefore, it is completely reasonable that it would like to handle the events for GUI elements.

We can work around the problem by just strip the burden of processing event handling logic out of the Systems. The Script holds a handle to each GUI element, the System checks the condition for is a button clicked, and sets a status flag in the Component data. Later the Script checks for that flag and does things correspondingly.

If we have a dynamic number of elements, such as a file list, we can use a range object to keep track of a dynamic number of Entities and recreate them on demand.

We can do the same thing for the animation system: The System only increments the animation counters and status. The Script decides what to do by checking for those values during each frame.

Passing on The Output of Input Systems

Still, there left the problem of passing the output of Input Systems to the Entity Database for the game world.

The Script creates those Entities of our special interests and holds their Entity Ids in order to brutally modify them later. The Input System has to know these Ids so it can know which Entities to modify according to the input events. The System can have a copy of these Ids, but this feels dirty since we have to keep them in-sync with the version in the Script.

Since the Script also creates those Input Systems, it actually owns those Systems. Therefore, the Script can just inject Entity Ids into the Input Systems without holding them on its own.

struct SystemInputGameLogic
{
    struct EntityReferences
    {
        EntityId player;
        EntityId vehicle;
    } ref;

    template <typename EntityDatabase>
    void update(
        InputEventQueue &ieq,
        EntityDatabase &world_edb)
    {
        // Read the realtime state of the keyboard
        auto &&keyboard_state = ieq.end().keyboard_state;

        if(keyboard_state.isPressed(KEY_W))
        {
            USAGI_COMPONENT(
                world.entity(player),
                ComponentAnimation
            ).animation_sequence_name = "walk_forward";
        }
    }
};

class GameScript
{
    Archetype<ComponentPlayer, ComponentAnimation> mArchetypePlayer;

    EntityDatabase</* ... */> mInputEventQueue;
    EntityDatabase</* ... */> mGameWorld;

    SystemInputGameLogic mSysInputGameLogic;

    TaskGraph</* ... */> mTaskGraphInputHandling;

    SystemA mSystemA;
    SystemB mSystemB;
    SystemC mSystemC;
    SystemD mSystemD;
    SystemE mSystemE;

    TaskGraph</* ... */> mTaskGraphGameLogic;

    Clock mMasterClock;
    RuntimeServices mRuntime;

public:
    GameScript()
    {
        // A >> B means A must be executed before B
        // A | B means that A and B does not have dependency
        // in the execution order
        mTaskGraphInputHandling =
            mSysInputGameLogic;

        mTaskGraphGameLogic = 
            mSystemA >>
            (mSystemB | mSystemC | mSystemC) >>
            mSystemD >>
            mSystemE
        ;
    }

    void initWorld()
    {
        mSysInputGameLogic.ref.player =
            mGameWorld.create(mArchetypePlayer);
    }

    void frame()
    {
        mRuntime.input_source.pump(mInputEventQueue);
        mTaskGraphInputHandling.execute(mInputEventQueue, mGameWorld);
        mTaskGraphGameLogic.execute(mGameWorld, mRuntime);
    }

    void saveState()
    {
        mGameWorld.dump();
        mInputEventQueue.dump(); // ?
        // binary dump the mSysInputGameLogic?
    }
}

Notice that in this scheme, the Input System doesn’t hold any runtime resources. The only states it has are the references to Entities in the game world. Therefore, the Input System can still be safely serialized. As long as there is the same event sequence in the queue, the System should principally always do the same thing, too.