MICAH MATEO VARGAS
Game Developer
Project Type:
Engine:
Language:
Other Tools:
Team Size:
Role:
Duration:
Machine Learning
Unity
C#
Neural Networks
5
Developer
3 months
AlphaJoust is a machine learning demo where an agent was trained to play the classic arcade game Joust using the NEAT algorithm (NeuroEvolution of Augmenting Topologies).
A key part of the project involved recreating Joust in Unity. Since our focus was on AI, our team built upon a pre-existing Joust remake, which provided a foundation while allowing us to modify and extend it. We made several enhancements, including adjustments to the physics to better match the original game, which was coded in Assembly. We referenced the original Assembly code to ensure accuracy and fidelity to the originals gameplay.
For the AI training, we utilized UnitySharpNEAT, an open-source implementation of NEAT for Unity. UnitySharpNEAT builds on SharpNEAT, a C# NEAT framework, providing features like object pooling, better coroutine management, and multi-experiment support. This library streamlined the integration of neural networks into Unity, making it well-suited for Joust’s mechanics, which require rapid decision-making and adaptive strategies.
To better structure the agent's iteraction with the game environment, we designed a modular system, as shown in the UML diagram. This system manages game state input, fitness evaluation, and neural network outputs efficiently.
NEAT works through three primary techniques: applying three key techniques:
genetic encoding, tracking genes through historical markings, and protecting innovation through speciation.
1. Genetic Encoding: Each neural network consists of node and connection genes, whcich mutate over generations by adding new nodes, connections, and modifying weights.
2. Tracking Genes through Historical Markings with Innovation Numbers:
NEAT assigns a unique global innovation number to each new mutation, such as an added connection or node.
This allows the algorithm to efficiently track the evolutionary history of genes.
When comparing genomes, innovation numbers help determine which genes share common ancestry and which are unique, facilitating seamless crossover without requiring topological analysis.
This approach ensures structural compatibility between networks, allowing effective recombination of solutions.
The diagram illustrates the genome crossover process, depicting how NEAT handles disjoint and excess genes by aligning genes based on their innovation numbers. This visual representation clarifies how NEAT ensures genetic compatibility and preserves meaningful structures during evolution.
3. Speciation for Innovation Protection: NEAT protects innovation by dividing the population into species based on topological similarity. This is achieved by comparing innovation numbers to determine the degree of similarity between genomes—more disjoint genes result in lower similarity, leading to different species classifications. Speciation preserves diversity by employing explicit fitness sharing, where individuals within a species share fitness values, ensuring that novel or complex structures have the opportunity to evolve without direct competition from more optimized existing solutions. This mechanism played a crucial role in the Joust use case, allowing different strategies to develop and preventing premature convergence on suboptimal behaviors.
We trained the agent by using Joust's score as a fitness metric, giving us a clear representation of performance. The agent received inputs for its own position and enemy positions, and produced outputs for horizontal movement and flapping. By leveraging Unity's timescale adjustment, we were able to train from 1 to 10x speed on personal devices which resulted in steadily increasing scores over generations.
The flow chart here visually represents the evaluation process of genomes during training, making it easier to understand how fitness evaluation and evolution proceed step-by-step. It outlines the sequence in which genomes are initialized, evaluated based on their performance in the game, and subsequently evolved by applying genetic operators such as mutation and crossover. This structured approach ensures that the most promising solutions are iteratively refined across generations, leading to increasingly effective gameplay strategies.
To monitor training progress, I developed a feature called MultiCam that dynamically displayed multiple simulations on screen simultaneously. Each simulation had its own camera rendering to a texture, which was applied to a grid of quads. I programmatically created a Camera for each instance of Joust, outputted the camera's view into a RenderTexture, placed that RenderTexture into a material, and applied that material to quads arranged in a square grid. While this approach had some performance cost, it allowed for quick evaluation of the agent's progress and could be toggled on or off as needed.
