Developing Intelligent Bots for Real-Time Strategy Games

Abstract

Real-Time Strategy (RTS) games impose multiple complex challenges to autonomous game-playing agents (a.k.a. bots), that also relate to real-world problems. The real-time aspect and the astronomical size of the decision and state spaces of an RTS game overwhelm the usual search algorithms. Monte-Carlo Tree Search (MCTS) was successfully applied in games featuring large decision and state spaces, such as Go, and was able to attain super-human performance in agents like AlphaGo and AlphaZero. Thus, researchers turned to MCTS as a potential candidate for solving RTS Games, and several RTS-specific enhancements were implemented, such as the support for real-time progression and combinatorial decisions. Nevertheless, MCTS is still far from replicating its Go success in RTS games. In this thesis, we propose several approaches to ease the RTS dimensionality burden on MCTS, in hopes of finding a path towards higher performance. To this end, we have made use of a detrimental-move pruning approach, proposed an integrated action/state abstraction process, and optimized its parameters through an Evolutionary Algorithm (EA). These approaches were tested and validated in the μRTS research platform, and the results showed moderate to significant performance gains. We expect the proposed approaches could be applied in commercial RTS games in the near future.