Techniques and Paradigms in Modern Game AI Systems
Round 1
Reviewer 1 Report
The article "Techniques and Paradigms in Modern Game AI System" aims to give a systematic overview of the techniques and paradigms used in modern game AI systems, and so it does.
It starts with a very well formulated introduction, giving a first insight into the subject area and the most important developments.
The aim of the work is also motivated and compared with and differentiated from previous work.
In Section 2 some preliminaries and background knowledge is conveyed. Starting with a chronological overview of the developments and milestones, through a structured overview of the relevant features of the considered games, to an overview of the most important modeling approaches used in game AI modeling.
In Section 3, different game AI techniques are introduced, based on an extended timeline that also indicates modeling approaches and game types tackled with each algorithm. Based on this introduction the different components of game AI modeling a separately and thoroughly treated.
This section is followed by a section on the milestones of Game AI Systems, broken down by the three game types introduced earlier. Here, the developments for the different game types are listed in great detail, with the methods used, computing time spent and successes also being presented in each case.
In the next section the game AI milestones are analyzed for their paradigms used, and techniques for different game features are discussed. Both sections are accompanied by well-structured tables that provide a quick overview.
In the last section 'Conclusion,' the presented work is again briefly summarized, and the main results and benefits of the work are highlighted.
This very well-written paper presents a neat taxonomy of AI in games that was not yet existing in this form. I would have loved to see games like simulated car racing listed here, which arguably fall into the video games category, but unlike the ones presented here, don't just get graphical information. But the subject area is large, and it is understandable if it can not be exhaustively reviewed.
Trifles that I noticed while reading the article:
line 97: "...optimal solution of the game. [17]. " - remove the first dot
line 113/114: "...it later won 99.4% of..." - maybe better: "won against" or "defeated" ?
Author Response
Thank you for your insightful and positive comments! Your summary of the structure of this article is very accurate and consistent with our design of writing. We also appreciate the typos and word misuses you pointed out, which we will fix in our next submission.
As for games like simulated car racing, there are some milestones in these games [1], which is touched on in our survey (Section 5.3, line 1062~1064). We would certainly like to include it in the case study of milestones if we could afford the space, but there are too many famous AI systems of video games for this survey to cover, such as first-person shooting (FPS) games like Quake III Arena in Capture the Flag (CTF) [2] and simulated car racing like Gran Turismo as you mentioned. Meanwhile, the AI systems in [1] and [2] share similar methods and paradigms with those selected in our survey, which is also deep reinforcement learning combined with self-play schemes. It is worth noting that though the observations of simulated car racing contain non-graphical information like sensor data, the AI systems for MOBA games do not use graphical information alone but also non-graphical data from APIs of the gamecore. In this way, they are quite similar in their methods, and we believe three cases are already representative to reveal the paradigm of AI systems for video games. We choose StarCraft, Dota II, and HoK as the cases of video games in our survey mainly because they are the most popular video games played worldwide, and their milestones are well-known even by people in non-scientific fields.
[1] Outracing champion Gran Turismo drivers with deep reinforcement learning.
[2] Human-level performance in 3d multiplayer games with population based reinforcement learning.
Reviewer 2 Report
The paper Techniques and Paradigms in Modern Game AI Systems presents a survey about latest achievements in AI technologies used in games.
The language of the paper is good in grammar and readability.
The survey is very interesting and valuable.
The authors processed 93 articles and presented the survey in text, tables, and one figure.
In such a long article it would be nice to get more tables and figures. Tables and figures are good techniques to summarize information.
The use of heuristics is slightly underweighted, although heuristics are used in many algorithms from alpha-beta pruning to neural network initialization or parameters.
The article presents such a broad and comprehensive investigation in this field that it can be extended in the form of a book.
As an article it gives much more background compared to the deep technical information acquired from relevant articles.
Many times we do not get exact performance information which is a usual attribute of a scientific article in artificial intelligence field.
Despite the above criticism the article is good quality and it worth to be published.
Author Response
Thank you for your valuable and positive comments! We will explain your concerns point by point.
Point 1:
This article is a little short on tables and pictures.
Response 1:
We agree that tables and figures are good techniques to summarize information by presenting structured information or illustrations. Our survey provides a systematic review of AI techniques and paradigms used in game AI systems, where many AI system milestones are analyzed, and the game features, AI techniques, and their paradigms are compared and summarized. These points are all supported by tables and figures: an extended timeline for basic AI techniques and the game features they tackle, a table for games and their features, and two tables to summarize the techniques and paradigms in each of these milestones. As a survey, our task is to systematically review the existing literature and present our taxonomy of AI in games, so we do not have many figures to illustrate the implementation of each AI system in isolation, but mostly tables to summarize their similarities and compare their differences as a whole.
Point 2:
The use of heuristics is slightly underweighted, while heuristics are widely used in AI algorithms.
Response 2:
We agree that heuristics are widely used in AI algorithms, from the earliest AI systems that relied almost entirely on heuristics to modern AI systems that combine heuristics with deep learning. In the first draft of this survey, heuristics was listed as a whole subsection in Section 3 (Game AI Techniques), and aspects like behavior trees and handcraft evaluation functions were discussed. However, we deleted the subsection in later revisions (before our submission) for three reasons. First, heuristics are not a single algorithm or a group of algorithms, but as long as human experience is introduced, making it difficult to fit in our taxonomy of AI algorithms, such as the timeline in Figure 1. Second, all of the AI system milestones covered in our survey use various heuristics related to specific games each of them tackles, so including heuristics when we analyze the components of these milestones seems to be somewhat trivial. Third, using a subsection to discuss heuristics made the section on AI techniques so long that it broke the balance of content in our survey. After all, the task of our survey is not only to summarize the AI techniques themselves but also to analyze how they are used in modern AI systems. If we could extend this survey in the form of a book, there would be enough space to discuss heuristics, but as an article we decided not to cover heuristics considering our taxonomy and the length of our survey.
Point 3:
As an article it gives much more background than deep technical information from relevant articles.
Response 3:
This article is a survey that aims to give a systematic overview of the techniques and paradigms used in modern game AI systems. The many AI techniques systematically introduced in Section 3 are not background information, but one of the main topics we want to summarize in this article: the basic algorithmic components commonly used in game AI systems. Meanwhile, we do convey some preliminaries and background knowledge in Section 2, including a chronological overview of the developments and milestones of game AI, a structured overview of the relevant features of the considered games, and an overview of the most important modeling approaches used in game AI modeling. This section provides the necessary foundation for the introduction of the specific game AI techniques and milestones of AI systems for various types of games in the following sections, and it does not amount to much: only making up four pages out of the whole 23-page survey.
Point 4:
The performance measures of some AI milestones covered in this survey are not reported and compared, though they are commonly reported in scientific articles in AI fields.
Response 4:
In the field of computer vision (CV) and natural language processing (NLP), there are many standard tasks like image classification, object detection, machine translation, etc., where many benchmarks are available to measure the performance of newly proposed AI algorithms. However, in the field of game AI, each game is itself a task where algorithms are designed to achieve a higher level of play, and the performance measure depends on the scoring rule of each game. The many milestones of game AI systems covered in this survey tackle a wide range of games, so there is not a consistent performance measure that can be compared, but we have reported the performance of each AI system in Section 4 when they are introduced one by one in detail. It is worth noting that head-to-head performance / winning rate is the most common performance measure in multi-player games, so beating professional human players or human champions is already an important performance measure for many milestones listed in this survey.
