A lot of interesting bots were submitted to the StarCraft AI Competition. However, even the best could not beat an expert human player.
But what about amateur players? Kotaku claimed that most casual StarCraft players would be unable to defeat the bots. The goal of this post is to test that claim. To do so, I played against the winner of each of the four tournaments. While I consider myself a hardcore StarCraft player, my skill is nowhere near professional.
Tournament 1: Micromanagement
The first tournament focuses on micromanagement in flat-terrain environments. The winner was FreScBot, which uses a multi-agent, finite-state-machine approach. The results are shown in the video below:
Man (Blue) vs Bot (Red)
Results:
Dragoons: 2-1 bot advantage
Mutalisks: 2-0 bot advantage
Zealots: 2-1 bot advantage
Winner: Bot
While I was able to win 2 of the games, the bot’s micro was too much for my skill.
Tournament 2: Small-Scale Combat
The second tournament builds on the first tournament by adding interesting terrain to the maps. The winning bot was again FreScBot and my encounter versus the bot is shown below:
Man (Blue) vs Bot (Red)
Results:
Match 1 – Dragoons: 1-0 man advantage
Match 2 – Marine/Medic: 1-0 man advantage
Match 3 – Dragoons: 1-0 man advantage
Winner: Man
Adding complex terrain into the mix really offset the skill of the bot. While I was unable to defeat the bot’s dragoons micro in the first tournament, I was able to use the topology of the map to my advantage in the second tournament.
Tournament 3: Tech-limited Game
The tech-limited game tournament evaluates bots in a simplified StarCraft environment. The winner was Mimic Bot, which attempts to mirror your build while also stealing your gas and expanding as necessary:
Man (Orange) vs Bot (White)
Man (Blue) vs Bot (Yellow)
Man (Yellow) vs Bot (Red)
Results:
2-1 man advantage
Winner: Man
This match was surprisingly difficult. The first game I went for dragoons and was simply out micromanaged by the bot. In the second game, I focused on zealots and used the lack of collecting gas to gain a slight advantage in zealot count. In the third game, I had some fun with the bot, and somehow pulled out a win. While I was able to win a majority of the matches, the games were very entertaining and I enjoyed the challenge provided by the bot.
Tournament 4: Full Gameplay
The complete StarCraft tournament evaluates bots in the full-blown StarCraft game. The winner was Berkeley’s Overmind, which uses a shallow planner, A*, potential fields and reinforcement learning:
Man (Blue) vs Bot (White)
Man (Red) vs Bot (Purple)
Man (Orange) vs Bot (Yellow)
Results:
Terran: 1-0 man advantage
Protoss: 1-0 man advantage
Zerg: 1-0 man advantage
Winner: Man
In the first game, I went for a Fantasy-style build with a vulture drop. I had several issues with my build, such as a failed wall-off, but I was still able to pull out a victory, eventually. Once the Overmind produced mutalisks, it relentlessly attacked my base, so it took me awhile to deal with the opponent’s expansions. In game two, the bot simply could not handle the zealot timing. Part of the issue is that the bot went for a 12-pool instead of a 9-pool, even though it scouts quite early in the game. And in the final game, the bot could not handle my zergling speed while it attempted to tech too quickly.
Conclusion:
Despite advances in AI, man still reigns supreme, even at an amateur level. There are a few interesting takeaways from this experiment:
- These bots are really fun to play against! Overmind and MimicBot challenged me in ways I never imagined, and it really required adapting my strategy to overcome the opponent.
- Building a zerg bot is hard! Berkeley was pretty brave in deciding to go with Zerg and I commend them for their effort.
I’d like to encourage people to play against the bots, but the setup is a little challenging. All bots are now available on the tournament website.
