Yesterday, OpenAI announced its release of a massive multiagent game environment for reinforcement learning agents. The company’s newest Neural MMO platform reportedly supports a “large, variable number of agents” within a “persistent and open-ended task.”
In a statement, OpenAI wrote:
“In recent years, multiagent settings have become an effective platform for deep reinforcement learning research. Despite this progress, there are still two main challenges for multiagent reinforcement learning. We need to create open-ended tasks with a high complexity ceiling: current environments are either complex but too narrow or open-ended but too simple.”
According to OpenAI, more benchmark environments are required to quantify the learning progress of AI agents in large population scales and persistence. Apparently, an MMO structure is the best solution as it simulates a vast ecosystem where a variable number of players can persistently compete within large environments.
MMO Reinforcement Learning Simulation
OpenAI claims that its Neural MMO reinforcement learning simulator can address the critical challenges of training AI agents. These challenges include persistence, scale, efficiency, and expansion.
The MMO simulator allows players or agents to join available servers or environments containing an auto-generated tile-based game map of configurable size. There are traversable tiles like food-bearing forests and grass tiles along with non-traversable tiles like water and solid stone tiles.
Agents will spawn at random locations along the edges of the environment where they need to gather food and water while avoiding damage from other agents.
OpenAI reported that they trained their AI system to reward agents for staying alive during simulation. The researchers observed that the longer the AI agents communicate with each other, the better they execute tasks.
OpenAi further wrote:
“In the natural world, competition among animals can incentivize them to spread out to avoid conflict. We observe that map coverage increases as the number of concurrent agents increases. Agents learn to explore only because the presence of other agents provides a natural incentive for doing so.”
This is not the first time that OpenAI released an environment for AI agents. Last year, the company released a tool called CoinRun to measure an agent’s ability to transfer its experiences of unfamiliar scenarios.
The Neural MMO reinforcement learning simulation is now available for download on GitHub.
Maybe this will finally let me play a game without watching the AI blindly run into walls for half the campaign.