OpenAI Launches Neural MMO to Train AI in Complex, Open-World Environments
Over the past few years, we’ve seen a number of AI projects demonstrate how effectively artificial intelligences can play certain games, from classics like chess to the Chinese game Go and even DOTA 2 and Starcraft 2. Now, the non-profit OpenAI has released what it calls Neural MMO. The look is heavily reminiscent of Minecraft, but the long-term impact of the idea could be considerable.
We’ve actually touched on some of these topics in a discussion of in-game AI and how DirectX 12 might lead to improvements by freeing up system resources to be spent on AI instead of handling GPU rendering requests. (Spoiler alert: Nothing of the sort has happened yet). Nonetheless, this article should be considered a primer on the kinds of issues game developers often face when implementing AI systems.
OpenAI writes:
[T]here are still two main challenges for multiagent reinforcement learning. We need to create open-ended tasks with a high complexity ceiling: current environments are either complex but too narrow or open-ended but too simple. Properties such as persistence and large population scale are key, but we also need more benchmark environments to quantify learning progress in the presence of large population scales and persistence. The game genre of Massively Multiplayer Online Games (MMOs) simulates a large ecosystem of a variable number of players competing in persistent and extensive environments.
The suitability of MMOs for modeling real-life events has been extensively explored over the last decade. During certain in-game events in World of Warcraft — the Corrupted Blood incident and the Scourge Invasion being the most prominent — players spontaneously enacted their own quarantine protocols in response to game issues that infected unkillable NPCs with a transmissible disease that allowed them to infect others within a specific radius. During the Scourge Invasion, it was possible to become infected with a disease that transformed you into one of the undead yourself. In both cases, the ways players responded to the incident were later studied in epidemiological research, it being generally difficult to secure funding for a research effort in which half of a real-time town is exposed to a pathogen to study how the other half reacts.
MMOs, in other words, are of great interest to researchers because they offer a limited subset of activities that require people to make sophisticated decisions about how to prioritize time and resources, but do not contain nearly as many competing claims or variables to track.
In this case, the implementation is fairly simple. The AI agents must acquire food and water to stay alive, and they move across the map to gain both. This brings them into conflict with other agents and requires the AI to move carefully in order to maximize the chance of finding resources as it explores. Agents forage for food and must refill their water supplies while competing against each other with three different attacks.
OpenAI states that “One purpose of the platform is to discover game mechanics that support complex behavior and agent populations that can learn to make use of them. In human MMOs, developers aim to create balanced mechanics while players aim to maximize their skill in utilizing them. The initial configurations of our systems are the results of several iterations of balancing, but are by no means fixed: every numeric parameter presented is editable within a simple configuration file.”
One of the research team’s strongest findings? Training more agents per map always results in stronger performance when servers are “merged” and the agents from each are set to compete against each other.
Training multiple species (populations) of agents resulted in different exploration patterns. Training just one species tended to produce a deep exploration path through the map, while training multiple species resulted in a very different exploration pattern, as AI agents attempted to spread out to colonize different niches (in this training instance, entities from the same population were unable to outcompete one-another).
The gap between the work “real” AI researchers do and what games and game developers tend to incorporate is massive, but projects like this hint at ways to one day bridge the two. Imagine playing a game where the NPC characters weren’t just well-scripted, they were actually capable of learning and fighting more effectively, adapting themselves to your own methods of playing a game, and working with you to achieve mission goals. It’d be much more like playing the game with a partner, absent all the jokes about your mom.
Continue reading
Elon Musk’s Neuralink Shows Off a Monkey Playing Pong With Its Mind
The secretive company has just released a video demonstrating its brain-machine link technology. It features a monkey drinking smoothies and playing Pong with its brain.
wfoojjaec Explains: What is a Neural Net?
We talk a lot about AI, machine learning, and neural nets, but what's a neural net in the first place?
wfoojjaec Explains: What is a Neural Net?
We talk a lot about AI, machine learning, and neural nets, but what's a neural net in the first place?
Researchers Devise a ‘Honey Memristor’ for Neuromorphic Chips
Researchers have devised a way to use honey as a memristor, paving the way for sweet new forms of neuromorphic hardware.