Facebook Is Building a Minecraft AI Because Games May Be Great Trainin

It turns out that video games may be an excellent method of teaching skills to artificial intelligence assistants. That’s the theory of a group of researchers working for Facebook, who have focused on Minecraft as a potential teaching tool for building generalist AI — a so-called ‘virtual assistant.’ The research team isn’t trying to build an artificial intelligence that’s super-good at classifying images or other content — it wants to build a generalist AI that can perform a much larger number of tasks reasonably well.

This is, to-date, an under-studied area of research. The authors’ write:

There has been measured progress in this setting as well, with the mainstreaming of virtual personal assistants. These are able to accomplish thousands of tasks communicated via natural language, using multi-turn dialogue for clarifications or further specification. The assistants are able to interact with other applications to get data or perform actions.

Nevertheless, many difficult problems remain open. Automatic natural language understanding (NLU) is still rigid and limited to constrained scenarios. Methods for using dialogue or other natural language for rich supervision remain primitive. In addition, because they need to be able to reliably and predictably solve many simple tasks, their multi-modal inputs, and the constraints of their maintenance and deployment, assistants are modular systems, as opposed to monolithic ML models. Modular ML systems that can improve themselves from data while keeping well-defined interfaces are still not well studied.

According to the team, they picked Minecraft because it offered a regular distribution of tasks with “hand-holds for NLU research,” as well as enjoyable opportunities for human-AI interaction, with plenty of opportunities for human-in-the-loop research. Minecraft, for those of you who haven’t played or heard of it, is a block-based crafting and exploration game in which players explore a 3D voxel grid universe populated with various types of materials, neutral characters, and enemies. The team’s goal is to build an AI virtual assistant that can be given instructions in natural language by a Minecraft player, and that can reliably complete some of the primary tasks that player might engage in, including gathering materials, building structures, fighting mobs, and crafting items.

The authors of the paper target three specific achievements: Create synergy between machine-learning and non-machine-learning components, allowing them to work together; create a “grounded” natural language simulation that allows the AI to understand what players want it to do, and can communicate its success or failure back to the end-user; and create an AI that shouldn’t just be capable of doing what the player wants it to do, but that also its performance should improve based on observation of the human player.

They write:

We intend that the player will be able to specify tasks through dialogue (rather than by just issuing commands), so that the agent can ask for missing information, or the player can interrupt the agent’s actions to clarify. In addition, we hope dialogue to be useful for providing rich supervision. The player might label attributes about the environment, for example “that house is too big”, relations between objects in the environment (or other concepts the bot understands), for example “the window is in the middle of the wall”, or rules about such relations or attributes. We expect the player to be able to question the agent’s “mental state” to give appropriate feedback, and we expect the bot to ask for confirmation and use active learning strategies.

The machine learning code being used for the Facebook-Minecraft bot is available on GitHub. Better AI tools could be useful in many games, though they could also raise serious questions about what constitutes cheating in multiplayer.

Continue reading

Minecraft With Ray Tracing Now Available for All Windows 10 Players

You don't usually think of Minecraft as a realistic game, but the developers have been hard at work adding RTX ray tracing to the game for the last eight months. It's finally out of beta today, and it really works with the blocky look of Minecraft.

Minecraft Bans NFTs and Blockchain Technologies

Developer Mojang has clarified its position on NFTs, and it's not good news for anyone who was hoping to inject crypto scarcity into the block-based world. Mojang says NFTs will not be supported in Minecraft.

NFT Group Blocked from Minecraft Vows to Make its Own Version of the Game

NFT Worlds, which has made a business of selling crypto-infused Minecraft content, is in search of a new path forward and may have to pivot by creating its own NFT-based version of the game.

New Path-Traced Shaders Make Minecraft Look Amazing

Adding path tracing to Minecraft turns it into a beautiful game.