This AI created Pac-Man from scratch just by watching it being played

Nvidia is known for its graphics cards but the company also invests a lot into artificial intelligence (AI) and one of their latest projects has been an AI system that recreated the Pac-Man game by just just watching it being played.

The software did not use any coding or any pre-rendered images. The AI system worked with visual data of the game being played along with the accompanying controller inputs. From that it recreated the game frame by frame. The end product is playable and Nvidia plans to release it online in the future.

The AI version is not perfect, though. The images are a little blurry and the AI might not have managed to “capture the exact behaviour of the game’s ghosts, each of which is programmed with a specific personality that dictates its movement”. But, the basic dynamics of the game are there - eat the pellets, avoid the ghosts and try not to die.N

“It learns all of these things just by watching,” Nvidia’s Rev Lebaredian, vice president of simulation technology, told journalists in a briefing.

“[It’s] similar to how a human programmer can watch many episodes of Pac-Man on YouTube and infer what the rules of the games are and reconstruct them,” Lebaredian added.

Lebaredian said “the work had been done in collaboration with Pac-Man’s creator, Bandai Namco, which celebrated the 40th anniversary of the arcade classic yesterday (May 22)”.

Nvidia believes that work like this shows that artificial intelligence can be used for game design in the future - developers can put their work into the AI and use it to create variations or design new levels.

“You could use this to mash different games together, giving additional power to games developers by [letting them] blend together different games,” ” Sanja Fidler, director of Nvidia’s Toronto research lab, told journalists.

An AI that can learn virtual world rules simply by watching things in action is also a marker for programming robots.

“Eventually we’d like it to learn the rules of the real world,” says Lebaredian. The AI might “watch videos of robotics trolleys navigating a warehouse, for example, and use that information to design navigation software of its own”.

The program that recreated the new Pac-Man is called GameGAN. GAN stands for generative adversarial network and is “a common architecture used in machine learning”. 

The basic principle of a GAN is that it works in two halves - first, “half of the GAN tries to replicate the input data, while the second half compares this to the original source”. Then, if they don’t match, “the generated data is rejected and the generator tweaks its work and resubmits it”.

AI has been used to generate virtual worlds (like in video games) before, but Nvidia’s research “introduced several new aspects, including a ‘memory module’ that allowed the system to store an internal map of the game world”. This allows for greater consistency in the game world, “a key characteristic when recreating the mazes of Pac-Man” and “also allows for the static elements of the game world (like the maze) to be separated from the dynamic ones (like the ghosts), which suits the company’s goal of using AI to generate new levels”.

David Ha, an AI researcher at Google who’s worked on similar tasks, told The Verge that the research was “very interesting”. Ha said that teams have tried to recreate game worlds using GANs, “but from what I know, [this] is the first to demonstrate good results”.

“All in all, a very exciting paper, and I look forward to see more developments using this approach,” said Ha.

Some elements of the process definitely need more work and demonstrate the “particular fragility of artificial intelligence when learning new tasks”. Fidler told journalists that to recreate Pac-Man, GameGAN had to be trained on some 50,000 episodes. Getting that gameplay data from humans wasn’t feasible so the team behind GameGAN used an AI agent to generate the data. Unfortunately, “the AI agent was so good at the game that it hardly ever died”.

“That made it hard for the AI trying to recreate the game to learn the concept of dying,” says Fidler.

Instead, in early versions of the AI-generated Pac-Man, “GameGAN tweaked the game so that ghosts never actually reached the title character but trail directly behind it like baby ducks following a parent”.

“It’s a funny effect of the way we trained it,” says Fidler.