Guess how many cameras Google used to train just one AI photography feature in Pixel phone

The feature uses Google’s algorithms to alter the lighting in a portrait image so it looks like the source direction is from somewhere else.

Google Pixel 4A. (Google)

Google Pixel smartphones are known for their camera prowess and how well they can capture lights or use AI to deliver a good portrait shot. However, it takes a lot behind every feature, something that is available in Pixel at the tap of the button. When the firm came up with an improved portrait mode in Pixel 3, it used a 5-phone case to train the machine learning models to the resulting pictures in Pixel handsets are near-perfect. But that 5-phone rig seems nothing before the one created by the firm for its new Portrait Light mode.

As revealed by the search giant, the custom lighting rig included as many as 64 different cameras along with individually programmable 331 LED light sources. And this massive rig was used on 70 different people. This showed how the face of the same subject in the same position gets a different perspective when the light is thrown from different directions. 

Not sure which
mobile to buy?

64-camera custom rig for Portrait Light mode training.  (Google AI blog)

64-camera custom rig for Portrait Light mode training.  (Google AI blog)

“We generated training data by photographing seventy different people using the Light Stage computational illumination system. This spherical lighting rig includes 64 cameras with different viewpoints and 331 individually-programmable LED light sources. We photographed each individual illuminated one-light-at-a-time (OLAT) by each light, which generates their reflectance field — or their appearance as illuminated by the discrete sections of the spherical environment,” stated Yun-Ta Tsai and Rohit Pandey, software engineers, Google Research.

This resulted in the Portrait Light mode in Pixel 4A, which came to India in October this year. It is also present in Pixel 5, which Google said, won't be coming to India at all. The new mode, in case you haven't tried yet or are unaware, uses Google's algorithms and the learning from the 64-camera custom rig to alter the lighting in a portrait image so it looks like the source direction is from somewhere else.

It is worth adding that Pixel 4A and Pixel 5 applies Portrait Light automatically when the image is clicked in default or Night mode when it detects faces. The tech also works for group images. The Pixel 2 and Pixel 3 users can try it on existing portrait shots from the edit feature in Google Photos app.