The coolest tech in Apple’s new iPads: What is LiDAR and how does it work?

Apple launched their new iPads yesterday. And amongst cool tech like the A12Z Bionic Chip and trackpad support, these new devices come with a LiDAR scanner. So, what is it and what can it do?

Apple’s new LiDAR tech in iPads explained (Apple)

Apple's new iPads, launched yesterday, come with a new A12Z Bionic chip, ultra-wide cameras, studio-quality mics, trackpad support and their 'breakthrough' LiDAR (Light Detection and Ranging) Scanner.

The new lineup includes the refreshed model of both the 12.9-inch and 11-inch variants but the highlight is the inclusion of dual rear cameras and this is also the first time that we are seeing dual rear cameras in an iPad and a 'Z' series chipset by the firm. You can read about all the new features here.

The LiDAR Scanner, along with pro cameras, motion sensors, pro performance, pro audio, the Liquid Retina display and Apple's suite of apps potentially makes the new iPad Pro one of the best devices for augmented reality (AR).

So, what exactly is the LiDAR Scanner and what can it do?

The LiDAR Scanner measures the distance to surrounding objects up to 5 meters away, it works both indoors and outdoors and operates at the photon level at nano-second speeds.

Every existing ARKit app on Apple will automatically get instant AR placement, improved motion capture and people occlusion. (Apple)

Every existing ARKit app on Apple will automatically get instant AR placement, improved motion capture and people occlusion. (Apple)

The new depth frameworks in the iPadOS combine depth points measured by the LiDAR Scanner, along with the data from both cameras and motion sensors - all of this together is then enhanced by computer vision algorithms on the A12Z Bionic for a more detailed understanding of a scene to create more intense AR experiences on the iPad Pro.

For example, the LiDAR Scanner can turn your living room floor into an AR version of Hot Lava (the video game that is based on the game we played as kids where you jumped from furniture to furniture without touching the floor that is covered with imaginary lava).

A player scans a living room with an iPad, and the app generates hot lava on the floor and platforms for the game's character to jump on in AR. You can play along.

Every existing ARKit app on Apple will automatically get instant AR placement, improved motion capture and people occlusion. Using the latest update to ARKit with a new Scene Geometry API, developers can use the new LiDAR Scanner to create scenarios never before possible.

The LiDAR Scanner also improves the Measure app on iPads making it easier to calculate someone's height and measure objects more accurately.