Google Developer Days Sheds Some Light on ARCore

During Google Developer Days, taking place now at the ICE Congress Center in Krakow, Poland, an introductory session on ARCore provides some insights on how the platform operates.

The presentation by Googler Tom Slater includes an overview on the three pillars by which ARCore enables augmented reality for Android phones.

Motion Tracking

The challenge with motion tracking is converging virtual scenes with the camera view. This requires translating the movement and position of the smartphone. Inaccurate translation can lead to virtual objects floating and swimming on the view of the physical world.

Google Developer Days Sheds Some Light on ARCore — *Image by Google/YouTube*

"This sounds simple; it's actually really, really hard. While your phone gyroscope is really, really great for rotation, it can drift over time. And while your accelerometer is great for those instantaneous inputs, it's not so great to figure out actual position," said Slater.

Instead of using only the gyroscope, Google designed ARCore to leverage the device's camera and inertial measuring units (IMU) to determine the position of the device relative to the physical world. Using a process called concurrent odometery and mapping (COM), the device captures visually-distinct locations to build a point cloud that the device can reference position against IMU data.

Plane Finding

To understand the environment, ARCore looks for clusters of feature points that appear to form horizontal surfaces. ARCore also defines the bounds of these surfaces as polygons.

Because ARCore uses feature points, though, it has difficulty detecting smooth or reflective surfaces.

Light Estimation

Like ARKit, ARCore takes into account the lighting of the real world and applies it to the virtual scene so that the shading of the two worlds match. This is necessary to give the virtual objects a more realistic appearance.

The session proceeds with a tutorial on getting started with ARCore, including a plug for two other tools available to Android developers.

WebXR: Based on WebVR standard, WebXR acts as a prototype for web AR standards. It is compatible with experimental versions of Chrome on Android and iOS.

Blocks: Google also offers Blocks, a VR app for building 3D content in VR via the Oculus Rift or HTC Vive. Blocks also offers a library of "remixable" 3D imagery that developers can access.