ARKit 101: How to Detect & Measure Vertical Planes with ARKit 1.5

In a previous tutorial, we were able to measure horizontal surfaces such as the ground, tables, etc., all using ARKit. With ARKit 1.5, we're now able to measure vertical surfaces like walls!

In this tutorial, you'll learn how to make your augmented reality app for iPads and iPhones by using ARKit. Specifically, we'll go over how we can detect vertical planes and their dimensions.

What Will You Learn?

We'll be learning how to detect vertical planes and determine their sizes accurately using ARKit 1.5.

Minimum Requirements

• Mac running macOS 10.13.2 or later.
• Xcode 9.4 or above.
• A mobile device with iOS 11+ on an A9 or higher processor. Basically, the iPhone 6S and up, the iPad Pro (9.7-inch, 10.5-inch, or 12.9-inch; first-generation and second-generation), and the 2017 iPad or later.
• Swift 4.0. Although Swift 3.2 will work on Xcode 9.4, I strongly recommend downloading the latest Xcode to stay up to date.
• An Apple Developer account. However, it should be noted that you don't need a paid Apple Developer account. Apple allows you to deploy apps on a test device using an unpaid Apple Developer account. That said, you will need a paid Developer account in order to put your app in the App Store. (See Apple's site to see how the program works before registering for your free Apple Developer account.)

Step 1: Download the Assets You Will Need

To make it easier to follow along with this tutorial, I've created a folder with the required 2D assets and Swift file needed for the project. These files will make sure you won't get lost in this guide, so download the zipped folder containing the assets and unzip it.

Step 2: Set Up the AR Project in Xcode

If you're not sure how to do this, follow Step 2 in our post on piloting a 3D plane using hitTest to set up your AR project in Xcode. Be sure to give your project a different name, such as NextReality_Tutorial7. Make sure to do a quick test run before continuing on with the tutorial below.

Step 3: Import Assets into Your Project

In the project navigator, click on the "Assets.xcassets" folder. We'll be adding our 2D images here. Then, right-click on the left pane of the area in the right side of the project navigator. Choose "Import" and add the "overlay_grid.png" file from the unzipped Assets folder.

Next, once again in the project navigator, right-click on the yellow folder for "NextReality_Tutorial7" (or whatever you named your project). Choose the "Add Files to 'NextReality_Tutorial7'" option.

Navigate to the unzipped "Assets" folder, and choose the "Grid.swift" file. Be sure to check "Copy items if needed" and leave everything else as is. Then, click on "Add."

"Grid.swift" should then be added to your project, and your project navigator should look something like this:

This file will help render an image of a grid for every vertical plane ARKit detects.

Step 4: Place a Grid to Show Detected Vertical Planes

To quickly go over ARKit's plane detection capabilities, take a quick look at our tutorial on horizontal plane detection. Although this covers horizontal plane detection, the strategies and logic to detect vertical planes are quite similar.

Open the "ViewController.swift" class by double-clicking it. If you want to follow along with the final Step 4 code, just open that link to see it on GitHub.

In the "ViewController.swift" file, modify the scene creation line in the viewDidLoad() method. Change it from:

let scene = SCNScene(named: "art.scnassets/ship.scn")!

To the following, which ensures we are not creating a scene with the default ship model:

let scene = SCNScene()

Next, find this line at the top of the file:

@IBOutlet var sceneView: ARSCNView!

Under that line, add this line to create an array of "Grid's" for all vertical planes detected:

var grids = [Grid]()

Copy and paste the following two methods as listed below to the end of the file before the last curly bracket ( } ) in the file. These methods will allow us to add our Grid on the vertical planes detected by ARKit as a visual indicator.

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    guard let planeAnchor = anchor as? ARPlaneAnchor, planeAnchor.alignment == .vertical else { return }
    let grid = Grid(anchor: planeAnchor)
    self.grids.append(grid)
    node.addChildNode(grid)
}

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
    guard let planeAnchor = anchor as? ARPlaneAnchor, planeAnchor.alignment == .vertical else { return }
    let grid = self.grids.filter { grid in
        return grid.anchor.identifier == planeAnchor.identifier
        }.first

    guard let foundGrid = grid else {
        return
    }

    foundGrid.update(anchor: planeAnchor)
}

Let's quickly go over what's happening in these two methods:

1. The didAdd() is called whenever a new node is added to the ARSCNView. Here, we ensure the detected ARPlaneAnchor corresponds to a vertical plane and add it as our Grid object, which adds the grid image we imported to any plane detected.
2. The didUpdate() is called whenever newer ARPlaneAnchor nodes are detected (again, we ensure that they correspond to vertical planes), or when the plane is expanded. In the latter case, we want to update and expand our grid as well. We do that here by calling update() on that specific Grid.

Now, let's enable feature points. Under this line in viewDidLoad():

sceneView.showsStatistics = true

Add:

sceneView.debugOptions = ARSCNDebugOptions.showFeaturePoints

Next, let's turn on vertical plane detection. Under this line in viewWillAppear():

let configuration = ARWorldTrackingConfiguration()

Add:

configuration.planeDetection = .vertical

This part is very important! It will ensure that ARKit is able to detect vertical planes in the real world. The feature points will allow us to see all the 3D points that ARKit is able to detect.

Now, run your app on your phone and walk around. Focus on a well-lit wall or flat, vertical surface. You should be able to see blue grids appear whenever a vertical plane is detected, similar to the image below.

Checkpoint: Your entire project at the conclusion of this step should look like the final Step 4 code on my GitHub.

Step 5: Measure the Detected Vertical Planes

Now, we'll use the "extent" property of the detected planes in order to acquire its dimensions and print them out with the help of SceneKit. All of the code changes below will be done on the "Grid.swift" file. Be sure to open that and follow the steps below. Feel free to follow along with the finished version as you go through these steps.

Open the "Grid.swift" class and under this line:

var planeGeometry: SCNPlane!

Add a new geometry for the text that will display the dimensions of the detected planes:

var textGeometry: SCNText!

Next, find the line that imports ARKit:

import ARKit

Under this line, and above the line that creates the Grid class, add this code:

extension ARPlaneAnchor {
    // Inches
    var width: Float { return self.extent.x * 39.3701}
    var length: Float { return self.extent.z * 39.3701}
}

Here, we're extending the functionality of ARPlaneAnchor. By default, the size is returned in meters. Therefore, we're going to create two properties: "width" and "length," which converts the meters to inches. We'll also use the "extent" property of the ARPlaneAnchor and get its "x" and "z" value.

Next, find the setup() method. Delete this method and replace it with an updated version of the method as shown below:

private func setup() {
    planeGeometry = SCNPlane(width: CGFloat(anchor.width), height: CGFloat(anchor.length))

    let material = SCNMaterial()
    material.diffuse.contents = UIImage(named:"overlay_grid.png")

    planeGeometry.materials = [material]
    let planeNode = SCNNode(geometry: self.planeGeometry)
    planeNode.physicsBody = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: planeGeometry, options: nil))
    planeNode.physicsBody?.categoryBitMask = 2

    planeNode.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z);
    planeNode.transform = SCNMatrix4MakeRotation(Float(-Double.pi / 2.0), 1.0, 0.0, 0.0);

    // 1.
    let textNodeMaterial = SCNMaterial()
    textNodeMaterial.diffuse.contents = UIColor.black

    // Set up text geometry
    textGeometry = SCNText(string: String(format: "%.1f\"", anchor.width) + " x " + String(format: "%.1f\"", anchor.length), extrusionDepth: 1)
    textGeometry.font = UIFont.systemFont(ofSize: 10)
    textGeometry.materials = [textNodeMaterial]

    // Integrate text node with text geometry
    // 2.
    let textNode = SCNNode(geometry: textGeometry)
    textNode.name = "textNode"
    textNode.position = SCNVector3Make(anchor.center.x, 0, anchor.center.z);
    textNode.transform = SCNMatrix4MakeRotation(Float(-Double.pi / 2.0), 1.0, 0.0, 0.0);
    textNode.scale = SCNVector3Make(0.005, 0.005, 0.005)

    addChildNode(textNode)
    addChildNode(planeNode)
}

The extra code added to the setup() method allows 3D text displaying the dimensions of the plane to be added on top of the grid. So now, let's run down what we've set up here.

1. We set up a geometry for the text using the SCNText class. We then set the text to be the plane's width and height, calculated using our custom extension.

2. We've created a new SCNNode with the SCNText geometry from the above step and updated its position and scale appropriately. We also made sure to also rotate the text node in the same direction the grid is updated by modifying its transform property.

This allows us to show the size of the plane when it is first set up. Additionally, we should keep updating the size of the plane as it keeps getting larger. Let's do that in the next code changes.

Find the update(anchor: ARPlaneAnchor) method. At the end of the method, before its last curly bracket (}), add the following code:

if let textGeometry = self.childNode(withName: "textNode", recursively: true)?.geometry as? SCNText {
    // Update text to new size
    textGeometry.string = String(format: "%.1f\"", anchor.width) + " x " + String(format: "%.1f\"", anchor.length)
}

This will make sure that the dimensions of the plane are modified dynamically as the plane expands.

Now, save and run the app. Walk around and find a well-lit, textured vertical flat surface such as a wall. Note: For a wall, you might have to get up really close, as most white walls lack the texture that ARKit needs to detect vertical planes. I would suggest finding a window or another colored, vertical surface to allow ARKit to detect the planes. You should see something like this:

Checkpoint: Your entire project at the conclusion of this step should look like the final Step 5 code on my GitHub.

What We've Accomplished

Great job! Assuming you followed the instructions correctly, you are now able to successfully detect walls or other vertical flat surfaces using ARKit 1.5! Before the advent of vertical plane detection, it was very complicated — almost impossible — to detect vertical planes accurately. It's amazing to see how ARKit has allowed us to do this without any extra hardware needed.

If you need the full code for this project, you can find it in my GitHub repo. I hope you enjoyed this tutorial on ARKit. If you have any comments or feedback, please feel free to leave it in the comments section. Happy coding!

• Follow Next Reality on Facebook, Twitter, Instagram, and YouTube
• Sign up for our new Next Reality newsletter
• Follow WonderHowTo on Facebook, Twitter, Pinterest, and Google+

Cover image & screenshots by Ambuj Punn/Next Reality