Welcome to your first lesson on ARKit. Whether you're looking to build a new app centered around augmented reality or you want to add augmented reality to an existing app, you'll need to learn some additional Xcode project configurations. Once you have that challenge out of the way, you can dig into the basics of the augmented reality framework.

What You'll Learn

How to create a new augmented reality-based project using ARKit
How to add ARKit into an existing Xcode project
Understand how SceneKit is utilized alongside ARKit
How to enable different debugging options for AR apps

Vocabulary

ARConfiguration
ARKit
ARSCNView
ARSession
origin
scene
SceneKit

WWDC 2017 Introducing ARKit: Augmented Reality for iOS

Guide

Open Xcode and create a new project. To get started quickly with ARKit, you can use the new Xcode template called Augmented Reality App. This template conveniently includes the configuration files and code required for your AR projects. Select this template over the standard Single View App template that you've used in previous lessons.

On the next screen, you'll need to specify a new option, Content Technology. Your choice—SceneKit, SpriteKit, or Metal—determines which framework Xcode will use in conjunction with ARKit to add objects into your augmented world. The SpriteKit framework is focused on working with 2D assets, while SceneKit and Metal are focused on 3D. Throughout this guide, you'll only be using SceneKit. Select it now.

Name your project "ARKit-Template." Connect your physical device to your computer, then set it as the target device, rather than Simulator. Before you can proceed, you may need to update iOS on your device. Testing your app will only work if your device is running an iOS version that supports ARKit, otherwise the device cannot be selected as the target.

When your app launches for the first time, it will ask for permission to use the device's camera. Without permission, ARKit can't begin a new session. After you've confirmed access, the rear-facing camera is enabled, and you'll see a 3D ship floating in the scene as you move your device. You'll also see a bar near the bottom of the screen, where you can select the plus sign (+) on the left to reveal statistics regarding memory and performance.

Take a few minutes to experiment with how ARKit tracks your device's position and orientation in the real world. You may notice that the ship doesn't appear fixed in space as you move around. Are there certain conditions that impact the fidelity of the experience? The framework is doing a lot of heavy lifting in the background to enable AR, and it relies on having an environment rich with visual cues. Try changing your environment to observe how it affects the app.

Dissecting the Project Template

So what's the difference between this Xcode project and ones you've created in previous lessons? Before jumping into the Swift code, use the project navigator to examine the other files for your ARKit-Template app. Compare them to the files you're accustomed to seeing with the Single View App template.

When you think you've finished comparing, see if your findings match the following list of differences:

The art.scnassets catalog contains all the SceneKit assets included in an app. The first asset, ship.scn, is a SceneKit scene file, which contains details about the 3D model, including its size, colors, and positioning. Typically, you'd have a 3D artist create these files, but you can create your own inside the interface. The second asset, texture.png, is the texture that will be applied to the 3D object. For example, you can imagine this flat image being perfectly bent and glued all the way around the ship model.

In Main.storyboard, a subview has been automatically added to the view controller's view. This subview is of type ARSCNView, and it has been configured to fill the entire view controller's view. This special subclass displays the user's camera and allows for any 3D objects to be added to the scene.

To inspect the Info.plist file, select it from the project navigator, then right-click inside its contents and select Show Raw Keys/Values. There are two additional entries in this file when compared to the Single View App template. The first is a key, NSCameraUsageDescription, whose value is the text the user will see when prompted for camera access. If you deselect Show Raw Keys/Values, this key will show as "Privacy - Camera Usage Description". The second entry is the additional arkit value under the UIRequiredDeviceCapabilities key. With Show Raw Keys/Values deselected, this key shows as "Required device capabilities". This informs the App Store that your app uses ARKit, and ensures your app is only available for supported devices.

Basics of ARKit and SceneKit

Now that you understand the project structure, open ViewController.swift to see what code you'll need to get a basic AR app up and running. At the top of the file are two new import lines. Since the ViewController class needs to work with classes that are available within ARKit and SceneKit, those frameworks have been imported for you.

import UIKit
import ARKit
import SceneKit

ARSCNView

The sceneView outlet is connected to the ARSCNView that you saw in Interface Builder. It has a scene property, which is the scene to be displayed in the view. In viewDidLoad(), the property is set to the scene that includes the 3D ship. The view's showsStatisticsBoolean value is set to true, which enables the bottom bar that you saw earlier.

@IBOutlet var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()

        // Set the view's delegate
        sceneView.delegate = self

        // Show statistics such as fps and timing information
        sceneView.showsStatistics = true

        // Create a new scene
        let scene = SCNScene(named: "art.scnassets/ship.scn")!

        // Set the scene to the view
        sceneView.scene = scene
    }

ARSession and ARConfiguration

When the ARSCNView is loaded, it initializes its own session property of type ARSession, which manages motion-tracking and processes the camera image data. But before a session can do all that, you must call the run(_:) function and supply an ARConfiguration as the argument. Since you'll call run(_:) whenever the ARSCNView is displayed, it's a good idea to call this function in viewWillAppear(_:). To track your device's position, you'll need to initialize and run the ARWorldTrackingConfiguration. That way, ARKit can maintain the correct size and position of the ship.

override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        // Create a session configuration
        let configuration = ARWorldTrackingConfiguration()

        // Run the view's session
        sceneView.session.run(configuration)
    }

What about when the ARSCNView isn't displayed? If the camera view is no longer the focus, there's no reason to continue running the session. To prevent unnecessary work, you can pause() the session whenever the view is no longer displayed. By calling the function in viewWillDisappear(_:), you'll trigger a pause() when you dismiss this view controller or when you present a new one.

 override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)

        // Pause the view's session
        sceneView.session.pause()
    }

ARSCNViewDelegate

There are a few additional pieces of code inside the Augmented Reality App template. The view controller declares itself an ARSCNViewDelegate, assigns itself as the delegate of sceneView inside viewDidLoad(), and has some empty methods at the bottom of the class definition. This allows the ARSCNView to send information about events related to the scene—-when objects are added, when the session is interrupted—-to the view controller. It's as if the view controller tells the scene view, "Let me know when those events take place."

For more information on delegates, refer to Unit 4 of the App Development with Swift [LINK] curriculum.

Scene Positioning

As you move your device around, the ship uses the ARWorldTrackingConfiguration to maintain its position in 3D space. But how does the ARSCNView know where to place the ship initially? Open the ship.scn file and select the top-level node, "ship," in the Scene Graph View. (You can toggle this display on/off using the button in the lower left of the editor.)

The ship is positioned on a grid that represents the world coordinate system. Notice the set of three arrows? That's the origin of the node, which is aligned with the origin of the world — (0,0,0) in 3D space. The x-axis is represented by the red line, y-axis in green, and z-axis in blue. If you were to travel in the direction of the red, green, or blue arrow, you would increase the corresponding value. Since the ship is positioned along the z-axis in the opposite direction of the arrow, it has a negative z-value. The values themselves can be found in the Node inspector in the utilities panel. Select "shipMesh" from the Scene Graph View, and observe that the z-value is 0.8.

If you've never done 3D computing before, you'll have to get used to how 3D coordinate systems work as you complete this unit.

Would it be helpful to see the origin in your app? An ARSCNView has a debugOptionsproperty that can draw additional overlays, which might include the origin or 3D boxes around objects in the scene. Add the following line to viewDidLoad() to show the world origin.

sceneView.debugOptions = [ARSCNDebugOptions.showWorldOrigin]

Build and run the app again. Notice how the ship is positioned in the correct location compared to the origin. It may seem weird that you're initially pointed in the negative z-direction. That's because the world origin is determined by wherever you are when you launch the app. It's also helpful to note that, unlike UIKit, the x- and y-axes are laid out according to the standard Cartesian coordinate system (which you may remember from math class) with y-coordinates increasing in value as you go up.

Review Questions

What unit of measurement does ARKit use to determine sizes and distances?

Exercises

Start with a Single View App template, and adjust the project to use ARKit and SceneKit. When you run the app, the camera should be enabled and the world origin displayed.
Modify the ARKit-Template app by adjusting the position of the ship to float directly above the world origin.
Investigate the utilities panel in your ARKit-Template app to determine where the ship's texture is being applied. Modify the property so that the ship is solid orange.
Use the Xcode documentation to make a list of all the available ARSCNDebugOptions. Experiment to see which options affect the Augmented Reality App template and what they do.