Augmented reality () is suddenly the buzzword on everyone’s tongue. Thanks to , developers can easily augment the physical world with digital content.

Although the launch of ARKit and ARCore have helped raise the awareness of augmented reality, many other platforms and SDKs have been on the go for a while now. One of these is Vuforia; and recently, Unity partnered with Vuforia and integrated the AR platform into the Unity engine.

In this tutorial, you’ll make an AR game using the Vuforia SDK that comes integrated with the Unity game engine. Along the way, you’ll learn the key components of:

  • Setting up an augmented camera feed
  • Image tracking — what it is and how it works
  • Attaching your digital content to a physical object
  • Creating custom interactions triggered by what the camera sees
  • What topping goes best on a pizza :]
Note: This Vuforia tutorial assumes you know the basics of Unity development. If you’re new to Unity development, check out this great Getting Started In Unity tutorial.

Developing AR Games in Unity

There are many AR headsets on the market, but the primary target of AR apps is within the mobile space, like iOS and Android. This is where a combination of Unity and Vuforia is convenient — cross-platform development. Another perk is that you can test your app in the editor. To do this, all you need is a webcam! :]

Before getting started, download the latest version of Unity and install the Vuforia Augmented Reality Support component. If you’re using the Unity Hub, you can add this component to an already installed version of Unity.

Getting Started

Now that you have Unity, Vuforia and a webcam, you need to download the sample project. You can do that by clicking one of the Download Materials buttons at the top or bottom of this tutorial.

Once downloaded, extract the files and open the How to Make an AR Game Using Vuforia Starter project in Unity. With the project loaded, open the Starter scene in the Scenes folder, and look at the Hierarchy window:

After a quick review of what’s there, hit the Play button in the editor and get ready to fulfill some orders in this pizza-related game.

Cool, that was some mean pizza play right there, but the real challenge is to turn this into an AR game.

Note: The UI assets for this tutorial were downloaded from

Introducing the Sous Chef, Vuforia

To do this, you will make the virtual pizza appear on a tracker image that you can hold in the real world. Once you have added the toppings for the order, you will physically “serve” the pizza by moving it out of view of the camera :]

First things first, you need to swap the Main Camera for an AR Camera. Delete the Main Camera from the scene, then right click in the Hierarchy.

Next, add Vuforia -> AR Camera. When you do, you’ll get this pop-up:

Click Import and wait for the Vuforia Package to import into the project, which adds quite a few files.

You can ignore the new files in the Editor and Streaming Assets folders. Those files are just template files; the ones you need were included with the project.

The Vuforia folder holds all of the prefabs and scripts the Vuforia platform uses, which you’re about to use.

The last file worth mentioning is VuforiaConfiguration, located inside the Resources folder. This file is included because you need a license to develop a Vuforia application. However, if you’d like, you can obtain a free developer license using the Vuforia Developer Portal.

Note: If the App License Key is missing from your project, you can find the key in the README file. Copy the string into the inspector field of the VuforiaConfiguration file.

Now that you have the AR Camera in the scene, there’s just one more thing you need to do. Turn on Augmented Reality!

Go to Edit -> Project Settings -> Player. Scroll down to the bottom where the XR Settings are located, and verify the Vuforia Augmented Reality Supported checkbox is checked.

Push Play in the editor, and say hello to yourself!

Note: If you hit play and are faced with a “Vuforia Initialization Failed” screen, try restarting Unity. This is a random bug that only appears on certain systems.

Introduction to Image Recognition

I’m sure you look lovely, but what happened to the pizza game?

The default way Vuforia works is with Image Recognition. Image Recognition, also called Tracker Recognition or Image Tracking, is the process by which the camera detects a predetermined image and knows what to do with it, such as rendering some content on top of it. This works best when the tracking image matches the content in some way, for example, floor-plans to render a building might work well on top of a picture of a building.

How it works: It’s critical to select a good tracking image. When using Vuforia, you can upload your chosen image to the developer portal to see how well it’ll track, which is something you should do before you start development. The developer portal gives your image a rating, but more importantly, it’ll show you the “feature points”. For a good tracker, these feature points should be densely spread around the image with no repeating patterns. At runtime, the camera looks for these feature points to help calculate where to position itself in relation to the image.

For this tutorial, an image and a tracking database have already been set up. The pizza image file is in the Materials folder you downloaded earlier. Ideally, you should print this out. Alternatively, you can put it on some digital device that you can then “show” to the camera. Here’s what the image looks like in the Vuforia Tracker Database:

As you can see, there is a good distribution of feature points. Also, it’s a pizza! That’s handy. :]

Adding Image Targets to the Scene

It’s time to make some magic! You’re going to make the virtual pizza appear on top of the image of this pizza. Using the Create menu in the Hierarchy again, choose Vuforia -> Image.

Now you’ll have an Image Target GameObject in the scene. Look at this target in the inspector and you’ll see a number of components. The key ones are Image Target Behavior and Default Trackable Event Handler. You’ll dive into the second one later, but for now, make sure the drop-down options of Image Target Behavior are set up as below:

  • The Database is set to RW_ItsaPizza
  • The Image Target is set to PizzaClipArt
  • Remember the files that were added to the project earlier that I said you could ignore? You’ve found them! Vuforia adds a bunch of sample Image Target databases for you to begin developing and using their samples with — such as the one that you’ll be using, already included in the materials.

    Note: The Pizza tracker is the only image target in your database; however, you can have hundreds of images in a single database. Plus, you can have multiple databases in a single app.

    Now that you have an AR Camera and an Image Tracker set up in the scene, you have everything you need to make augmented reality work! Press the Play button in the editor and hold your printed image up to the camera.

    Somewhere, there’s a pizza!

    Attaching Game Objects as Children to the Trackers

    That pizza is a bit small, but it’s stuck to the tracker image. You may also notice that if you take the tracking image away, the pizza is left floating there.

    What’s happening here is while your webcam can see the tracker image, Vuforia can update the position of the AR Camera in the scene. If you want to see this in action, set up your Unity Editor in a way that you can see both the Game and Scene views, and have the AR Camera selected when you hit play.

    So, how do you get the pizza to behave while the camera is working? Select the ImageTarget in the Hierarchy. You can see that its scale is set to 10 in each axis. The Image Target Behaviour component controls this. In the advanced section, you can see the Width parameter set to 10. This was defined when the image was uploaded to the Vuforia Developer site.

    Tip: Are you wondering why the Image Target for this tutorial has been given the width of 10? When the size isn’t an essential factor for your app, setting the size to 10 gives the camera a better chance of tracking while maintaining a good position for your content in between the default near and far planes of the AR Camera. However, there may be times when you need your augmentation to be at a certain/real-world scale. In this case, you would set the size of the image tracker to match its physical dimensions when uploading it to the developer portal.

    Now, select the Pizza GameObject in the Hierarchy. Drag it onto the ImageTarget to make it a child. Its scale will have changed to (X: 0.1, Y: 0.1, Z: 0.1). Increase this back to (X: 1, Y: 1, Z: 1) and also nudge its Position up to 0.01 in the Y axis. This makes the Pizza fit nicely onto the ImageTarget.

    Hit Play again, and you’ll see that the pizza fits nicely on top of the image, but it also disappears if you hide the image.

    Exploring the DefaultTrackableEventHandler

    This behavior comes from the DefaultTrackableEventHandler on the ImageTarget. Open the script and look it over.

    The script is already well commented, but take note of a few things:

  • The Start function registers this script as the Event Handler for the TrackableBehaviour (in this case, the ImageTargetBehaviour).
  • OnDestroy removes that reference.
  • OnTrackableStateChanged is the key function. The code in here dictates what should be done when tracking states change.
  • OnTrackingFound and OnTrackingLost are called from OnTrackableStateChanged. In DefaultTrackableEventHandler, they toggle the Renderer, Collider and Canvas components on any child object.
  • When the camera detects the tracking image, it isn’t just moving the AR Camera anymore; it also tells the Pizza GameObject to turn on all of its Renderer components, and when the image goes out of view of the camera, it tells it to turn them back off again. :]

    Digest that information for a moment.

    Making Your own Tracking Actions

    It’s time to do something with this information!

    Remove the DefaultTrackableEventHandler component from the ImageTarget. Then, add the PizzaTrackableEventHandler which you’ll find in the Scripts folder. Next, open the PizzaTrackableEventHandler. This a clone of the DefaultTrackableEventHandler, but the code in OnTrackingFound and OnTrackingLost has been removed – it’s up to you to solve this!

    The turning on and off of Renderer components makes sense in almost every AR application, so add that code back in. If you get stuck, you can copy it from DefaultTrackableEventHandler or find it in the spoiler block below.


    protected virtual void OnTrackingFound()
        var rendererComponents = GetComponentsInChildren<Renderer>(true);
        // Enable rendering:
        foreach (var component in rendererComponents)
            component.enabled = true;
    protected virtual void OnTrackingLost()
        var rendererComponents = GetComponentsInChildren<Renderer>(true);
        // Enable rendering:
        foreach (var component in rendererComponents)
            component.enabled = false;


    You’re ready to make this game an augmented reality game!

    Look at the UI and locate the button the player needs to click to complete their pizza.

    You can see there’s an Event being called when the player clicks the button, GameManager.CompleteOrder(). Instead of making the player tap a button, you can make them have to “hand” the pizza over (move the tracker out of view) to complete the order.

    At the start of PizzaTrackableEventHandler, add a UnityEvent to call when the Image loses tracking.

    using Vuforia;
    using UnityEngine;
    using UnityEngine.Events;
    public class PizzaTrackableEventHandler : MonoBehaviour, ITrackableEventHandler
        public UnityEvent OnTrackingLostEvent;

    And now, invoke this event in the OnTrackingLost method:

    protected virtual void OnTrackingLost()
        var rendererComponents = GetComponentsInChildren<Renderer>(true);
        // Enable rendering:
        foreach (var component in rendererComponents)
            component.enabled = false;
        //Trigger our event

    This makes the PizzaTrackableEventHandler flexible because now you can set whatever actions you’d like to have happen when an image loses tracking. Save the PizzaTrackableEventHandler, go back into the Unity Editor and wait for it to compile. Once it has, set the GameManager.CompleteOrder() to be invoked when the pizza tracker image loses tracking. Finally, turn off or delete the CompleteOrderButton in the UI.

    Save the scene, hit Play, and serve some pizza!

    Where to Go From Here

    Congratulations on finishing the tutorial!
    You can download the sample project by clicking one of the Download Materials buttons at the top or bottom of this tutorial.

    For more info on developing with Vuforia in Unity, you can check out the Vuforia Developer Library, or the Unity manual.

    Don’t forget to set up a developer account for yourself in the Vuforia portal. Why not try uploading a few different image trackers to see how well each one tracks?

    For an advanced topic, look into virtual buttons. There’s some sample material on the Vuforia website to get you started. See if you can you make virtual buttons to replace the UI in the pizza game.

    What other augmented reality game ideas can you come up with? A card battle game where creatures jump out of their cards? How about a tower defense game where you’re controlling an attack helicopter through the phone? Whatever you think up, let us know in the comments below! :]

    Source link


    Please enter your comment!
    Please enter your name here