#imagerecognitionwitharfoundation

Introduction to AR Foundation:

Augmented Reality can be used in Unity through AR Foundation. This interface makes Unity developer work easy. To use this package you also need some plugins which are mentioned below.

AR Foundation includes core features from ARKit, ARCore, Magic Leap, and HoloLens, as well as unique Unity features to build robust apps that are ready to ship to internal stakeholders or on any app store. This framework enables you to take advantage of all of these features in a unified workflow.

AR Foundation lets you take currently unavailable features with you when you switch between AR platforms.

The AR-related subsystems are defined in the AR Subsystems package. These APIs are in the UnityEngine.Experimental.XR namespace, and consist of a number of Subsystems.

AR Foundation uses Monobehaviours and APIs to deal with devices that support the following concepts :

Device tracking: track the device’s position and orientation in physical space.

Plane detection: detect horizontal and vertical surfaces.

Point clouds, also known as feature points.

Anchor: an arbitrary position and orientation that the device tracks.

Light estimation: estimates for average color temperature and brightness in physical space.

Environment probe: a means for generating a cube map to represent a particular area of the physical environment.

Face tracking: detect and track human faces.

2D image tracking: detect and track 2D images.

3D object tracking: detect 3D objects.

Meshing: generate triangle meshes that correspond to the physical space.

Body tracking: 2D and 3D representations of humans recognized in physical space.

Collaborative participants: track the position and orientation of other devices in a shared AR experience.

Human segmentation and occlusion: apply distance to objects in the physical world to rendered 3D content, which achieves a realistic blending of physical and virtual objects.

Raycast: queries physical surroundings for detected planes and feature points.

Pass-through video: optimized rendering of mobile camera image onto the touch screen as the background for AR content.

Session management: manipulation of the platform-level configuration is automatically when AR Features are enabled or disabled.

(Above concepts reference are taken from: https://docs.unity3d.com/Packages/[email protected]/manual/index.html)

Feature Support per Platform:

Prerequisites

Unity 2019 version or above.

Android SDK level 24 and above.

Android device which has Android 7.0 and above.

ARCore XR plugin:

Currently, [email protected] preview is available. By adding this plugin it enables the use of ARCore support via unity multi-platform XR API Supported Features are,

Background Rendering

Horizontal Planes

Depth Data

Anchors

Hit Testing

For more current version information visit: https://docs.unity3d.com/Packages/[email protected]/manual/index.html

ARKit XR plugin:

Currently, [email protected] preview is available. By adding this plugin it enables the use of ARKit support via unity multi-platform XR API Supported Features are,

Efficient Background Rendering

Horizontal Planes

Depth Data

Anchors

Hit Testing

Face Tracking

Environment Probes

For more current version information visit: https://docs.unity3d.com/Packages/[email protected]/manual/index.html

Image Recognition Using AR Foundation:

Setup

To install AR Foundation you need to go window -> package manager, Press The Advance Button, and select show preview packages after that you will need to install AR Foundation version 2.2.0-preview.06, We also require to download the ARCore XR Plugin, ARKit XR Plugin the same way the version should be,

ARCore XR Plugin(For Android) : 2.2.0-preview.06 ARKit XR Plugin(For IOS) : 2.2.0-preview.06

Note: make sure the SDK you are using are updated beforehand. (The above-mentioned version are working for me that why I used these versions.)

Player Settings

In this player, the setting adds the company name and package name you want to give. Then enable AutoGraphic API. Then make Minimum API level to 24 and Target API level to highest.

Hierarchy

Add AR Session, AR Session Origin from pressing right-click on hierarchy go in XR tab and add them.

Then inside AR Session origin is AR camera give it the tag of the Main Camera.

Inspector

Click on AR Session origin and add AR Tracked Image Manager component.

We now need a Serialized library and image prefab.

Reference Image Library

Go in project tab Press right click Create – >XR -> ReferenceImageLibrary. This is a Serialized object, so we need to populate it.

After adding the data, give reference to the AR Tracked Image Manager component.

Prefab

Create a cube in the scene adjust it and drag it to the project tab and give reference to the AR Tracked Image Manager component.

Script

Create a script for recognition of image and what you want to do with it through AR Tracked Image Manager component.

Add the code written in the script above.

Build

Press CTRL + B to build your app, make sure you add the scene.

Now it’s completed you can open the image which you add in the reference library and you can see the prefab you added AR Tracked Image Manager component

Video

For the video click on: https://youtu.be/kYZiPnGhNsM

Conclusion:

Hence, Image Recognition is one of the features which are helpful in AR Related Applications, and hence it will be very useful in a Games.

This was just a basic setup to start working with AR Foundation in unity. There are many more features available for user-interactive gaming. Image Recognition is an example of engaging gaming experience.