#wwdc17

This is a brief writeup of my last hackday demo at Yahoo7 :)

At WWDC17, Apple introduced ARKit which makes it very easy to create an augmented reality experience which runs surprisingly well on Apple’s latest hardware. In this demo, there are 4 livestreams running simultaneously, along with a simple particle emitter:

When the screen is tapped, we first detect if an existing node (video) has been tapped with a hit test. If it is, we set the volume on the tapped video and mute all the others:

// If a node is tapped, set its volume and silence the rest let location = touches.first?.location(in: self.sceneView) let hits = self.sceneView.hitTest(location!, options: [SCNHitTestOption.ignoreHiddenNodes: SCNHitTestOption.ignoreChildNodes]) if let tapped = hits.first?.node { if let player = nodePlayerMap[tapped] { nodePlayerMap.forEach { (arg) in let (_, myPlayer) = arg myPlayer.volume = 0.0 } player.volume = 1.0 } return }

An AVPlayer is not a 3d object that can be placed, so we create a SpriteKit scene which is attached to the anchor. The SKScene has its geometry set, using the SKVideoNode (initialised with an AVPlayer) as a texture/material to be placed on the plane:

let videoSKNode = SKVideoNode.init(avPlayer: player); videoSKNode.size = spriteKitScene.size videoSKNode.play() videoSKNode.yScale = -1; videoSKNode.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0) spriteKitScene.addChild(videoSKNode) videoNode.geometry = SCNPlane(width: 1.280 / 2, height: 0.720 / 2) videoNode.geometry?.firstMaterial?.diffuse.contents = spriteKitScene videoNode.geometry?.firstMaterial?.isDoubleSided = true

The video players are placed 1 metre in front of the camera using the same transform that the camera has. So if you face down and place the player, the screen will still be correctly facing the camera. This code is derived from the sample ARKit scene with Xcode 9:

// Create a transform with a translation of 1.5 meters in front of the camera var translation = matrix_identity_float4x4 translation.columns.3.z = -1.0 let transform = simd_mul(currentFrame.camera.transform, translation) // Add a new anchor to the session let anchor = ARAnchor(transform: transform) sceneView.session.add(anchor: anchor)