Content app extensions in iOS 10
WWDC 2016 session "Advanced Notifications" packs a lot of material in showing how to create notification content extensions. However putting together a working example required a little more digging, so here's a summary of what you need to assemble.
Add the necessary keys to your notification payload
Two things need to be added to your normal notification payload to trigger an app content extension: the mutable-content key, and a category identifier to let the app know which notification content extension (with custom UI) to launch for the notification.
Optionally, you can specify images, audio, or video to embed in the custom UI for playback. These can be specified in the area outside of the aps dictionary in the payload in a format of your choosing.
Create a service app extension
The service extension is for processing content in the background before the user 3D touches on the notification to get the custom UI presented by the content extension.
First, create the extension through Apple's wizard by adding a new target to the project your main app resides in.
Download the content and save to local storage
In the didReceiveNotificationRequest function stub presented by the wizard, add a body to process the attachments as you like. In my case, I'm downloading the image and going to stream the music, so I only need processing for the image. The general flow is:
Set up a place to save the temporary files
Download the attachments one at a time
Create a new UNNotificationAttachment for each and add it to the modified UNNotificationContent
Some examples use NSURLSession to get the data and then save it, I'm using NSData's helper function directly to get the data synchronously off the web. The service extension will take the file you've downloaded and move it into a sandboxed area for other processes (like the content extension) to access.
A note about memory and time
iOS places significant restrictions on the resources a service extension can use. The entire operation must complete in thirty seconds or less, otherwise the serviceExtensionTimeWillExpire method will be called and processing will have to end. It's a good idea to minimize the size and complexity of content you want to move through the service extension. Memory also seems to be at a premium. Though I haven't found any documentation provided guidance, I noticed my service extension was terminated mid-process due to memory constraints whenever I went over four megabytes of RAM in Xcode. Since Apple provides media players for audio and video, I'd recommend not even messing with those here and just grabbing the media URL from the payload in the content extension.
Create a notification content app extension
The notification content extension allows custom UI and media playback when the user initiates 3D touch on the notification. Create a content extension in the same manner as the service extension, but this time adding some specific data to its Info.plist.
category for the content extension
initial aspect ratio for the height of the custom UI (0.5 is a good place to start)
UI for the custom notification is just like any other, use Interface Builder to modify the default storyboard, and add outlets to your notification view controller as needed to modify content. When the user performs a 3D touch on the notification with a corresponding UNNotificationExtensionCategory, the extension will be launched and the view controller's didReceiveNotification function will be called. Here you can process the newly assembled UNNotificationAttachment objects from yours service extension, and play streaming content.
To access file attachments iOS has moved from the the service extension, you need to request specific security access or attempts to open the file will fail.
Playing streaming audio is easy with the AVPlayer Apple provides. Since notification views do not respond to user input generally, there is a special exception for media playback, and Apple provides the UI for the control if you override a number of methods.
The delegate methods for media playback require an explicit frame for the button to appear in. This unfortunately is a bit antiquated and not in line with the spirit of autolayout and storyboards. The time that this function is requested by iOS is before layout may have finished in your views, so referencing other elements in the storyboard for their frames may not be appropriate.
The mediaPlay and mediaPause functions are called when the user toggles input on the system media player button in the UI. In this case, just hooking them up to the AVPlayer functions is sufficient.
How to debug an app extension
This took me a little bit to figure out, but since extensions aren't really apps themselves you can't run them directly from Xcode. First, set your breakpoints in your extension code, then launch an extension with the run button in Xcode. Xcode will then ask for a hosting app to start the debug session, it doesn't seem to matter which. I've had success with Compass and Today (system apps).
Then when the notification is received by iOS, you should hit the breakpoints first in your service extension, and then after a 3D touch, your content extension. At this point you'll have access to all of the notification, its body, payload, etc. in order to process and debug your data.
I've noticed that leaving the phone on a notification with custom UI maximized changes over time. Parts of the view disappear, strings return to default values, etc.
A number of cryptic messages about async failures appear in my log window.
I'm not sure if these are anything to worry about yet.
All of these may be due to the beta nature of Xcode 8 / iOS 10, or I could be doing something wrong. Have any new insight? Was something unclear? Let me know. :)
https://developer.apple.com/videos/play/wwdc2016/708/
https://developer.apple.com/library/ios/documentation/General/Conceptual/ExtensibilityPG/ExtensionCreation.html#//apple_ref/doc/uid/TP40014214-CH5-SW8
https://developer.apple.com/reference/usernotificationsui/unnotificationcontentextension
https://developer.apple.com/reference/usernotifications/unnotificationcategory/1649282-intentidentifiers?language=objc
http://www.atomicbird.com/blog/sharing-with-app-extensions
