Apple TV e a Transformação de Web Apps em aplicativos "Dual Screen" para Tablets e Smart TV's
Earlier this year we announced and demonstrated a new capability within our App Cloud platform for creating Dual Screen Apps for Apple TV. This got a lot of attention and opened up peopleâs minds to the way in which our lives are being transformed with tablets and TVs. If youâre not familiar with the concept or what we were announcing, embedded here is a demo of the capability.
A lot of the focus was on how people might provide apps that support watching video content, which is a natural use case. In our minds, however, the opportunity for dual screen apps is a much larger opportunity to transform how content and applications are experienced, whether in the home living room, the office meeting room, the classroom, the retail store, the hospital, and really any other context where people are interacting around content and information and where that information would benefit from rendering and display on a large screen such as a TV monitor.
To better understand this concept, itâs necessary to step back and reconsider the nature of how we write software and the user experience model for software.
Today, the predominant user experience model for software and applications online is a single screen. We browse to Web applications on a desktop PC, mobile browser or tablet browser and interact with and consume content and applications on that screen. It is very much a single, individual user task. Likewise, we install apps onto these devices and consume and interact with information, perform tasks, make purchases, etc. through these apps. Again, this is a solitary single individual task.
As a result, when software creators plan their applications, they are typically designed and developed with this single user, single screen concept in mind.
Dual screen apps change all of that by shifting the software and user experience model from one user to potentially many, and from one screen (PC/phone/tablet) to two screens (phone/tablet AND TV monitor). From a software development and UX perspective, the large monitor (which is the true 2nd screen, vs. the standard concept that puts the tablet as the 2nd screen) becomes an open computing surface where one can render any form of application functionality, information, data and content.
Importantly, designers and developers need to shed the concept that âTVsâ are for rendering video, and instead think about âTVsâ as large monitors on which they can render applications, content and interactivity that is supported by a touch-based tablet application.
While we have the greatest affinity for large monitors as fixtures of the living room, increasingly flat-screen monitors are a becoming a ubiquitous part of our social fabric. In fact, large monitors often sit at the center of any social setting. In the home, these large monitors provide a social surface for those sharing the living room space. Increasingly, monitors are a common part of nearly every business meeting room space -- not for watching video, but for projecting shared content and business data and presentations that support business and organization collaboration. Likewise, monitors are in medical and hospital settings providing visual information to patients. They are increasingly in nearly every classroom, whether through a projector or an actual TV monitor and support the presentation of information that is needed for a collection of students. Large monitors are increasingly ubiquitous in retail settings as well.
The key concept here is that this pervasive adoption of TV monitors is the tip of the spear in creating a social computing surface in the real world. Forget about social networks that connect people across their individual, atomized computing devices, the real social world is groups of people in a shared space (living room, office, classroom, store, etc.) interacting around information and data on a shared screen.
Until very recently, the way in which these TV monitors could be leveraged was limited to connecting a PC through an external display connector to a projector or directly to a TV. The recent breakthrough that Apple has fostered and advanced more than any other computing or CE company is AirPlay and associated dual screen features in iOS and Apple TV.
Specifically, Apple has provided the backbone for dual screen apps, enabling:
Any iOS device (and OSX Mountain Lion-enabled PCs) to broadcast its screen onto a TV. Think of this as essentially a wireless HDMI output to a TV. If you havenât played with AirPlay mirroring features in iOS and Apple TV, give it a spin, itâs a really exciting development.
A set of APIs and an event model for enabling applications to become âdual screen wwareâ (e.g. to know when a device has a TV screen it can connect to, and to handle rendering information, data and content onto both the touch screen and the TV screen).
With the existing Apple TV unit sales already outselling the XBox in the most recent quarter, we can see a world that goes from approximately 5M dual-screen capable Apple TVâs to potentially 15-20M in the coming 1-2 years, and eventually to 30-50M units as new and improved versions of the Apple TV companion device come to market.
As a result, itâs an incredible time to experiment with this fundamental shift in computing, software and user experience, to embrace a world where the Tablet is the most important personal productivity device, and the TV is a rich and powerful surface for rendering content and applications.
As we rethink the TV as a computing surface for apps, itâs really helpful to have some ideas on what weâre talking about. Below are a series of hypothetical examples of what is possible today and of course what will be even bigger as these new dual screen run-times proliferate.
Imagine that youâre looking into buying a house. You open your tablet app from a reputable home listing service and perform a search using criteria that you care about and begin adding potential fits to a list of houses youâd like to explore. When you select a specific house, the app detects youâre connected to an Apple TV and launches a second screen on the TV that provides rich and large visual displays about the house -- HD quality photos and contextual information about the house. Here, the power of dual screen is the fact that you and your spouse can sit in the living room and explore a house together without crouching over a computer or tablet on someones lap, and the house can be presented with HD quality media and contextual information.
Imagine launching the BMW app on your tablet and deciding to both learn about car models and configure a car. Like buying a house, often a âsocialâ decision between partners. On the TV, the app renders a high quality rendition of the car. As you explore the carâs features from your tablet, associated media (photos, video and contextual meta-data) render onto the large TV in front of you. As you configure your car using your tablet, it updates a visual build of the car on the large screen -- not sure what a specific feature of the car provides, it provides an inline HD video onto the big screen.
Looking to introduce your 3-year old to key cognitive development concepts? Launch a learning app where the child interacts with the tablet application and sees visual information, animation and other content on the TV screen. Their touches on the tablet instantly produce rich and relevant content on the TV screen. Learning to count? Feed cookies over AirPlay to the cookie monster on the TV who eats and counts with you. Learning about concepts like near and far? Tap the table to make a character move closer and away from you. Build a character on the tablet and watch the character emerge on the TV screen.
As a sales manager, you walk into your team conference room with a TV monitor mounted on the wall. You kick open your Salesforce.com tablet app on your tablet and begin filtering and bringing up specific reports on your tablet, and with the touch of a button you push unique visual reports onto the shared surface of the conference room TV. Here, the sales manager wants control of the searches and filters they have access to and only wants to render the charts and reports that are needed for the whole team to see.
Imagine playing Monopoly with your family in the living room -- 1 or 2 or maybe even 3 touch devices present (phones, iPod Touches, iPads) -- each player has their inventory of properties and money visible on their device. The app passes control to each user as they play. On the TV screen is the Monopoly âboardâ with a dynamic visual that updates as users play -- the movement of players, the building up of properties, etc.
A teacher walks into a classroom with an Apple TV connected to a HDMI capable projector that projects onto a wall or screen. From their tablet, they pull up an application that is designed to help teach chemistry and the periodic table -- they can control what element to display up on the screen, which on the TV display provides rich information, video explanations, etc. The app is designed to provide âpublic quizâ functionality where the TV display shows a question, presumably related to material just reviewed or from homework, students raise their hand to answer and then the answer and explanation is displayed.
You are meeting with your doctor to go over test results from an MRI scan. The doctor uses his or her tablet to bring up your results, picks visuals to throw onto the TV monitor in the room, then uses his or her finger to highlight key areas and talk to you about theyâre seeing.
Youâre at a Best Buy and interested in buying a new high quality digital camera. A sales specialist approaches you with tablet in hand and asks you a few questions about what youâre interested in while tapping those choices into their tablet app. From there, it brings up on a nearby TV display a set of options of cameras -- based on further probing, they drill into a specific camera choices which brings up a rich visual with a video overview of the specific camera that youâre interested in.
A major revolution has just broke out in a nation across the planet. Time Magazine has captured incredible audio, photos and video of the events. You and your friends sit down in front of the TV to learn more. You open the Time Magazine tablet app and bring up a special digital edition about the revolution. From the tablet, you flip through and render onto the TV rich HD quality photographs, listen to first hand audio accounts (accompanied by photos) and watch footage from the events. The app renders a huge visual timeline of the events that led up to the revolution. Itâs an immersive media experience that can be easily shared by friends and family in the living room.
Last but not least, of course, Dual Screen Apps will be essential to any app that is about consuming video -- whether a news or magazine app, a vertical website (think Cars.com, BabyCenter.com, AllRecipies.com, etc.), or of course a catch-up TV app from a TV network or show that you care about. You open the app on your table to explore what to watch, and when youâre ready to watch the show instantly pops onto your TV in gorgeous HD quality, and the tablet app becomes your remote control and presents relevant contextual information about the video, episode or what have you.
The Transformation of the Web into Tablet and TV Dual Screen Apps
Virtually every application that exists on the Web and Phones and Tablets likely has a dual screen use case. Simply put, Web and app designers and developers need to imagine a world where the tablet and TV are a single run-time for their applications which each screen providing distinct value for the user controlling the app and the user consuming rich media and information on a large display. Sometimes this is just one person (like picking and watching a show or playing a game or learning something), but crucially and very often I believe that these apps will be designed with multiple users -- and a social context -- in mind.
What are we doing about this?
A few months ago, we launched App Cloud Core (learn more here), a free online service and open source SDK that empowers Web and app developers to build cross-platform native apps (aka hybrid native apps) for iOS, Android and Apple TV. App Cloud Core includes a suite of developer tools for building, testing, debugging and compiling hybrid apps in the cloud using HTML5, CSS3 and JavaScript. In the App Cloud Core SDK we include rich libraries for using native device APIs (and soon writing native code plug-ins), for constructing rich touch-centric user interfaces, and data services libraries for things like content caching, offline usage, file downloads and other key capabilities. Weâre trying to help unleash the talent and creativity of the millions of existing Web designers and developers into the market of native app development for key consumer devices. You can think of App Cloud Core as a really strong, feature rich and free alternative to Adobeâs PhoneGap and Appceleratorâs Titanium.
As part of this, weâve released a set of additional APIs that enables developers to use HTML5 to build dual screen apps that work on iOS and Apple TV. Weâve abstracted the screen detection and communications plumbing, and updated our app development model to incorporate what we call multi-view applications (see simple API doc here) which enables a developer to use an HTML5 view on the TV and provides a simple means for each screen to communicate data and events to each other. (If youâre a developer, you can also find simple reference example source code for both Dual Screen Videoand Dual Screen Web Views on our Github repo).
Specifically, as a developer, letâs say you have an App that has several views and sections that can be used on a phone or tablet. Using the new dual screen multi-view APIs, your app can detect when a user has a connection to their Apple TV and automatically render a unique view or interface onto the TV screen, enabling all of the kinds of examples illustrated in the scenarios above.
This is such a groundbreaking approach to apps and software we expect lots of others to try and emulate what Apple is doing. Already, Microsoft is promoting the ability to use its Surface Tablet in conjunction with apps built for the XBox. Samsung has introduced features in its tablets and TVs to enable easy media sharing from your tablet or phone onto a Samsung Smart TV, and surely Google will follow suit with similar features to AirPlay in the Android OS. While for now weâve made a bet on Appleâs innovation in this space, weâre deeply committed to the idea of easily and cost effectively building cross platform apps and will monitor these developments closely and support them if it makes sense for developers and end-users.
Additionally, Apple is still early in deploying this technology, and it is a little bit hidden from end-user view -- e.g. learning how to turn on AirPlay on a phone or tablet is somewhat hidden in the iOS environment. There are also bugs with how it is implemented, sometimes resulting in flaky behavior. We expect major changes as well as native code capabilities directly on the Apple TV, and weâll be surely evolving our SDKs to stay on top of any changes that do come down the pike.
However, itâs very clear to us that there is a great future in dual screen apps, and we are excited to encourage designers and developers to start exploring and digging in and experimenting with what we expect to become a fundamental new paradigm in consumer software and media.
For more developer resources on dual screen apps, check out Todd Yard's recent blog post on "Building a Dual Screen Video Application with App Cloud".
