What Apple showed at WWDC expands computing in new ways
Think of this as the tl;dr trip report I would bring back for the team, after using the software for a week.
A version of this essay was originally published on Medium.
Many of us have been using the dev builds of iOS 11 and MacOS High Sierra this week. I wanted to share some thoughts on what I think are some of the important advances.
I have attended WWDC for many years, sometimes as a partner (working on Office), sometimes as a competitor (working on Windows), and sometimes just as an interested developer (grad school). There are always a range of emotions coming out of the event. In this era of massively global Apple, every event is galactic in scale, yet it is super important to keep in mind that this is still their developer conference.
My view is that iOS is the healthiest developer ecosystem right now. For sure, a big part is because the most revenue is accruing directly to developers, but just as important is the reality that the most valued targets for advertising and commerce are also using Apple devices. Together, these create a vibrant and lively community of developers actively working on updates and new apps. An even more important reason for me is the unprecedented scale of customers and ability for Apple to deliver new software APIs to developers that will make it to hundreds of millions of devices in short order.
While the ecosystem might be the most robust right now, we also know how quickly the landscape can change. That is why the importance of how Apple evolves software that takes advantage of their market position is so critical. I believe this WWDC had some incredibly interesting developments in this regard.
I won’t cover everything, of course, but I wanted to point out what I think are the most interesting innovations or opportunities for developers engaged in the platform. Think of this as the trip report I would bring back for the team, after using the software for a week.
Think of this as a conference report, not a review or coverage of the entire event — what rose to some level of strategic importance. Opinions are mine, and I of course have no insider or unannounced roadmap knowledge of Apple (or any other platform company mentioned).
The tl;dr version of this post would be:
- iOS 11: This is where there are significant advances in the kinds of apps that can be built for mobile devices. This comes from two areas. First, the changes to core user-interaction models for iOS bring what many believe are important features for “productivity” to the iPad (now iPad Pro). Second, new frameworks, but particularly for augmented reality, are moving well beyond incremental and addressing needs in building today’s apps. Both ARKit and Core ML are likely more interesting than I think many believe —democratizing two key aspects of computer science that will clearly differentiate platforms going forward. Privacy remains an extremely core theme to how the product is evolving and how Apple thinks about the way apps are implemented.
- macOS High Sierra: Apple probably surprised people with some of the “features” in High Sierra, but what should impress people more are the architecture and infrastructure advances. Apple has cleverly engineered work for iOS to “trickle down” to macOS, which picks up the APFS file system and video pipeline work. These are a big deal for iOS, but frankly an even bigger deal for macO,S where it is much more difficult to deliver these reliably, given the openness of the platform. There are a good share of advances in High Sierra that benefit developers as users (developers also includes creative professionals more broadly).
- Hardware Products: Delivering on “Pro” with the iPad Pro and iMac Pro is significant. The iMac answers the need for a high-end workstation, and does so with arguably the most powerful device available, yet in a sleek all-in-one form factor. The iPad Pro finally (almost?!) puts the iPad in a position to be a laptop for the masses, especially those who grew up only on phones. There’s a new holiday gift for everyone with the new home speaker HomePod, but in introducing this product, Apple can take this category (if it is one) in a different direction, like it did with home media players when Apple TV was introduced.
macOS High Sierra
From the first words of the macOS section, I was intrigued. I was intrigued not by the boldness, but by the expectation setting of what would follow — a focus on perfecting the platform. I was personally reminded the challenges of getting fans excited while also trying to achieve some level of excitement when unveiling a product release viewed as refinement.
Apple doesn’t do anything by accident, but I do think they significantly undersold not just the results but their own execution capabilities in discussing High Sierra. Part of this comes from the always difficult task of articulating innovation in an OS to a non-technical or business audience when an OS can (and should) fade to the background and let apps do the work. Part of this comes from the reality that the innovations were, as Craig Federighi said, “deep technologies for the future”.
Apple File System: AFPS was not the first thing to be discussed, but it is the first thing worth noting, simply because it puts the notion of “perfecting” in context. APFS is an entirely new file system enabling such features as clones, snapshots, encryption, 64-bit limits on file counts and sizes, crash protection, crazy performance for large file operations and more. It is really hard for me to overstate the incredible nature of delivering a file system.
I’ve lived through all the Apple migrations and all the DOS/Windows migrations, and not only is this among the most feature-rich releases, it is actually running right now on my Mac (and iPhone) after an in-place upgrade. I seriously sat there watching the install process thinking, “This is going to take like a day to finish, and it will probably fail and roll back in the middle or something.” After about 30 minutes, the whole thing was complete. The amount of amazing engineering that went into both this creation and deployment of APFS is mind-blowing. And that it was done on phones, watches and PCs is nothing short of spectacular, and except for maybe the transition from FAT to FAT32, I can’t recall anything even close to this. There are a ton of features under the covers that will surface in use of Apple devices, but mostly it will just make everything better seamlessly.
Safari: Most everyone in SV and most tech enthusiasts use Chrome on the Mac. There are some old memories of a slower, less compatible browser with “fewer features” (read, extensions) that keep that audience off Safari. I switched and ran into zero problems. The further refinement of features such as intelligent tracking protection and overall what was referred to as “serene browsing” are more worthwhile to me (and probably most people) than extensions in Chrome.
There’s some jealousy in Apple’s ability to pull off anti-tracking measures, as “Do Not Track” was a huge failure for me when I was working on Windows and IE. Apple’s anti-tracking is really about accountability — if you don’t visit a site, then it is likely the site should not be tracking you. It is a first step. Apple continues not pushing the browser as a platform, but pushing the browser as an application. This becomes very interesting, because the innovation in the browser platform might be past peak for now, and thus the developers out there looking to have differentiated (versus required) web experiences is minimal (perhaps except for Google). When a platform is relatively static, it becomes more important to compete on the quality of an app experience than on the differentiated use of new and incremental platform features.
Video pipeline: Improving the video pipeline with H.265, HEVC, is another one of those things that is really hard to demo and take note of today but over time, this is a critical innovation. For video in and out, hardware support is critical, and Apple has done a great job here. Expect other ecosystems to have variations in support across devices. The challenge, of course, is that this will become a bit of a standards battle between H.265 and Google’s open, royalty-free VP9. We’ve all seen this movie before. Because YouTube commands such a large amount of video usage, this will not get resolved simply or quickly, and one should expect some “wedge” applications or uses.
Thunderbolt Dev Kit enclosure: In a world where devs that need (really need) discrete graphics for rendering or significantly driving external monitors, this is both a welcome and important advance. This is the kind of thing that brings both excitement and some frustration to the dev community. It is exciting because this solves a real problem with the Mac line, albeit with a really big dongle. It is potentially frustrating because it isn’t clear (yet) just how broad and cheap the support for external graphics this will be — how many graphics cards, how much futzing, etc. The eGPU space has seen lots of attention, and at the same time the forums are filled with tips and tricks to get things working. Generally, as much as there is demand from a community for this, the ability to drive heterogenous graphics cards or live-swap cards from one kernel has always been a bit of a stunt. And. of course. the implications of arbitrary drivers to core system security are significant.
Other OS features: There were a slew of features in the OS at the “app layer,” as well, many or most of which are shared with iOS. As we’ve come to expect from a modern OS, there’s yet another revamping of notifications and control center stuff. Frankly, this is driven quite a bit by synergy with iOS, but because of the different expressions, one mostly had a “moved my cheese” feeling on a laptop. This all falls clearly under the “refined” moniker. Historically, Apple has been so minimal at moving around OS things, much to the disappointment of fans; interestingly, things seem to be moving more each release these days.
iMac Pro: The announcement of the iMac Pro is significant, and also shows the Apple approach as we’ve come to expect. This is one beast of a PC — really a workstation —and for $5000 it should be (maxes out at 18 cores, 128GB, 4TB SSD, 10Gbs Ethernet, 4x Thunderbolt, 2 x 5K display capable, etc).
That said, one would be hard-pressed to build a competitive device on Linux or Windows for significantly less, though expect many apples-to-oranges comparisons (sorry for the pun) that allow for fans of each platform to claim price/performance superiority. When you consider the new file system, new graphics APIs, new video pipeline and more, this is likely going to be the most powerful workstation available that at the same time provides the “out of box” quality and reliability that one would expect from Apple. Oh, and it has an SD card slot, which honestly is nice for most hobbyists, but mostly puzzling.
Some might believe that by packaging the device in a “sealed” and/or all-in-one form factor, it will be perennially behind the curve. In practice, this market is served by the slow-to-change “workstations” from major vendors and by DiY gamers. The former are incredibly well-served by this PC, and the latter will continue to build their own, often challenged by compatibility and variations in driver/systems support.
I’ve been running iOS 11 all week on both phone and tablet. Of course, it is not ready at all for daily use (not unexpected), and you can’t really see the benefits of the platform just yet, and many of the usage features aren’t really there yet. That said, the key takeaway for me is that there are some significant “market expanding” elements of the release that are deeper than just adding more frameworks.
From a developer perspective, the capabilities in this release of iOS, combined with the broadening device ecosystem (Watch and HomePod), are furthering the gap between iOS and Android. This is introducing important choices for developers. I continue to believe that “winners” in categories will integrate and exploit native platforms, and viewing every advance through a cross-platform lens is an innovation disadvantage. It is always important to remember that few customers own multiple platforms, so being consistent across platforms solves your problem but not customer problems.
ARKit: One measure of the potential for a new framework is how fast new applications show up that show off the capabilities. In the case of the new ARKit (augmented reality), demos showed up before the morning sessions the next day (The Verge even catalogs some, and @a16z’s @kylebrussell has this repo). The first thoughts about AR tend to gravitate to gaming, especially with that being such a big category.
My view is that ARKit will become closely connected with business scenarios across commerce, training, assistance, trials, advertising and more. It’s not hard to imagine a brand creating some “product placement” experience that goes viral. While “glasses” AR platforms are busy trying to recreate a phone and phone-software platform that either fits in glasses or can be easily tethered, Apple already has a device and platform at massive scale. Frankly, it is not hard to imagine a peripheral that takes advantage of this and brings hands-free. Or maybe not, since this capability will be so rich and accessible. With this release of ARKit, Apple will start the flywheel of API->Apps->Feedback->Features that is the hallmark of new and differentiated capabilities.
Core ML: Much continues to be said about Apple missing out on or even structurally incapable of taking advantage of machine learning (ML). Is Apple inherently a client/device company? Is Apple’s lack of a scale services platform the problem? Does privacy hold Apple back? My view has always been that there is a significant amount of cross-customer learning/data that is required for the broadest and most horizontal features — it is impossible to implement maps and the full range of routes without using a lot of data. That said, Apple’s aim is not to enable more broad horizontal services with the newly introduced Core ML, but to enable ISVs to build on the data they have/collect to deliver useful features and to then offload some of those uses to the device.
On the one hand, this is simply a way of using the computation on the device to deliver faster response; on the other hand, it is a way of providing features that might minimize data collection on device and thus improve privacy. The main parts of Core ML are vision tracking, natural language and models. Vision permits tracking, face and object tracking, text detection and so on — note the connection to ARKit as this type of layering is something Apple does so well. NL supports a bunch of features that greatly simplify what was previously rule-based and algorithmic work. Models provide access for using either existing models for basic classification (for images, for example) or to run your own models. Again, these features further the gap between device OS platforms (not just features, but how apps are structured) while significantly advancing the state of the art. Many will debate the merits of some of these features on the client, models in particular, but I believe that there are far more uses for scenario specific and vertical learning that can and should be done on the client, without shuttling data to services.
Tablet experience: The fun part of a developer conference is always the part where end-user features are shown, since everyone can relate, including the non-developers watching. The advances in launching, switching and working in general geared toward the iPad user are significant — so significant that there’s a good chance we are at a turning point where many more people will use (or just admit to using) their iPads for core productivity work.
Yeah, I’ve been saying this for years — that ARM-based mobile OS, with new apps geared to a new interaction model, will become dominant. I didn’t expect that to be just a pocket-sized device, but based on hours of usage, that is clearly the case (and at least partially responsible for iPad sales curves). What I believe Apple has cleverly done is introduce features such as “windowing,” drag-and-drop and app switching that will cause the industry to take note of the improved productivity potential while at the same time not forcing a “desktop” model on “everyone.” By and large, these features are likely to fall to power users, but that is often how markets tilt. The new Files app (which is very early) will prove to be a game-changer, and so clearly ups the “power” of the device as many core productivity scenarios are about juggling multiple files in some workflow. For the vast majority of people that define productivity as “Office” scenarios of notetaking, slides, lists, basic models, communicating (iOS was already the preferred mail platform by volume), and so on, the iPad, with its security, reliability, robustness, performance and also connection to phone (continuity, Messages, etc.) make for an extremely productive experience. Developers take note, as iPad-specific apps will become increasingly important in productivity categories.
One of many demos of iOS 11 files app (this one by Jake Chasan)
Messages: For developers, Messages has made apps more visible and easier to get to (and in the beta really difficult to remove without crashing). Messages is almost certainly the most-used app in U.S. (and potentially European) markets, but in those markets, the idea of apps in messages has not caught on. There’s an interesting opportunity global companies have, which is when adding features for one market (say China) can turn those seemingly “odd” features into global use cases. In this case, it is not clear if the “rest of the world” will follow WeChat to a messaging-centric world, or if there will be a whole generational upheaval to the mobile platform, the way we saw the uniqueness of Japan’s mobile infrastructure upended by smartphones rather than the adoption of that approach. Building an app for Messages is an odd investment right now, though of course the way Apple scales the platform makes this incrementally small, relative to building an app (as it does with Watch and Tablet support).
SiriKit: SiriKit was introduced last year, and for the most part it is the ability to add the rough equivalent of Alexa Skills to Siri. Perhaps because of the overall reputation for Siri, or maybe because of the excitement around Alexa, this hasn’t seen the noticeable takeup one might expect. It could also be that voice control is still too much of a gimmick, especially when your phone is attached to you. iOS 11 expands SiriKit with more domains (built-in apps of notes and lists) and support for payments, QR codes and ride booking.
More UI changes (camera, photos, CarPlay, Ink): iOS 11 also includes many improvements to the basic use and integrated apps. Most of these do not substantially enhance or differentiate the developer opportunity, but are an important part of advancing the overall ecosystem. You can read or experience these elsewhere.
However, the one major change worth noting is the new screenshot workflow, which is seriously game-changing. Not only does it address common capture issues (getting the wrong shot the first time, but still saving to camera roll, or always needing to crop out the carrier/time), but it adds easy markup, annotation, sharing without saving and more. Closely related is the ability to capture video of sessions, as well (but not video of sessions using the camera or capture!). In addition to random Ink, you can add text, a signature, shapes/callouts or add a magnifier. Crazy cool!
For Pencil fans, the Ink work is very nice, and shows Apple’s fundamental approach which is that Ink is like an acetate layer on top of everything or a region to fill, but not a keyboard replacement. There are many new features, some of which have existed in Windows (or Windows or iOS apps) for quite some time, such as shape recognition, that will prove valuable to some. The CarPlay “do not disturb” feature is exactly the type of feature that Apple can and should do, that few others would try, and so it is worth high praise for this work. Most platforms would think of CarPlay DND as too parental or even taking away freedom, but it is absolutely the right thing to do.
Like many iPad users, I’ve been surprised and disappointed at the pace of iPad hardware changes (as I’ve tweeted about often, iPad is my primary device in terms of hours of usage, and where I do writing, financials and slides, in addition to longer sessions of reading, social, etc.).
The software improvements in iOS 11 already make using a 9.7-inch iPad Pro better. But nothing makes new software seem better than new hardware.
The nature of information or knowledge work (selling, marketing, product management, financials, planning, executing and more) continue to undergo changes. These changes are never happy overnight, and so many look to the pace of change to insist that the change isn’t happening. At the same time, so much of the public dialog around work in technology is dominated by engineers and designers who can and should be using Mac or Windows PCs (and will continue to do so). That whole world is 50 million to 100 million people worldwide, and isn’t likely to change for some time (until the scenarios completely change, or those tools exist on an iPad).
The new iPad Pro looks is exciting, and having had a chance to see one, I am certain that a broader set of people will tilt toward iPad-only (assuming that we can still fly on planes with iPads!). The typical knowledge worker will move to the iPad — or some other device with ARM, mobile OS and those attributes — sooner rather than later. The iPad gives us a big screen to accompany our phone-sized screen.
The new iPad Pro with a slightly bigger screen is great. The new keyboard continues to disappoint (me). It is still a keyboard in the middle. It is too bulky for occasional use, and too constrained for all-day use. The origami unfolding is awkward, the lack of function keys limiting, and the stability poor. I have used the Logitech clamshell, and long for a much slimmer and better-designed clamshell container, which I know Apple (or anyone) is capable. There’s a new Pro case that turns the iPad into a Surface (like RT) with full-sized keys and a kickstand. It is still heavier and more awkward than it needs to be. That’s the one I’ll be using until a better clamshell case comes along (or Apple just does a clamshell iPad).
One more thing
HomePod was introduced at WWDC, even though there was not a direct developer message beyond SiriKit.
Music is a blind spot for me, so there are many better-qualified people to discuss this scenario. On the other hand, home control is a huge area for me. It is worth noting that in the real world, music is essentially infinitely more important than home control.
I’m aware of all those who use Alex/Echo and now Google Assistant for home automation tasks. As aware as I am, I am more skeptical that these will advance beyond demonstrations for some time. I’m especially dubious that a stationary speaker will serve as a broad entry into controlling home devices. The fact that the devices from others look to be adding to or relying on screens seems to scream “doesn’t quite work.” The idea of a stationary but smart screen in the home just seems weird when everyone has a screen with them or really close by all the time. And much like AR glasses makers rushing to make a full mobile OS/platform for glasses, speaker makers recrafting an existing OS to create yet another app platform seems tricky, especially if it is an interactive platform.
Home control and automation really doesn’t work at scale, except for a few silo scenarios now (Ring doorbell is a great example of something that works broadly). Something needs to replace one-off apps, but it isn’t clear to me that the replacement is me memorizing a vocabulary for each device and speaking to a stationary device.
That said, one needs to look carefully at not just the what (a really nicely done premium entertainment device) but the how. Apple has set expectations, and is likely to deliver a “just works” product that looks to deliver on the core scenario of music even better than competing products. That’s what Apple did with portable music and, later, video. It was after that initial entry that Apple chose to expand the developer opportunity and platform capabilities of the product. Perhaps with Apple TV, Apple failed to capitalize on that approach. Perhaps with Watch, Apple tried to do too much too soon, and did not deliver on the just works aspect. HomePod seems much closer to how iPod and iPhone came to be, and for that reason alone, I am optimistic.
Apple released a massive amount of software this past week. Most of it works, too! MacOS is ready for most any developer to use full-time in my experience; iOS 11 is less so.
There’s a tendency to see a cloud of analysis around Apple events looking at things from a perspective of Wall Street or how iPhone unit sales will be impacted, which I think can be short-sighted. What Apple showed at WWDC expands computing in new ways — new capabilities for mobile phones that can be used by a scaled ecosystem, new hardware combined with software capabilities that can (finally) change how typical business users accomplish productivity tasks, and hardware and software for consumers that bring a new take on the execution of existing by still-immature categories.