By Jason Snell

June 13, 2024 4:29 PM PT

Last updated October 24, 2024

OK, fine, here’s Apple Intelligence

It was clear that Apple didn’t want to be a part of this.

For years now, the company has resisted using the phrase Artificial Intelligence to describe what it does. It’s facile and misleading and not what Apple is really about at all. So it’s been working on advanced computing techniques and the hardware to drive them, all using phrases like machine learning and Neural Engine.

Then, a couple of years ago, the AI buzzword exploded into the consciousness of… if not the general public, then the tech industry and press. The image creation and chatbot demonstrations were, yes, facile and misleading, but also quite a party trick. The narrative shifted: Was this the future of computing?

And there was Apple, churning away on its ML photo tagging in Photos and eye tracking in visionOS and image segmentation for silly video effects, suddenly caught flatfooted. The area it had looked on with some disdain—and not without reason—was suddenly the next big thing in tech.

And, of course, the scary part was: What if it was the next big thing? What if—after investing billions in visionOS and a car project that never even saw the light of day after a decade, all as a hedge against anything that might eclipse the iPhone—the technology that it had pooh-pooed was the real existential threat?

According to multiple reports, there was a moment in late 2022 or early 2023 when Apple realized that it needed to aggressively embrace AI or risk being seen as old and out of touch—not just by its industry but by the general public. The result was a prioritization of AI features all around the company, and the first fruits of that prioritization were on display at Monday’s WWDC keynote.

Three kinds of AI

It seems to me that on Monday, we saw three different kinds of AI features. Apple won’t ever say which was which, but the rest of us can guess at it. The three categories are:

• Features Apple would’ve done anyway. The company has been rolling machine-learning stuff into its software for ages now. Would smarter photo tagging and better Inbox organization in Mail have been announced regardless of Apple’s new, big AI focus? Almost certainly. Apple has always been about leveraging advanced technology in the service of customer needs.

• Features Apple would’ve done… eventually. It sure seems like Apple’s shift of priorities led to some features being announced that seem very much like the sort of thing Apple would’ve done, but at a slower pace and more conservatively. There seem to be more WWDC announcements than usual that ask for our patience because they won’t be ready until later this year or even sometime next year. But if it was an AI-related feature and was on the drawing board, it probably got pulled forward for introduction at WWDC 2024. Ready or not, here they come.

• Features Apple would never have done. There are some areas where Apple shows the strain of fearing they’ll be seen as burying their heads in the sand and resisting the trends. ChatGPT support and Image Playgrounds both feel like areas where Apple is stepping far out of its comfort zone in order to make it seem more tuned in to the zeitgeist.

We’ll see how these different features end up faring over the course of time.

The difficult choice

Last week, I wrote that Apple should see itself as the grown-up in the tech industry’s rush toward AI at any cost. For all of AI’s potential, it can be unreliable, the training data sources can be questionable, and it all just feels overhyped. (To me, AI manages to be simultaneously overhyped and potentially world-changing. Great potential… lots of snake oil. All at once.)

Apple’s WWDC announcements show that the company wrestles with this idea too. In its most thoughtful moments, it has expressed the desire to approach AI features with clear, important guidelines. The tools are meant to be intuitive, personalized, deeply integrated into Apple’s product experiences—they’re features, not tech demos—and build with privacy in mind.

In most cases I think Apple’s announcements showed those principles in action. This is a company that realized it was not going to be able to perform all the AI actions it wants to from a user’s device, so it built its own server hardware and operating system and is hosting them in its own data centers, all so that it can ensure that private user data isn’t ever misused. Nobody else does that, but Apple knew it had to.

Most of what Apple announced is really about solving problems for users. Rather than creating a model that has eaten the entire Internet but has a troubling tendency to hallucinate, Apple has focused on serving a user’s personal contexts. It seems to have avoided buying into the broader tech industry fallacy that the future of computers is asking questions to AI chatbots that respond with clichés and dissembling and hallucinatory answers.

Well. Avoided for the most part.

Apple has always been good at building features that play to its strengths, both technical and market wise. It owns your phone, it’s got your data in its ecosystem, it’s invested in building neural processing into its chips, and now it can put that all together to build a whole new Siri that knows your personal context and can put it into action.

But at the same time… all the cool kids are making weird AI art in Midjourney and Stable Diffusion, I guess? And even if Apple doesn’t have a chatbot that’s been trained on “world knowledge” (the official buzzphrase of WWDC 2024), I suppose you’re not in with the in crowd unless you’ve got a chatbot?

This is where Apple has compromised, a little, at a distance: Its Image Playgrounds feature makes weird AI-looking art, built around a nicer interface than any fill-in-the-text prompt, but still kind of weird and off-putting. And Apple has wired in ChatGPT, because if Siri can’t help you, maybe a chatbot can.

Warning labels and defaults

Which brings me to Apple’s ChatGPT integration, which seems utterly necessary from the standpoint of making the iPhone not feel hopelessly behind Google, which is rolling its Gemini chatbot into Android. People who despise AI hype in general and chatbots in particular will probably be disgusted by Apple’s choice, but if you look closer, Apple’s clearly not comfortable with any aspect of the partnership.

Think about it. Apple made a much-ballyhooed introduction of its partnership with ChatGPT by showing off that every single time Siri offers to use it; it throws up a (scary?) alert dialog that asks if you’re sure you want to use the chatbot. When the ChatGPT results return, they’re appended with a warning label advising that you check all the information if it’s important!

That was just in the keynote. Before the day was done on Monday, Apple execs had also made it clear that the feature was turned off by default and that ChatGPT was just the first partnership of many. Google Gemini was mentioned by name as if it was only a matter of time before ChatGPT’s archrival would be integrated. They likened it to offering users different, interchangeable search engines in Safari. They also suggested repeatedly that the ultimate goal of the “external chatbot” feature wasn’t really general knowledge at all but chatbots trained on specific areas of knowledge, like medicine.

Now Bloomberg’s Mark Gurman reports that Apple’s not even paying OpenAI for ChatGPT. I get why OpenAI would do this deal—its biggest competitor has the home-field advantage on the world’s only other notable mobile operating system—but it really looks like they’re the one over the barrel here, not Apple.

I think that might say it all. The conventional wisdom going into WWDC was that Apple was the company that was flailing and desperate, trying to catch up to the giants of artificial intelligence and retain some level of relevance. What we saw this week, however, was a company that seems surprisingly confident in its own AI prowess, and one that continues to largely follow its own playbook rather than compromising its ideals.

The hottest company in tech, OpenAI, just gave its crown jewels to Apple for free, and Apple responded by introducing its integration as optional and tagged with numerous warning labels. So, who’s behind and flailing, exactly?

The truth is that Apple right now is like a duck: serene on the surface but paddling furiously underneath. It was clearly complacent about the pace of AI innovation and allowed itself to get a bit too comfortable, and now it’s hurrying to keep up. It will undoubtedly make a few mistakes along the way. We also don’t know how good Apple’s stuff is—note that most of the Siri section in the WWDC keynote was aggressively in the future tense—and have to take the company’s word for how private its cloud services really are.

There’s a lot to interrogate here, and if Apple can’t fulfill its promises, it will be in real trouble. That’s a story that will be written over the next year, but right now, I wouldn’t bet against them.

By Dan Moren

June 13, 2024 11:06 AM PT

Excluding your website from Apple’s AI crawler

One of the more contentious announcements from Apple this week is that it trained its foundation models used as the basis for its forthcoming Apple Intelligence features via, among other content, the open web.

Obviously that raised a lot of eyebrows among those of us that publish content to the web. Using copyrighted material to train AI falls under fair use is a question still being hotly debated, and one that may ultimately be highly dependent on the exact circumstances. It’s also a behavior that people feel justifiably uncomfortable with.

Setting aside feelings on that issue for just a moment, it’s worth looking at the mechanics behind this. Apple also said during its announcement that it’s providing a way for publishers to exclude their sites from being used for training its AI models, via a long established system built originally for search engines: robots.txt.

Robots.txt or not

If you’re not familiar with robots.txt, it’s a text file placed at the root of a web server that can give instructions about how automated web crawlers are allowed to interact with your site. This system enables publishers to not only entirely block their sites from crawlers, but also specify just parts of the sites to allow or disallow.¹

Apple’s own Applebot web crawler has existed for some time; it’s used to power search features in Spotlight, Siri, and Safari. For example, whenever you see a Top Hit in Safari or Spotlight, that information is coming from a search index created using Applebot. Podcasts are also fed by Applebot, though in those cases only via specific URLs registered with Apple Podcasts.

If you’re worried that blocking Applebot from crawling your site might impact your site showing up in traditional search results, good news: the new AI training element of Applebot uses a separate identifier, allowing you to block only that functionality without affecting your site’s appearance in Apple’s search features.

How to stop Apple from using your content in AI training

Apple provides a detailed support document about Applebot and the various directives to control how it interacts with your site.

To specifically exclude your whole site from being used for Apple’s AI training features, you can add the following to your robots.txt file:

User-agent: Applebot-Extended
Disallow: /

To test this out, I’ve added those directives to my personal site. This turned out to be slightly more confusing, given that my site runs on WordPress², which automatically generates a robots.txt file. Instead, you have to add the following snippet of code to your functions.php file by going to the administration interface and choosing Appearance > Theme File Editor and selecting functions.php from the sidebar. (You can also do this via a plugin like Code Snippets, which I use.)

add_filter('robots_txt', 'my_robots_commands', 99, 2); // filter to add robots

function my_robots_commands($output, $public) {
  $output .= "User-agent: Applebot-Extended\nDisallow: /";
  return $output;
}

If you want to go beyond Apple, this same general idea works for other AI crawling tools as well. For example, to block ChatGPT from crawling your site you would add a similarly formatted addition to the robots.txt file, but swapping in “GPTBot” instead of “Applebot-Extended.”³

Google’s situation is more complex: while the company does have a Googlebot-Extended that powers some of its AI tools, like Gemini (née Bard), blocking that won’t necessarily remove your site’s content from being crawled for use in Google’s AI search features. To do that, you’d need to block Googlebot entirely, which would have the unfortunate effect of removing your site from its search indexes as well.

Fait accompli

One challenge with these AI tools, however, is whether or not the damage has already been done. Many of these AI models have, of course, already been trained, and it’s not as though you can remove training data from them by blocking these crawlers now—it’s very much closing the barn doors after the horses have gotten out.

However, it does mean that you can prevent your site and content from being used for training going forward, so any material you publish from that point on will be excluded. If you feel like that’s cold comfort, you’re not alone.

Update: An earlier version of this article misstated the user agent for ChatGPT’s crawler: it’s GPTBot, not ChatGPT-User.

[Dan Moren is the East Coast Bureau Chief of Six Colors, as well as an author, podcaster, and two-time Jeopardy! champion. You can find him on Mastodon at @dmoren@zeppelin.flights or reach him by email at dan@sixcolors.com. His next novel, the sci-fi adventure Eternity's Tomb, will be released in November 2026.]

By Jason Snell

June 12, 2024 2:10 PM PT

visionOS 2 offers small, but promising, steps forward

I’m just back from Cupertino, and there’s an awful lot to think about. But before all that, I thought I’d cover what absolutely everyone is talking about: visionOS 2.

More seriously, I installed the visionOS 2 developer beta this morning—this is the entire reason I have a Vision Pro!—and I’ve got a few quick thoughts before plowing on to developer sessions, thoughts about Apple Intelligence, and the rest.

Spatial photos. The most impressive single feature I’ve seen in visionOS 2 so far is the ability to create spatial photos from out of your old, mundane 2D photos. Load pretty much any photo in the Photos app and tap the Spatial icon in the top left corner of the image, and a fun sound effect plays as an animation sweeps across your photo, representing the system’s machine learning-driven software scanning your photo and building an artificial depth map to provide the illusion of depth.

You know, very few movies are shot in stereo anymore. It’s more expensive and cumbersome to shoot, and conversion to 3-D after the fact is good enough. Upon viewing Apple’s spatial-converted photos in visionOS 2, I had similar thoughts: I don’t see why we would ever need to shoot stereo images if machine learning is this good at faking it.

Seriously, whether it was a photo taken last week or 50 years ago, Apple’s algorithm does a staggeringly good job at building a depth map. You don’t need embedded LiDAR or other depth information—the algorithm does it, and it does in incredibly well. Pictures of my kids taken when they were little are suddenly given more depth (literally and figuratively). Pictures of me as a kid, even. A complex shot with a tower made of LEGO bricks in the foreground was scanned and mapped perfectly.

It takes about 15 seconds to spatialize a photo, so it’s unlikely that Apple will ever ask the Vision Pro to churn in the background through every single photo in your library, but I’ve yet to see a photo that failed to become more interesting after being converted.

New gestures. Apple has added some new gestures to visionOS, which key off of you holding out your palm and looking at it. When you do that, a floating icon appears next to your hand indicating that you can tap your finger and thumb together to open the Home View. (There’s also a new Close button at the top of the Home View so you can close it again.) It’s a nice idea because as much as I’ve internalized reaching up and tapping the Digital Crown to bring up the Home View, the act of doing so is also jarring given that all other Vision Pro interactions are gestures happening in the air in front of me.

I’m actually more excited about the second gesture that keys off the first. After you look at your stretched palm (have you ever really looked at your hand?) you can flip your hand over to reveal a floating bubble that displays the time, battery percentage, and volume. Given how frustrating it was to quickly check the time in visionOS, this is a great new feature that I anticipate using a lot.

If you bring your fingers together while looking at the bubble, you can slide right or left to adjust the device volume quickly. And if you tap while looking at the bubble, Control Center opens. This, to me, is a much better way to access Control Center—though it’s certainly less discoverable than the little firefly that frequently hovers at the top of your vision.

Customize Home View. You can now move apps around in the Home View, and it works pretty much how you’d think: while looking at an app, bring your finger and thumb together to enter Jiggle Mode and then pinch the app to drag it around, even across pages. It worked exactly as I expected it to, and I’m happy to finally be able to put some of my favorite apps on page one.

Breakthrough keyboards. Typing on a keyboard in an Environment was previously very weird, because while your hands were visible, your keyboard itself was not. Apple has upgraded this in visionOS 2 so that it recognizes either the Apple Magic Keyboard or the keyboard on a MacBook, and that’s great.

But at least in the first developer beta, I found that it was a bit finicky—it needed the keyboard to be positioned just-so for it to appear, and even then it sometimes felt like the environment was eating away at the edges of the keyboard. I’m not even a hunt-and-peck typist, but being able to actually orient on the keyboard by seeing it is still valuable.

Also, I’m disappointed that Apple has limited this feature to its own keyboards. I realize it would be harder to generate a model that recognizes more generic keyboards, but most keyboards really do have some pretty obvious characteristics in common, don’t they? Alternately, maybe Apple should consider a feature like the one Meta offers, which lets you work in an environment but with a specific cut-out—like a tabletop—set to pass through.

Life’s a beach. Famously, visionOS shipped with two Environments marked as “Coming Soon”—one featuring a blurry beach image and another with fog-shrouded trees. The trees remain a mystery—Twin Peaks environment?—but the beach has arrived in the form of Bora Bora. As someone who loves beaches, it is spectacular. You can hear the sound of the waves, the palm trees gently blow in the breeze, and you can even see the color of the beach change as a thin cloud passes over the sun. I think I am going to be spending a lot of time in Bora Bora (the visionOS 2 environment, alas).

I didn’t get a chance to test out some other features due later this year, including support for a much larger display in Mac Virtual Display mode and multi-view support in the Apple TV app. In a Spatial Persona call with my podcast co-host Myke Hurley, the updated personas looked good (most notably hand gestures) but Myke’s mustache still prevented his persona’s mouth from moving. There’s more work to be done there.

Still, it’s exciting to get some major new features out of visionOS 2, despite the fact that visionOS 1.0 only shipped a few months ago.

By Jason Snell

June 7, 2024 2:35 PM PT

Siri’s biggest gains might be found at the edges

We’re on the verge, maybe, of something big. Apple broke ground with its voice-driven Siri assistant back in 2011, but over the years, Siri’s trajectory has been meandering and frustrating. It’s driven deeply into popular culture—”Hey Siri” is practically a catchphrase—but it’s still painfully inconsistent at best and a failure at worst.

The rise of AI chatbots, which (despite their tendency to hallucinate) manage to project the illusion of intelligent conversation, has really put Siri up against it. This week, the Wall Street Journal reported that Apple had a faster, more conversational, and more accurate Siri waiting in the wings at least three years ago—but decided to go with a more modest upgrade code-named “Siri X.”

If the idea that someone inside Apple felt that Siri didn’t need to be more conversational and accurate makes you angry or confused or both, I’m right there with you. We can only hope that Apple has reconsidered its complacency over the last 18 months, since Apple executives reportedly realized that the rise of AI chatbots was destined to change user expectations.

I hope that the people who lost the arguments for a better Siri back then may have been winning those arguments for the past 18 months. And it’s made me optimistic—here in the shadow of WWDC, where everything’s possible, and nothing’s decided—that Apple might really be on the road to giving Siri the biggest upgrade it’s ever received.

That’s a big deal—and not just for the obvious reason, which is that I’d like to use Siri for something other than a tea timer or to add something to my shopping list. That’s because with a good, reliable, conversational Siri, a lot of Apple’s products can benefit.

When I’m running or walking my dog, I generally use just AirPods and my Apple Watch. Theoretically, I’m covered by Siri, but I generally avoid talking to it because it’s unreliable. I’ve also turned off all of the features that summarize messages I’m receiving because I find them awkward and unhelpful.

But imagine what a good Siri experience could unlock for users of AirPods, Apple Watch, or both. Humane built an entire product—the Ai Pin—out of a voice assistant that you carry around with you. But with a better Siri interface, Apple’s already most of the way there. The better the voice interface, the more useful your device can be when you need to interact with nothing but your voice.

Similarly, an upgraded Siri could be a huge step toward improving the CarPlay experience. As is the case with most contexts in which Siri is available, I’ve tried Siri in the car and learned my lesson—namely, to never use it again. When I’ve got to keep my hands on the wheel and my eyes on the road, being able to have a proper voice assistant that can summarize what’s going on, understand context, and follow up when I ask for it could be incredibly valuable. But Siri being unreliable has degraded that experience.

If there’s one place where the Humane Ai pin and those Meta Ray-Ban sunglasses have an edge over Apple’s stuff, it’s that they both offer a forward-facing camera to interpret the world around you. With an upgraded and competent Siri, it might make more sense for Apple to sell a pair of glasses or a clip-on camera pin as another accessory to help improve the data being input into Apple’s AI systems.

And then there’s the kitchen, where a device that combines a HomePod with a screen and camera might end up being quite welcome—I’ve used Google and Amazon devices in my kitchen for years—but it really needs to be rock-solid when it comes to voice commands. The Siri on my current HomePods doesn’t measure up.

Of course there are numerous ways that a better Siri and more AI-focused features can improve Apple’s core platforms. (I’d love a reason to use Siri on my Mac, for example, but I’ve never really found one.) But it strikes me that Apple’s biggest opportunity might actually be at the edges of our lives, where traditional interfaces are harder or even impossible to use. A useful Siri instantly makes the Apple Watch, AirPods, CarPlay, and HomePod better—and could facilitate even more interesting products to come.

But first thing’s first: Siri needs to get vastly better. I hope that we get the first sign of that on Monday.