By Joe Rosensteel

September 19, 2024 9:08 AM PT

When it comes to traveling abroad, Apple Maps could use a little direction

Over the past year of international travel I’ve been taking notes on the apps and services I use to get around and how they’ve changed over time. Dan and Jason have done the same. While we—as a species—have come a long way from being completely lost when we’re dropped into a new place, in my estimation we still have a ways to go.

Apple seems less convinced: the only major update to Maps in iOS 18 was the addition of US-only hiking directions. I mean, it also added thick strokes and drop-shadows to its tiny, visually busy icons, so I guess that counts for something. Google is a little ahead of Apple in a few places internationally, but not leaps and bounds. So while you might not get lost while traveling abroad, the experience is certainly rockier than it could be.

Continue reading “When it comes to traveling abroad, Apple Maps could use a little direction”…

By Joe Rosensteel

February 26, 2024 3:55 PM PT

Apple’s mixed-up Messages

Whether it’s attempts to regulate iMessage, or attempts to circumvent Apple’s hardware requirements to use iMessage, there’s sure been a lot of interest in Apple’s meager messaging platform lately. From a competition standpoint, iMessage has a grip in North America, but little penetration elsewhere where more platform agnostic messaging apps are preferred.

What is it that we like so much about iMessage and the Messages app? I use them multiple times a day, across the Mac and iPhone, and yet I’m not sure I would call the experience “good” or advocate for it in any meaningful way that didn’t invoke security and privacy concerns.

Reliability

iMessage delivery has been pretty reliable for many, many years. You send it, a little piece of gray text pops under your message a few seconds later and says, “Delivered” and you don’t have to worry about it.

Sure, there was the weird thing that would happen when you’d try to send someone a photo, but the network connection wasn’t strong enough, and then it would just hang that little blue line, and none of your following messages would get through. You’d have to wait a few minutes until the iMessage failed to send. Surely they’d make that experience better some day, instead of… never improving it?

Then there’s the weird thing that happens when you wake your Mac and it starts notifying you about old messages, and maybe a chunk of message history is missing. Oh well. Sometimes it pops up later.

Occasionally read status gets out of sync, but never anything as bad as Slack, which just celebrated 10 years of not being able to remember what I’ve read.

More often than not I’ve been told that I have Do Not Disturb enabled, when I don’t. Just toggling the little DND in control center resets it, but why does it do that without any rhyme or reason?

There still isn’t an official way to export or archive my iMessage history, which has become something I’m more concerned with these days as I’ve had two occasions, in the past two months, where my iMessage conversation history with my boyfriend of 14+ years temporarily disappeared while I was on cellular, but then just magically popped back when I was on Wi-Fi.

So do I still think of iMessage as reliable, or am I just used to the ways in which it is less than reliable?

Features

I often think that all I really expect out of Messages is the ability to send clear, legible text messages and photos. But even the simplest texts can sometimes trigger message effects that were never my intention. (Congratulations!)

The Apple Cash integration, which highlights every monetary amount with an underline so I always look like I’m trying to ask for money, is especially obnoxious. Clearly someone at Apple who is sweaty for people to use Apple Cash considered it a win-win, but I’m almost never sending money.

As for sharing photos… if I send one photo, it shows up in the original aspect ratio, with some pixels shaved off to round the edges and give it that little message speech bubble tail. If I send two or more photos, then all of a sudden we’ve steered into Whimsical Stack Town where Messages has decided that the clearest way to present the photos I’m sharing is a game of 52 pick-up.

The right thing to do is to tile the photos to fit the space without overlap to maximize the use of our limited screen real estate. I want a contact sheet, not a quirky slideshow. Tapping on the “[X] Photos” to bring up the contact sheet view doesn’t help, because it appears entirely outside of the context of the conversation.

iOS 17 and macOS Sonoma also made it take more effort to share photos via Messages. In iOS 17, everything except audio messages got sucked into the new, terrible, “+” menu. Which is not a menu, but a completely modal screen that obfuscates everything to show you a handful of common message buttons, including Camera and Photos. If you don’t tap the invisible bounds of the thin font used for Camera and Photos, or the small circular icon, you’ll dismiss the dialog entirely.

(Pro tip! In a completely unintuitive and non-obvious stroke of sheer un-genius, you can long-press on the “+” to get to the Photos picker.)

In Sonoma, where even the smallest Mac screen is gigantic, the “+” icon has been replaced with a tiny app store icon button, along with a series of lines as a sort of waveform for the audio message and an emoji icon. Why are the icons different? Who could say?

What I can tell you is that I have to click on the App Store icon, then click the Photos icon, and then wait for that to spawn a floating photo picker panel that is attached to the App Store icon, and can’t be moved or resized. Oftentimes I find it easier to find a photo and copy and paste it into the conversation, which seems more than a little absurd if I stop and think about it.

Fun!

Of course a messaging service, and its apps, need to go beyond the ability to send text and photos. We want to have fun with our conversations. That’s why every chat and messaging app includes the ability to react to messages with fun emoji. Oh, I mean every platform except for Messages, which Jason Snell has been on Apple’s case about for a long time.

The emoji sticker reactions suck. I absolutely loathe the jaunty angle that Messages applies to everything. I didn’t place it at a jaunty angle because I don’t want it to be a haphazardly applied sticker. This isn’t some three-ring binder that I’m trying to jazz up with Lisa Frank stickers.

Apple’s attempt to harness the raw power of fun with the Messages App Store hasn’t died yet, so I guess that still counts as “fun”. It still seems to provide dozens of people with access to official Starbucks Messages stickers.

Probably the most “fun” Messages-only feature is exclusively available to the Apple Watch’s version of Messages, and that’s Fitness notifications. My friends and I use the feature in an almost passive-aggressive way. We send ironic congratulations over short walks, or the baffling “Can I call you later?” prompt. All the other platforms get replies to the fitness notifications, even though the Mac still can’t display the Fitness notification that was replied to! But only the Watch can see the initial Fitness notification to start that conversation.

Pump up the jam

I’m uncertain if Apple’s warmed over iterations of Messages are because they see no reason to really compete in the messaging arena, or if they would be exactly as uninspired if they were regulated out the wazoo. Personally, I would rather see Apple innovate of their own volition to provide us with things like increased reliability and support across their platforms. Give us cleaner interfaces to our most used functions, and fun that feels like actual fun, instead just knocking things slightly askew and telling us they’re fun.

Sent with lasers.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

January 25, 2024 8:00 AM PT

iPhones need to understand that their users are mobile, too

This summer Mercury Weather added trip forecasts and it opened up my mind to a new way to think about how my iPhone could be used to help me track future events while I’m in a state of travel. I could have the same kind of glanceable 8 day forecast I was used to, but those eight days could be strewn across the globe. I wasn’t confined to my iPhone’s current location.

It works in a really dead-simple way. You hit “Add Trip” and you pick your start and end date, and your location. You can stack up as many as you want: Like London, Santorini, Naxos, Athens, and Vienna — hypothetically. As soon as there’s a day without a trip it snaps right back to showing you your home weather. Brilliant for when your trip is winding down and you want to see what you’ll be headed back to. It’s only as granular as a whole day, so on travel days you need to pick which is more important to get a heads up about.

It sure would be great if I could pipe a calendar with a trip itinerary right into the app, and maybe even get the hourly forecast based on my flight time. And it would be even better if that was something that happened at the system level instead of forking over my whole calendar for such a feature.

With apps like Flighty, or even United’s updated app with Live Activities, I could travel across time zones as smoothly as a progress bar. Instead, I’m always adding and subtracting the change in time zone and the total flight time.

Unfortunately, support for this level of travel detail isn’t built into iOS or watchOS. Apple devices always think that wherever they are right now and whatever timezone they’re in right now are constants. All this, even though my iPhone has emails, boarding passes, and calendar entries that indicate I’m on the move.

Any scheduling I do during the trip needs to be offset appropriately before the journey, or for once I return home. I don’t want to see what time a dinner reservation in London is in Pacific Standard Time, because that’s not how I will think about it when I’m there, and vice versa.

Add to that that any time-based modes like Sleep¹, Do Not Disturb, and other Focus Modes will all trigger based on the time zone my devices were in before I switched on Airplane Mode at the start of a flight. Ironically, Airplane Mode doesn’t think the time will ever change.

Not every journey is a dramatic crossing of the international date line, hoping across the pond to Europe, or even a flight from California to the East Coast², but you do gain and lose hours that can have an effect on when you want to sleep or start your day. Hello jet lag!

There’s an app that a friend of my swears by for jet lag, Timeshifter, which I haven’t ventured to try, but it sure seems like the kind of nagging “health” notification that Apple Watch product managers would salivate over.

Flights of fancy

Apple should add a layer of travel savvy to its devices. Sure, there’s complexity here—programmers detest time-zone programming for a reason—but this is exactly the kind of lifestyle feature that users appreciate. It’s not as if Apple doesn’t already have the components lying around: if my flight information is in Calendar, that’s everything needed for time zones and locations for weather. It’s not like I’m asking for an AI virtual travel assistant.

Beyond the basics, there’s even more Apple could do. When I arrive in Paris, instead of flashing a notification that transit directions are available, or telling me there’s a detailed map of CDG³ available, my phone should ask me if I want to switch my navigation preferences to transit or walking while I am in Paris… and then revert to my default when I leave.

My phone should allow me to designate my home-away-from-home. Let’s stick with Paris as an example: I want to have something as easy as tapping “Home” but for my hotel. You can favorite the hotel, but it gets lost in all your other favorites, recent searches, and other detritus, unlike “Home,” which Maps thinks is a helpful navigation suggestion in Paris. (This also works for when I’m visiting my mom and staying at her house.)

In addition to keeping that hotel at the ready, my device should also be able to understand things like the address of the next hotel, so if it’s check out time and there’s a trip to Strasbourg, apps won’t keep suggesting my Paris hotel as a destination for navigation. We’ve moved on. Literally.

You don’t even need to travel far to see how things might benefit you. If you live in a place with mountainous terrain and microclimates, you could even be alerted to a chance of rain or snow a short distance from you just when you’ve got an event on your calendar there. Or imagine being given a heads-up when you’re headed to an office in a nearby region that’s ten degrees warmer, so you can dress appropriately.

From the simple to the more complex, there are plenty of ways Apple could make its devices better at anticipating our movements. This goes for Apple’s own apps, but also allowing third-party apps—like Mercury and Flighty—to tie into the same information. That would be the real ticket to success.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

December 18, 2023 3:05 PM PT

Add to Dock: Safari’s sweet solution

The Mac Community generally loathes web views and Electron apps. We all want perfect, native Mac apps—even when an interface could be entirely native, and still not be very good. Pragmatically, we ought to recognize that we just aren’t going to get bespoke SwiftUI versions of every little thing we use in our lives. There certainly isn’t a financial incentive to do it on the Mac App Store, and the scary OMG SECURITY warnings deter a lot of non-App-Store use of smaller apps.

This is where Safari’s new Add to Dock command comes in.

You may remember Steve Jobs suggesting we add web pages to our iPhone home screens as a “sweet solution” to not having an App Store, but back then we didn’t have the rich mix of web technologies we have today—they were really just glorified bookmarks. Progressive Web Apps (PWAs) aren’t new either, and in a lot of cases they might be overkill. Making a little container for a site is more than enough, in some cases.

There have been utilities that make apps out of a site, including Fluid, Coherence X, and Unite. Chrome has had the feature for years. But in Sonoma, Apple finally introduced Add to Dock in Safari, building into macOS the ability to package up any site. Here’s how a web app differs from a web page:

When you use a webpage as a web app, it looks and behaves just like it does in Safari. Yet the experience of using a web app differs in several ways.

• A web app functions independently of Safari. It shares no browsing history, cookies, website data, or settings with Safari. In this way, it keeps your browsing separate, similar to using a Safari profile. What you do in a web app stays in the web app.
• A web app has a streamlined toolbar, with only a back button, forward button, and Share button. If you need Safari features such as bookmarks, tabs, or extensions, you can easily switch to Safari: Click the Share button, then choose Open in Safari. Or choose File > Open in Safari.
• A web app can have any name or icon that you want.
• For websites that send notifications, the web app’s icon in the Dock can show the number of unread notifications.

In all other ways, a web app works like any other app. You can even add it as a login item so that it opens automatically when you log in.

While the feature is called Add to Dock, that’s not quite what’s happening. The ‘app’ lives in /Users/your-user-name/Applications, or if you prefer, ~/Applications. If you remove the app’s icon from your Dock, macOS still leaves the ‘app’ in your user folder’s Applications folder.

Break out of tabs

I love tabs as much as everyone else, but sometimes you have a specific site you visit often, and it can get lost in the shifting tabs, bookmarks, etc. You might have a specific window size you want to use for that web site, but not all of your other sites.

This isn’t for every site, just some, and what people find useful will vary, but that’s the beauty of it. Like you wouldn’t want to have an ‘app’ for Six Colors. That would be goofy.

Personally, I wouldn’t try to do this for Discord or Slack, which can be accessed in Safari, but have a lot of AV infrastructure inside of their bloated Electron apps that might not work well in a little Safari container.

Here are some examples I’ve been using:

Fastmail. The first ‘Add to Dock’ for me was Fastmail, which hosts one of my many email accounts. I like to keep all my emails siloed by app, not a unified inbox with work and personal stuff intermingled. Different strokes for different folks. I don’t like Mail on the Mac, even though I use the iOS client, and I’m not going to hop on the latest email client du jour. I also want access to the features Fastmail has on the website, like generating custom email addresses, that I can’t access any other way. Now it’s an app sitting right there in my dock.

Gmail. I’ve historically used Chrome to check my Gmail, and leave Safari signed out of my Google account. Because I prefer siloed email experiences, this wasn’t a huge deal, but sometimes you get those confirmation emails that you have to click on but you really want to keep doing what you were doing in Safari and it’s a whole back and forth that’s not any fun. Also there are all the settings and blah blah blah. It’s why third-party email clients that handle Gmail don’t really do it for me, either. Just like with Fastmail’s web site, the Gmail web site works with all of its own features in an expected way.

Gmail is a little less friendly to set up, if you want to have Safari logged out of your Google account, because you need to log in in Safari, add Gmail to your Dock, then log out in Safari, and then log in to the ‘app’. If you don’t care about where your profile is logged in then this doesn’t matter.

Now you have a Gmail window that can be as big or as small as you want. Any links you click on in these emails will open in your default web browser (Safari, for example). That also means any other Google services you use will open in your default web browser and if it’s not logged into your profile you will be asked to log in. If you’re going back and forth between your Gmail, Calendar, Sheets in Drive, etc. then it’s best to just leave all of this in Chrome where you won’t get the nagging pop-ups to install Chrome, and log in.

The icon Safari uses for Gmail is just the favicon, and it’s hideous. I’d recommend replacing it with your own image. For some reason the Safari team doesn’t let you drag and drop images into the icon field when you’re in the ‘Add to Dock’ dialog, but there is a button where you can select a file. If you want to change the icon later, you can’t do it from the Get Info dialog either. You open the ‘app’ and go to the Settings dialog under the application name, and then click on the icon to bring up a file browser. No drag and drop there, either. Any image you put in will stretch to fit, so stick with square-ish icons. It will pad it out to a squircle shape with white behind it.

Even if I stick to using Gmail in Chrome, on the occasion I need to click on a confirmation link in an email I can always get to it from here with less fuss.

Spotify. I’ve been disenchanted with Apple Music (was I ever enchanted by it? No, no I wasn’t.) People are always so jazzed about Spotify so I figured I would give it a try. I didn’t want to install their Mac app, which people complain about, so I just did it in the browser at first, but I didn’t want to always have Safari open. It’s in my Dock now and it works just fine.

Of course, that also works well because Spotify saves my playback progress of what I was doing so every time I open and close the “app” I can resume exactly where I left off. A feature that Apple, with all the native software, iCloud backends, and trillions of dollars can’t pull off in the Music app.

Mastodon. I love Ivory and use it as my Mastodon client. But sometimes there are some non-Ivory settings I want to get at. If you’re reluctant to pay for Ivory, I whole-heartedly suggest saving your Mastodon page as an app rather than keeping it in a browser tab. Despite Mastodon’s general ugliness, the site is responsive and can be resized down to a little phone-like app, or sized to fit all the columns in the advanced layout.

Single Serving Success

I’d definitely be interested in hearing more examples of sites that work well as an ‘app’, and maybe some web site developers have some ideas to polish up their app to work well in this format. No code signing, no App Store headaches, and no Electron update mechanisms.

The web, and web apps, aren’t going anywhere, so let’s find ways to make useful, less bloated, apps. Sally forth, and Add to Dock.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

November 20, 2023 8:00 AM PT

Going in-depth on iPhone Spatial Video

The iOS 17.2 beta has brought the ability to shoot in Spatial Video for the forthcoming Vision Pro, and a handful of press participated in a demo where they could view Spatial Video on the Vision Pro headset. While the stuff recorded by Apple with the cameras in the Vision Pro headset naturally had better stereo separation than the iPhone, most members of the press seemed impressed by the content taken from a device that’s far more likely to be available to capture memories. (I’m more than a little curious to see a demo like that myself, but I’d settle for some good sushi.)

Earlier this summer I gave a quick overview of stereoscopic terms and filmmaking. Part of that post had to do with guessing at what Spatial Video was. In Apple’s marketing materials, they show third-person vantages of people experiencing perfectly separated, holographic experiences of things, but the reality is that it has a lot more in common with the left-eye/right-eye combo of traditional stereoscopic video.

In my piece this summer I linked to Chris Flick’s WWDC video, which covers general stereo terms and how Apple is handling streaming stereo content. The file container has a left-eye video stream and then metadata covering the differences between the two eyes in order to reconstruct the right-eye view. When Apple unveiled the iPhone 15 Pro and Pro Max, they touted that a beta update would bring the ability to shoot that spatial video, but they didn’t get into details, and showed another sci-fi hologram thing.

The iPhone 15’s Wide and Ultra Wide cameras were arranged so that when the iPhone was held horizontally, the software could crop in on them and get two similar-ish views. A reminder on the tech specs for the iPhone 15 Pro lenses and sensors that are being combined for Spatial Video:

• 48MP Main: 24 mm, ƒ/1.78 aperture, second‑generation sensor‑shift optical image stabilization, 100% Focus Pixels, support for super‑high‑resolution photos (24MP and 48MP)

• 12MP Ultra Wide: 13 mm, ƒ/2.2 aperture and 120° field of view, 100% Focus Pixels

I wondered what Apple would do to augment the left and right eye video capture to match them better, as anyone with an iPhone knows that there is a perceptible quality difference between these 0.5x and 1.0x lenses, but as my friend Dan Sturm pointed out on Mastodon, it doesn’t seem to be doing a whole heck of a lot:

First things first, I have to admit, I’ve been obsessing over trying to pull this stuff apart since the beta came out. It’s so easy to get caught up in the excitement around these types of things because it’s a new, magical experience. But there is no magic. This is exactly what you would expect Stereo3D footage from an iPhone to look like.

It’s very interesting to me how many [Stereo3D] “rules” they’re just ignoring here. The [depth of field] on the lenses does not match. The detail, color, compression, stabilization (or lack there of) does not match. The final image is not what one would call “good”, but it does work. It is [Stereo3D] footage from an iPhone.

Admittedly, for many people, it will feel like magic.

Dan’s referring to the slight differences between the two vantage points, which was one of the problems with iPhone video capture that I described back in June. Stu Maschwitz, and others found similar results, so it’s pretty safe to say it’s not a fluke.

To capture good 3D video, you ideally want identical lenses, and sensors, synchronously capturing what’s happening so that the only difference is the horizontal offset. Any differences in color, value, softness, will all seem to shimmer as your brain combines the two images. It’ll still have the illusion of being 3D, but it would be fatiguing, or uncomfortable to watch.

Without personally having access to a Vision Pro, I can only tell you things based on these videos we pulled apart using Spatialify, a iOS app that’s available only via a TestFlight beta. It is possible that visionOS is doing some additional processing of these videos as it decodes them, though my hunch is that it will continue to be exactly what it appears to be: two images from two very different cameras, put together.

There is also the matter of the fact that these videos are limited to 1080p30. I understand that the different focal ranges require substantial cropping on the Ultra Wide camera and produce a substantial drop in quality, but I’m less certain why the crop is exactly 1920×1080, since that’s not even the sensor’s aspect ratio. This video isn’t going into a 2000s-era TV, it’s meant to be viewed in an infinite canvas.

This limitation, more than anything else, really hampers why someone might want to capture Spatial Video right now. No, resolution isn’t everything—but it’s also not nothing. People also tend to shoot vertical videos because of how we hold our phones for both recording and viewing. This feature is asking people to choose between sharing a video optimized for phone viewing, or recording something that’s going to be part of a personal viewing experience.

Also consider the fact that Apple isn’t letting the iPhone 15 capture Spatial Photos. Stereoscopic photography has been around longer than motion picture film. That there’s no function to take a photo suggests something about the quality of the imagery. After all, it’s very easy to scrutinize a single still frame, while it’s a lot easier to forgive flaws in a constantly moving image.

I’m not saying that Apple’s Spatial Video implementation is bad. But I would be hesitant to recommend anyone switch their Camera app over to Spatial Video and shoot all of their videos with it right now. For the time being, I think people generally would be happier if they continued to shoot and share video as they do right now. You can always watch a video floating on a card in space with a Vision Pro headset, and at 4K resolution you can make it fill as much of your field of view as you might like.

So you still want to do it, huh?

If someone really does want to shoot Spatial Video, I’d recommend considering the subject matter first. In the demos Apple provided, they had a sushi chef making sushi for the journalists to record. The chef was near enough to camera that the chef would have internal depth, and also depth relative to the environment. Apple’s other videos also centered on people in environments.

From CNet’s Scott Stein:

The camera app makes recommendations on turning the camera sideways, and staying a certain distance from a subject. I was told to stay within 3 to 8 feet of what I was shooting for a good spatial video, but when I shot my test recording of someone making sushi at a table I got up closer and it looked perfectly fine. I also recorded in a well-lit room, but apparently the spatial video recording mode prevents adjustments on brightness and contrast, which means low-light recording may end up grainier than normal videos.

Shooting things like a wide-open space, with nothing in the foreground or mid ground, is not going to look or feel like much of anything. Please, I beg you, don’t shoot Spatial Video of fireworks—there will be no depth at all. Just because you think, “it’ll be in 3D” doesn’t mean it has any internal depth at that distance. You want that? Then record someone holding a sparkler.

Jason Snell took his iPhone 15 Pro to a Cal game for a couple of to shoot some Spatial Video. (See my video breaking it down.) Being in a stadium might feel big and grand, because you’re immersed in a large space that surrounds you—but it’s not something the iPhone can really capture. Spatial Video doesn’t surround you at all. You’re looking into a window at a stadium, and are very much separated from it instead of immersed. It will feel pretty flat without someone in the foreground as a subject. So definitely keep that window metaphor in mind.

Shooting something extremely close would likely cause an issue with things potentially breaking frame, so you could get close to something, provided it was “sticking out” at camera and not crossing the entire field of view.

Apple tries to mitigate issues with things breaking frame by having that fuzzy falloff at the edges instead of a hard termination where you’d be seeing something that’s exiting the frame in only one eye potentially causing strain. So be mindful of that, because you’re not going to see a soft-matte edge as you’re recording in the Camera app.

Try to always record Spatial Video in well-lit areas. There are still going to be subtle shifts in everything, because the lenses and sensors will match better when they don’t have to compensate for high ISO sensor noise and differences in aperture.

Eyes toward the future

I might sound pessimistic, but I’m not—I’m only skeptical of what’s currently on offer. It’s early days for Spatial Video. The first pass at Portrait Mode was really, really bad, but it’s now refined enough to where it can be executed as a post-processing option. Improvements over many years in hardware and software got us to this point, and if it’s a focus for Apple, I’m sure it’ll get there too.

I would caution that I’m only making that comparison to Portrait Mode in terms of the march of progress, not because I ever envision Apple shipping an entirely post-processed Spatial Video mode. As I mentioned in my previous post, when you offset something in post you’re cutting it up into layers, and every place where there’s something in front of something else is an occlusion. Where there are occlusions, there’s no data—and that has to be filled in. Could Apple do that with generative fill that’s stable across a frame range? Maybe, but that seems like more of a Google thing.

Maybe in a future iPhone we’ll have a better Ultra Wide camera sensor, lens, and optical stabilization? Perhaps we’ll see more advanced machine learning to transform the more detailed Wide camera data to cover over the inconsistencies in the Ultra Wide’s view? Maybe there will be the option to record 4K and Spatial Video with a later iPhone so you feel like you have the best of both worlds?

When Apple lets you take a still Spatial Photo, then that’s the signal that they’re confident in the image quality and not just the emotional content.

There’s no FOMO here, not right now, for Spatial Video. If the primary reason you shoot video is to remember moments with people, then make sure you keep that in mind. The people are more important than whether or not the video is 2D or 3D.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

October 26, 2023 9:00 AM PT

Is there in communication no beauty?

With the new iteration of flagship smartphone cameras comes the new iteration of arguments about reality—and not the “fun” kind that you strap to your face. Google’s Pixel 8 and Pixel 8 Pro offer a new generation of AI editing tools, and Apple’s iPhone 15 and 15 Pro integrate the latest round of Apple’s Photonic Engine¹ and Deep Fusion to process every last detail.

What we should really be talking about when we talk about these cameras isn’t some representation of “reality,” but communication. What a person shoots with any camera is not truth, but a timed amount of light from a selected angle shining through a lens focused onto a recording medium. Even without any editing tools, you can produce a photo that is real but isn’t true entirely in camera at the time it was shot.

Take a selfie outside the entrance gates to Disneyland, posted to Instagram. The image communicates the narrative that the subject spent the day at Disneyland—but that’s not what the photo shows, it’s an inference communicated by the context. There are no fancy generative AI tools changing the background. It was all recorded in camera. But whether or not the inference is true isn’t recorded in the photo at all—it’s one viewers make based on trust.

A person at a landmark wants their picture taken, they hand the phone to someone and the person takes two photos with slightly different framing. One has just the person with the landmark, and the other has the person and a rando with the landmark. The rando isn’t what anyone wanted to communicate to the viewer, and choosing the photo without the rando isn’t deceit. The two photos, one framed with the rando, and one without, are both real, and both true, but only one of them clearly communicates what the person wanted. If they only had the one photo with the rando, is it deceit to crop, or use a generative fill tool to remove the stranger?

The ultimate computer

Now that we’ve established that we can have real photos that aren’t true, let’s discuss fancy computer-assisted editing tools.

A group of three friends is taking selfies with their smartphones. They take three shots. A different friend is blinking in each shot. Each of those photos is true, in that the light was recorded, and that person blinked, but it is also true that there were moments that the friends were there with the camera and none of them were blinking, but it just wasn’t recorded by a camera. Is it a lie to combine two of the photos to communicate something true that simply wasn’t recorded?

A person with an iPhone takes a shot of city lights at night. Green dots from the internal reflections of the lens elements dance across the image. It is true that the camera recorded those green dots, and they physically happened in the lens elements, but is that what this photo should communicate? Are the internal reflections what the photo is about?

Google swapping heads in a group photo isn’t quite the problem that people seem to think it is. Apple having no proper retouching tools at all in Photos on the iPhone is a problem. If it’s based on a philosophical argument about the nature of things “looking real,” then it’s misguided as iPhone photos are an idealized result of heavy computational work anyway.

Apple’s goal should be to help people communicate clearly.

Even when we’re not making some big artistic statement with our iPhones, we might want to retouch something. I’ve taken photos of things on my desk that, after Photonic Engine and Deep Fusion did their work, showed dramatic contrast and detail… of the specks of dust on the table I hadn’t noticed before taking the shot.

Me carefully wiping off all the dust and retaking the photo won’t communicate something more pure about that photo, but the dust is a weird distraction. I’ve used third-party apps on iOS for many years just to remove those specks of dust, or those green dots. It’s silly that there’s no tool in the iOS Photos app to do it.

We’ll always have Paris

My issue with Google is that they have gone so far into generative editing tools that can produce strange technical artifacts in the final image which distract from what the image is supposed to be communicating. In Marques Brownlee’s video on the Pixel 8 and 8 Pro, he does some very unconvincing sky replacements and other edits with the Magic Editor. The same sorts of artifacts are present in DPReview’s article, where pieces of hair get chewed up, and in one case where a woman’s shoulder is replaced/caved in.

This is very similar to the problems introduced by Google and Apple’s first forays into post-processed blur, like floating liquid in wine glasses, or chunks of ears or eyeglasses missing. Those errors have become less glaring over time, but there are still errors. The problem here isn’t that people can alter their images, but that the alterations are sloppy. The photographer doesn’t have the skill or the eye to know there’s a problem in the final result they’re sharing.

The ones who will really be able to take advantage of the tools are the ones that will understand the situations in which the tools do and don’t work. Like framing a shot so a green dot from a bright light is over a solid-colored area of the frame, and not in someone’s hair, or an intricate pattern or edge. Framing a portrait mode shot to minimize any tangents with background objects and foreground objects. Then it really is an editing tool, and not just generative mush.

There is a concern that the general public will heavily edit all of their photos if the tools are easy enough to access. This is perhaps the least concerning part. People want to remember things that actually happened in their lives, and the desire to heavily edit everything is unappealing. They want to remember the ups and downs of their vacation to Paris. They want to remember loved ones as they were, or a night time stroll by the Louvre.

Context is for kings

There is a long, long history of altering photos and videos that goes back to the beginning of celluloid, even before we get to computers, digital cameras, or smartphones. Democratizing these tools, and making it so that technical errors are less obvious, consistently scares people that wish to assume everything they see is some kind of legal promise. We need to consider the context: the sources of the photos we see, and who’s sharing them with us.

Not all deception is malicious, either. As a visual effects artist, I’m a professional deceiver for entertainment purposes (which is a completely acceptable form of communication). The context is that you’re watching something categorized as entertainment. We don’t treat TV and films like historical documents, and we shouldn’t treat social media posts that way either.

The flood of malicious misinformation, with real consequences, is perpetrated by people that don’t need to use a Pixel 8 Pro, and aren’t stymied by the iPhone 15 Pro’s lack of retouching tools. If people are more aware of the tools that can edit photos, because they have them in their own hands, perhaps they will also be less likely to fall victim to deception.

We should all be receptive to what someone wants to communicate while knowing the context for that person, that medium, etc. This is especially true when something is inflammatory, and not just “look how great my life is” posts on Instagram. Not everyone is lying, but if you’re getting very emotional about something, seek out more information from trusted sources. Sharing something because of an emotional reaction to a photo without understanding if it’s true isn’t the fault of the image—that’s on the person sharing it. The sharer’s act of communication is spreading a lie. The editing tools didn’t lie.

A photo can communicate something that really happened, but the context for deciding whether or not it does exists beyond the photograph itself.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

September 21, 2023 6:53 PM PT

iPhone delivery anxiety

Apple works very hard to try and manage the massive preorder demand for day-one iPhone deliveries every year.¹ Apple originally allowed members of the iPhone Upgrade Program customers to set their orders up in advance, but eventually allowed everyone to pre-configure their phones before the official order time, leaving only the matter of the financial transaction to that Friday. Pre-approvals from financial institutions were started earlier than the Friday to try and prevent the rejections that happen when servers start melting down.

And yet, there’s still always some drama.

Some people live for that adrenaline rush of not knowing what will go wrong. Whether or not their orders for themselves, and family members, will all make it across that finish line. Oh, the stories they can tell about how they had the perfect Apple Store app force-quit workflow! Those people are living on the chamfered edge.

There are also people who were never in that group, or phased out of that group. The urgency isn’t there. A desire for a new phone still exists, but it doesn’t have to be the first day. Those people still might want their phone before December, though.

Both groups have to contend with the decision between shipping and in-store pickup. Here, Apple hasn’t been able to make the strides that it has with the ordering process. When purchasing an iPhone, the options are to preorder for delivery within a certain estimated window, or in-store pickup on a certain day at an appointed time. Both can wreak havoc on a person’s schedule, and have actually made me put off ordering an iPhone (and other Apple hardware) before, because I just couldn’t figure out if I would be able to receive the item.

What’s in store for you?

Being forced to pick your delivery method in your initial order can also lead to some regrets, since availability changes over time. For example, you might initially see a late October delivery window, but with in-store pickup happening on day one. Easy choice, or so you think! But once that initial pool of store iPhones is depleted, you won’t be able to select in-store pickup at all until that local store gets more iPhones in stock.

Store appointment pickup times also seem to suffer from cascading delays, especially during the first launch weekend. Apple will tell you that you have a 15-minute appointment window, which is laughably precise given how busy the stores can be. Your pickup could be prompt, or it could take three hours, which is what happened to a friend of mine.² You just have no idea if you’re going to get to the store and find an Apple employee that just hands you the phone and lets you leave, or if you’re going to wait in multiple lines and eventually check in at the door to the store only to be told that they’re running behind schedule.

However, if you wait until after launch because you want to reduce the chance of being stuck in the invisible DMV of an Apple Store’s overloaded appointment system, then you’ll be checking the inventory of every Apple Store near you every morning to see which models (and colors and storage options) have in-store pickup available. Then you’ll weigh that day’s schedule and guess how smoothly things will go. You might even avoid some stores based on past bad experiences. But you’ll never know until you get to the door and that Apple Store employee checks their iPad and reveals your fate.

Mail privilege

But Delivery is a bummer, too! Even though the majority of my time is spent at home, I truly have no ability to guarantee I will be able to reach my door for a delivery. When I worked in an office, there was still the off chance that the iPhone would be out for delivery on a Saturday. When I’ve needed to travel, I couldn’t redirect the order to a new location. And sometimes the delivery window slips back—or moves up—rendering your plan useless.

I wish Apple could offer something more like an Amazon Locker. Or at the very least, let customers ship to the store after the initial launch-palooza, and allow the customer to set an appointment time once the store had the device. Even Best Buy lets you do that! (But I don’t want to buy my iPhone from Best Buy. I want to buy my iPhone from Apple.)

Shipping to the store gives the safety and security of knowing that your phone is safe and nearby, but removes the pressure to bolt to the store when they’re going to be swamped.

Apple could even contact you if there was a delay with appointments at the store, so things could be rescheduled. And if a delivery window moves up, or slips back, Apple could ask the customer if they want to hold the original date… or change to in-store pick-up.

Many happy returns

The one really positive thing about this entire process is that trading in a device is mercifully easy to do. You can send your expensive, old iPhone into Apple for a undervalued discount on your new expensive iPhone without really feeling rushed, or that you’re putting your investment at great risk. I’ve heard a few stories of people having bad experiences with trade-ins, but not many.

The last time I did this, I had even missed the return window because of travel, and they… just sent a second return kit. No financial penalty incurred. Everything was smooth.

Getting a new iPhone causes me anxiety, when it should be a source of great joy. I’d love to see some creative retail solutions to smooth that out. Apple, please take my money—but don’t stress me out. I’ve got enough going on.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

August 22, 2023 5:01 PM PT

Shake it off: Remembering Apple’s Academy Award–winning failure

People are reminiscing about 25 years of the iMac, which is fine and all, but I’m doing to do something a little different. I’m reminiscing about 21 years of Apple’s Shake.

What’s that? You have no idea what I’m talking about? You just want candy-colored computers? Too bad. You’re going to learn something you’ll never need to know, whether you like it or not.

In February of 2002 Apple acquired Nothing Real, a software company based in Venice, California that made the industry-leading Shake compositing software. It was in that weird period where Apple was picking up steam, and trying to convert PC users to Mac users. There was no Shake version for the Mac, but there was for Linux, Windows, and Irix (RIP).

Just a few months after that acquisition, Apple released Shake 2.5 for the Mac (along with the existing supported platforms). Shake 2.5 would be the final version for Windows. Apple’s goal was pretty clear just from the press release—Shake cost half as much on Mac as it would on another platforms. (And this is a product that cost $10,000 per seat.) By Grabthar’s hammer, what a savings!

Apple was trying to convert high-end professionals, who would theoretically buy high-end Macs. As with Apple’s acquisitions of Final Cut Pro and Logic Pro, here was another pool of potential Mac buyers. (For context, this is the same year Apple released the second-generation iPod that added Windows support in an effort to get people to use Apple products in their personal lives.)

There were some problems with Apple’s strategy, though. While Apple managed to kill Shake for Windows, it could never do the same to the Linux version. That’s because all the big VFX houses were doing their work using Linux. It was easier to convert an editor or a small group of editors to Final Cut Pro than it was to pitch a corporation on converting hundreds of desks, and their render farms, to Power Macs or Mac Pros—even with the steep software discounts. (Not even Pixar!)

For Mac users, Shake never made any effort to fit in. It was born of an era where professional software tended to have its own custom interfaces so that the apps could run on a variety of platforms and work more or less the same. It never even adopted the Mac’s file open and save dialogs, buttons, or anything. It had bevels out the wazoo, which was the style at the time. It had some 3D-effect turquoise slider elements, but I wouldn’t call the interface “lickable.” Don’t lick the bevels, kids.

This custom UI was part of the reason it could be quickly ported to Apple’s Mac OS X Unix-compatible system from Linux and Irix, because no one needed to worry about how interface elements would be placed. It was literally the same interface.

Nodes and noodles

While Shake was not the first piece of software with a node-graph interface, it was arguably the most widely available at the time of its initial release.¹

The benefits of a node-based approach might not be clear at first glance. After all, when most people think of image editing, they think about it linearly: You open a file, you change some colors, add a blur, and save your file. (Or, alternately, consider a bunch of vertically stacked Photoshop layers.)

When it comes to video compositing, things get a lot more complicated. You might need to re-use the same element twice, or combine many elements instead of just editing one. And what if all your work needs to be done over a thousand frames of moving footage? A linear workflow just won’t do. You need to have the ability to branch and merge your work.

The node graph allows for the visual representation of complex, interconnected assets and adjustments. It allows for multiple inputs and outputs. It allows for reusing work, where the file being operated on can be swapped out for some other file while still applying the same edits.

This interface also helps you evaluate the output at any point along the line, including walking up or down the tree to see which node is causing a certain effect. This comes in handy when you’re working on version 100 and your client has decided that the thing you added 80 versions ago is something they want to take out. You can’t just keep pressing Command-Z—you need to alter just that one choice.

This was real non-destructive editing. And it was powerful.

It’s also important to note that this interface wasn’t a stack (like Apple’s Shortcuts, or Photoshop’s History palette) that runs from top to bottom. It was a web of nodes, all connected to one another. Shake visualized these connections between nodes with noodles—not straight lines, but a curving piece of spaghetti. (This is whimsy, as manifested by the developers of high-end professional software.) If you preferred straight lines, and you could straighten out those noodles by adjusting a preference called Noodle Tension. (I’m not making that up.) As for me, I’m a monster. I love my curvy noodles.

Everything flowed logically. Nodes that generated images didn’t have a little input bump on the top of the node, because they didn’t accept input, only generated output. Nodes that did operations had at least one input and one output. Any time there was more than one input or output, there would be a discreet and separate connection point for each so that things didn’t all collide in one spot. Each of these nodes also had a mask input, where any kind of alpha could be connected to limit the area that the node was editing. It was sort of like a Clipping Mask in Photoshop, but way easier to reuse and adjust for multiple nodes at once.

Shake even had a pretty comprehensive set of paint tools, where paint strokes were all saved nondestructively. There were warping tools that were used to create talking animals. All those things were there, editable, non-destructive, and portable.

And if you ever got a collection of nodes that you used a lot, you could group them and save them as a macro, which would appear as a single node. This was a great way to save time and reduce visual complexity.

The namesake feature of Shake was that if you wanted to remove a node and take it somewhere else, instead of disconnecting every noodle going in and out of it, and reconnecting those up and downstream, you could just hold down on the mouse button, and shake the node around until it until it was released. It wasn’t something you needed to do a lot of the time, but when you did do it, it was charming (and you were very relieved you didn’t need to reconnect things).

Multiple endings for Shake

From Apple’s March 2004 press release on Lord of the Rings: The Return of the King winning the Academy Award for Best Visual Effects in a Motion Picture:

“We’re thrilled that for seven years in a row, movies created with Shake have won the Oscar for best visual effects,” said Steve Jobs, Apple’s CEO. “Shake is helping Hollywood film editors communicate their vision and deliver their art at an Academy Award winning level. We couldn’t be happier.”

It’s always been weird to me that Steve Jobs referred to them as editors, but maybe that’s because of how Shake was often positioned as a companion to Final Cut. (It didn’t work anything like Final Cut, but whatever!) If you were a small shop that had Final Cut Pro and needed to do a few effects, than it was great to have this class-leading software available to you, even if you weren’t making a “Lord of the Rings” movie.

But Shake just didn’t pan out the same way Apple’s purchases of Final Cut Pro and Logic Pro did. While it was wildly popular, award-winning software, it was only ever wildly popular and award-winning on other platforms. The last major version of Shake was released in 2006, with a minor update was released in 2008. Shake’s price was slashed and slashed again over its lifetime, with the final offer being a $50,000 site license so a VFX house could intall it on as many Linux boxes as they could muster. (That would have been five Linux workstation licenses under the original Shake pricing.)

This meant that I got to use Shake once again at Imageworks on Watchmen, including a wild day and night where I was working in a Shake script on Dr. Manhattan, and then copying and saving those changes to the main comp script where the plate photography was being stitched together for a bunch of mobsters exploding. The other compositor would tell me when he was done, and I’d copy more nodes and adjustments to the file. Back and forth. The workflow was still powerful, even if it hadn’t seen active development in three years.

Shake was unceremoniously discontinued in 2009. Apple wasn’t the company trying to convert visual effects houses to buy cheese graters any longer. It had become a company printing money from selling iPhones. Shake’s developers were moved on to other projects, or departed the company altogether.

I know there was never a financial case for Apple to have continued development of Shake. But it’s sad to see that misguided acquisition kill a product that was formative for me, and others. (Christa Mrgan mentioned it as an influence on Audio Hijack 3’s redesign.

Fortunately, node-based compositing didn’t die with Shake—far from it. Foundry helped “productize” Digital Domain’s Nuke just as Shake died, and that’s now the industry standard.

Apple was the wrong company to acquire Shake and the wrong company to attempt to be an industry leader in tools for visual effects artists. Sadly, Shake’s legacy isn’t about those smart nodes and fun noodles—it’s as a cautionary tale about misguided acquisitions.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

August 1, 2023 8:52 AM PT

57 app icons and nothing good on

Every year I start my annual ritual of installing the tvOS beta on the secondary Apple TV. Not the one in the living room—I’m not crazy—but the one in my office.

This year brings on the biggest, boldest change in tvOS since the release of dark mode in 2017… and that change is scaling down the Home Screen icons so there’s one more app icon per row.

They said it couldn’t be done, but they did it. They increased information density in an Apple product.

The other big changes in tvOS 17 feel like they’ll have more limited appeal. VPN support will help people “visiting” other countries to watch their “local” TV. Adding FaceTime support through continuity camera is something that benefits parent-grandparent conversations on comfortable sofas and people in the one conference room connecting to the people in the other conference room. That kind of stuff.

The high-traffic navigation areas of the Apple TV interface are the Home Screen and the TV app. But why do we still have a Home Screen and a TV app? One is for browsing services, and the other is ostensibly for quick access to media and media discovery that’s relevant to you. On other platforms these things can be combined, so you have a row of favorite apps next to your watch next queue.

But other than icon size, tvOS 17 hasn’t improved anything about the most vital part of the OS: the part that lets us watch TV.

In the beta, the Apple TV app behaves exactly like it has since Apple’s ill-conceived sales-bro-esque revamp of the TV app last October. Since it’s much maligned introduction, it has been slightly tweaked 16 to keep the Up Next row at the bottom of the screen, instead of pushing it off to require a swipe-scroll down.

Apple has also made tweaks to the editorial content of that top carousel of recommendations: It’s still random stuff, including seasons of shows Apple knows I have already watched in their entirety—but now the first 23 items in the carousel are from Apple TV, when previously they were more of a 50/50 split between Apple’s own stuff and the rest of the content on the platform.

While using this beta, I realized that I hadn’t used the TV app in a while. It turns out that Apple’s changes — adding autoplaying video and requiring all those extra swipes — have driven me away entirely. I’ve gone back to just using the Home Screen to launch the app I need, like an animal. (Maybe developers at Apple have changed their behavior too and that’s why we have more Home Screen icons?)

Here’s a pro tip for people on tvOS 16 or 17: Go into Settings: Apps: TV and change “Top Shelf” from “What to Watch” to “Up Next.” Then the top shelf row of the Home Screen will show you your shows without having to open the TV app at all.

Impersonal personalization

Apple’s personalization features are still largely useless if you live in a household with two adult human beings. We watch TV together, and share accounts that are registered to him or me. Apps don’t seem to have done anything with the feature. There’s a new ability for Siri to recognize “up to six” family members voices and recommend content for them, but how relevant can that feature ever be if most viewing is a shared experience or through shared apps that don’t use Apple’s profiles? And wouldn’t it be better to personalize those ridiculous content carousels in the TV app rather than forcing us to than ask Siri what to watch?

My “What should I watch” yielded Alone, Avatar: The Way of Water, Shoresy, Guardians of the Galaxy Vol. 3, The Orville, John Wick: Chapter 4, Yellowjackets, Raiders of the Lost Ark, Schitt’s Creek, Patriot Games, Blue Bloods, Fantastic Beasts: The Rise of Dumbledore, Survivor, Black Adam, Modern Family, Fringe, Men in Black, The Golden Girls, Pirates of the Caribbean: The Curse of the Black Pearl, The Marvelous Mrs. Maisel, Total Recall (the bad one), NCIS, The Suicide Squad, Bob’s Burgers, The Untouchables, So Help Me Todd, Cloverfield, Justified, Alien Code, It’s Always Sunny in Philadelphia, and Saving Private Ryan. None of those are Apple TV+ titles because—unlike the TV app—Siri seems to know what services I’m subscribed to, and I’m not a current Apple TV+ subscriber. Again, why the inconsistency?

That list doesn’t actually seem personalized, either. I have watched several of those movies, and I’ve also watched an episode of some of them and then stopped watching because it wasn’t for me. It just seems like these results are matched to a generic cis-hetero dad who’s open-minded about progressive issues. Not that there’s anything wrong with that, but that is not me.

Also, this list is not the same as the very, very, very buried “For You” row in the TV app. It has overlap, with dad energy, but it’s not identical, and still doesn’t have an understanding of things I’ve watched before. In neither system can I long-press to express that I’ve already watched something, or don’t like the suggestion.

I’m smart, not like everybody says

Apple has no outward facing improvements to smart home stuff via the Apple TV, which is weird, because it’s one of the two types of devices that operate as a hub for all the smart home devices you have. The Matter future that we were promised just hasn’t materialized. Perhaps the silence means that Apple is regrouping on how to handle this, but in the meantime, what it mostly means is that the smart devices connected to my office Apple TV stopped working the other day.

The tvOS Control Center has been refreshed, mostly to shrink it down. The previous version hid smart home stuff behind a tap—but now it’s hidden behind a swipe. I don’t get it. This isn’t a Casio watch—the whole TV is right there, offering plenty of space for buttons. Apple is, in fact, using that space to cram more buttons on the Home Screen. Yet home stuff is buried.

Also, why doesn’t tvOS have widgets? Why can’t I put a smart home widget, or a weather widget, on my Home Screen or in Control Center? Apple has put widgets just about everywhere else—including Stand By on the iPhone, which is all about ambient information on a display in a household.

Beta Luck Next Year

We’re not likely to see new features announced for Apple TV between now and next June, which is a bummer. tvOS 17 avoids a lot of obvious areas for improvement, especially in regard to it as a connected home TV viewing platform.

The thing that should absolutely be tweaked—on all of Apple’s platforms—is the content displayed by the TV app. (Given how much emphasis has been given to Apple TV+ promotion, that seems increasingly unlikely.)

Apple also can’t keep kicking the can on addressing the conflict between the Home Screen and the TV app. (It can’t, right?) So I hope we’ll see something sooner rather than later. But in this case, “sooner” still probably means a year from now.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

June 26, 2023 9:00 AM PT

Vision Pro and the challenge of 3D movies

No matter what you think of the Vision Pro headset or 3D movies, it’s become apparent over the last few weeks that a lot of people need a primer on 3D, stereoscopic movies. Love them or hate them, there’s no escaping that they’re going to be a subject of conversation again, just as they were more than a decade ago.

Captive audience

Back in the 2000s there was a push to increase movie ticket prices without making major alterations to seating. Stereoscopic movies were an interesting possibility. Sure, they were more difficult and expensive to make, but the advent of digital projectors meant that theaters could be adapted to show them relatively easily. And of course, a 3D blockbuster with impressive visual effects that would give audiences a reason to pay a bit more.

Most 3D theaters are set up with a digital projector and a polarizer set up in front of the projector from a company called RealD. Left and right images would be projected onto the screen at the same time, and special glasses worn by the audience would filter the polarized light to display separate images in both eyes. It’s the same principle as polarized sunglasses or circular polarizer filters for cameras. (Extremely bright or contrasty parts of the image might bleed through from one eye to the other, creating “ghosting.”)

The problem with this approach is that the single projector can still only output an image at its maximum brightness, which is then cut in half by the polarizing glasses. The result is that 3D movies often seem dim. There are also the gross plastic glasses, which would also have to fit over any prescription lenses you might need to wear.

So to recap: They wanted you to pay more in order to see a movie that wasn’t as bright or clear, and you’d need to wear some weird glasses for the privilege.

Continue reading “Vision Pro and the challenge of 3D movies”…

By Joe Rosensteel

May 17, 2023 3:33 PM PT

Software subscriptions feel weird, but they work

With the recent announcement of Final Cut Pro and Logic Pro for the iPad we have renewed the same, silly argument about subscription pricing that happens far too often. Without having the apps in hand (literally) there’s very little to say in the way of assessing the value offered by the subscriptions at this time, but there’s plenty to say in favor of recurring payments from behind the safety of this paywalled post (thanks for subscribing).

When Adobe switched to Creative Cloud pricing, there was a lot of sturm and drang, and while it hasn’t been all rainbows and puppies it’s largely worked out for all involved. Yes, Adobe executives have probably bought themselves a nice vacation home, but Creative Cloud has been a success for me, personally. I use Lightroom for my photo hobby, I use Premiere for cutting my demo reel every so often, and I use Audition for editing podcasts.

Let’s unpack why I’m satisfied paying Adobe every month:

1. The barrier to pay Adobe’s monthly rate was far lower than paying the the upfront cost for Logic Pro and Final Cut Pro in a time where I was uncertain how many months, let alone years of use, I would get out of the apps.
2. Adobe releases predictable updates. I felt particularly burned by Apple as a loyal Aperture customer, and as someone who used Shake for my jobby-job. Without much warning, someone at Apple decided it wasn’t worth it to make these apps. Would they do that again, to their flagship pro software? Maybe. Aperture and Shake won prestigious awards. Prestige isn’t the whole story.
3. Familiarity was also important, and I had experience with Premiere from college, so for cutting demo reels it was easier than having to learn Final Cut Pro. Likewise, Audition’s sort-of similarity to Premiere meant it was a more natural fit with my existing experience.

My credit card was put on file, software was downloaded, and Adobe drops in some new stuff every now and then. There’s no major concern about upgrading to a version of macOS and losing my software. (Although Adobe is not always timely with addressing some compatibility issues, they do get there eventually—and I know better than to download a new release of macOS on day one.)

The Creative Cloud a desktop app that pesters me about Adobe stuff is bad, but the underlying service itself? It works as expected. I can turn off auto-updates, or download old versions if I need something for compatibility.

I could have spent less money by knowing ahead of time how wildly successful podcasting would be for me, or by cutting that occasional reel update with a single purchase of Final Cut Pro, but no one really knew how many years of free updates Logic Pro and Final Cut Pro would receive. (I’m not even sure Apple has ever had a clear picture. It mostly just seemed like they didn’t want to add paid upgrades to the App Store so they sheepishly just kept updating the one they sold.) It is an aberration in the world of selling software that Apple has kept investing in these two pro apps with no income from the existing user base.

My point about familiarity might not seem that important when it comes to talking about subscriptions but it absolutely is. Kids generally can’t afford to pay for their own pro software. Back in the day, they might have used cracked codes for downloaded software, but generally their access to pro software would be through the world of higher education. A five-dollar expense makes sure little Braden or Madison is predisposed to consider Apple’s pro software.

The only weird thing is that the iPad announcements didn’t come with any Mac announcements about pricing. Surely, at some point they will need subscription pricing too. It’s been over a decade of free updates for these apps. Deeply unserious people might continue to argue in favor of free updates for apps for eternity, subsidized by RAM and SSD prices from pro Mac sales, but this is an unhealthy business model. It also suppresses competition in the pro app market.

Ideally we should live in a world where if you’re unhappy with your pro software you can cancel your monthly subscription and subscribe to another pro software solution. If you need a specific piece of software for a project as a freelancer, you’re just paying for the timeframe you’re working on that project.

It’s far more predictable to have a monthly recurring fee to estimate your expenses than it is to try and figure out when large sums of money should be plunked down and if you’ve timed your purchase with this year’s NAB conference, and out-of-cycle updates. Particularly if you’re an institution purchasing volume licenses.

Now, one fear that even some sensible people have is this: We might one day find ourselves in a situation where the subscription prices just keep ticking up, or that the software you rely on will be placed inside an expensive bundle of software you don’t need. But those are all fears that could be true in a world of software license purchases, too! Which is exactly why it’s important to have a variety of software vendors with sustainable revenue streams, instead of just Adobe which has the pro bundle of all pro bundles, or Apple which hasn’t seemed all that motivated by drops or gains in app sales.

Perhaps the presence of Apple’s pro apps on the iPad will nudge Adobe to compete for dollars there as well. Other than Lightroom, and now Photoshop, Adobe has only made overly simplified apps for iPad. There’s an Adobe Premiere Rush app, but it’s not Premiere, and there’s not even a toy version of Audition. If Apple just offered their Pro apps for free on the iPad, I doubt Adobe would ever even consider trying harder.

One final point: The App Store is not a free market. Adobe is big enough that it gets to sidestep the App Store with a Creative Cloud login that you acquire on adobe.com, but not a lot of developers get that benefit. If someone has to compete with Apple in the App Store, they do so at an extreme disadvantage for either sales or subs. That dramatic imbalance is really why we should rise up and overthrow our pro app oppressors.

But until then, I’ll just let Adobe charge my credit card so I can focus on getting work done.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]

By Joe Rosensteel

April 19, 2023 3:27 PM PT

Have you considered using a camera?

Look, there’s no interesting Apple news. We’re all waiting on tenterhooks for the big hardware announcement so we can find out the only thing anyone cares about: Will the 15″ MacBook Air come in a chip configuration that supports two external displays? While we wait for that—and only that—to happen, we might as well take advantage of this spring weather and go outside to shoot some photos. Snap some pics. Record those photons.

I’ve been on a little bit of a camera hardware kick, as you could tell from my last, thrilling post about tracking camera information with spreadsheets. But something also seems to be in the air (along with the pollen), because the fellas over at the Accidental Tech Podcast have been talking about cameras for a few weeks. Additionally, there’s a simmering dissatisfaction that people seem to have about the over-processed nature of their iPhone photos, and some hand wringing about iPhone camera modules for this September.

Perhaps it’s a good time to take a step back. Rather than just thinking about photos, what about taking a bunch of ’em? Open that closet and get out that neglected SLR, DSLR, or point-and-shoot from yesteryear. This isn’t just some sort of luddite “it used to be better” thing. This is also to make you appreciate what a magical little device your iPhone is. I am deeply uninterested in pitting the iPhone against other cameras, but feel the need to remind people that they’ve got some options in their life, and there are no wrong answers. (The images I’ve included in this story are unlabeled, resized to typical social media resolution, and stripped of metadata.¹)

For the purposes of this post, I’m just going to talk about cameras that take SD cards, as that is basically 20 years worth of cameras, and SD cards are the easiest things for most people to deal with. You can spend $100 to buy and develop a single roll of Kodak Gold if you want that warmth two to three weeks from now, but let’s try for some immediate gratification first.

Charge the battery, pop in a SD card, and have an SD card slot or adapter ready to receive your photos. Personally, I recommend making the investment in Apple’s Lightning to SD Card Camera Reader. You can, of course, use any other SD card reader you’re comfortable with, but I find that having the reader at my side is easiest. Don’t waste your time with Wi-Fi apps—they’re always a terrible experience, and making the attempt will ruin your fun photo adventure.

But Joe, you say, won’t an old camera have—gasp—a low resolution? Yes. However, you’re not using these images to print a large-format photo mural, and up until the iPhone 14 you were only working with a 12 MP sensor, anyway. Chances are good that you’ll uploading the end result to a failing social-media company to data mine details of your life from, and they’ll show the images at much lower resolutions. Even early six megapixel cameras can get good images—you just can’t crop in a lot. Lower than six is a little iffy, but gen Z would say it’s very aesthetic. Some people really lean into what are generally considered to be defects.

Prime time to kit zoom

If you’re taking a camera out with you, odds are it either has a built-in zoom lens, or an interchangeable lens mount that came with at least a kit zoom lens. Maybe you also picked up some other lenses once upon a time. I find, personally, that it’s helpful to at least have a zoom lens with you. Just bring two lenses at most. You’re not trying to take up juggling.

Another reason I recommend bringing a zoom is to do a little back and forth comparison with your iPhone, and it’s the fastest way to hop around a range of focal lengths. It can also help you get an idea of what you do and don’t like about your lens(es). You can quickly google your iPhone model, and “35mm equivalent” to get the answers you seek (or divide the 35mm equivalent number by the crop factor for your sensor, such as 1.5 for APS-C), but here’s the info for the iPhone 13 and 14:

• iPhone 13 “wide” lens: 26mm on a full frame (35mm) camera, 17mm lens on an APS-C sensor camera
• iPhone 14 “wide” lens: 24mm on full frame, 16mm on an APS-C sensor camera.
• iPhone 13 Pro and 14 Pro “telephoto” lens: 77mm for full frame, 51mm for APS-C
• iPhone 13 Pro and 14 Pro “ultra-wide”: 13mm on full frame, 8mm² on APS-C.

Again, this isn’t about generating an A/B comparison library to quiz your friends with. It’s just to build an personal understanding of how things translate.

On a later date, move on to the lenses that can do things your iPhone can’t do. Like the 70-350mm lens for my Sony that is equivalent to a 105-525mm zoom on a 35mm camera. Great to take photos of birds, or the moon with. Eat your heart out, Samsung.

Get in the zone

Get in the zone Auto Zone by walking around with your hand holding the camera grip, not just letting it dangle from your neck and bounce off your stomach every time you take a step. Head somewhere with some flowers, interesting shadows, varied textures, something that makes you think “what if I drew a little rectangle around that and it was a photo?” This might trigger some muscle memory from the last time you used that old camera.³ There’s no shame in auto modes, either, if you’re a little rusty—but try and go back and see what settings the camera picked and whether you agree with it.

This is where plugging in the SD card to your phone can come in handy in 2023, because you can more accurately assess the image the camera took on an iPhone screen. All camera screens are garbage, even new ones, when compared to the iPhone’s display. Your iPhone will give you the most accurate view of what you shot. When you get used to a camera, you can use the bad display to make relative judgments, but don’t assume that your photo will exactly look like that. This is very true when it comes to evaluating lens glare/flaring which might not look like much on the crappy display.

Shoot a couple shots with your iPhone too. Any photo you take will have metadata, and you can apply that to your older camera’s shots that might not have location data or accurate time. Also, while shooting, consider using one of the many third-party camera apps (like Halide or Obscura) to make the same kinds of shooting adjustments you were doing on your camera.

The aperture stuff, as it relates to bokeh, is going to be different, but it’s going to be the same as it relates to light hitting the sensor. The blur is based on distance of the object to the sensor, and sensor size, not just the amount of light coming in. This is why Apple invented Portrait Mode. Feel free to shoot with that too. Don’t take photos of wine glasses, and while it’s gotten better at handling some subjects, it’ll still cut off an ear or do weird stuff with drinking straws, even though it’s not as bad as before.

You’ll also notice, after being in the zone, that even at relatively similar exposures your images will have different dynamic ranges and tonality. The iPhone does some tricks to boost certain subjects, and drop those blown-out skies… and your old-school camera is probably not doing any of that. Also, your camera might have a larger sensor than the iPhone, but that sensor might be very old, limiting the potential range between the brightest brights and darkest darks it can record. Images may wind up looking more like slide positive film, but with an unpleasant highlight roll-off. Try shooting RAW and exposing for the highlights in your image (or using exposure compensation to force it down a notch), then take up the darkest parts of your scene in post if you want. Camera companies also added adjustment features to help boost the darkest areas too in-camera. Look up things like Nikon’s Active D-Lighting, Canon’s ALO, or Sony’s DRO, etc.

Lower range can also be something you can lean into in creative ways, like purposefully silhouetting subjects, or letting shadows completely fall away. Your iPhone will try to make everything a bright, even tone with some contrasty edges and highlight pops. Sometimes it looks a lot like what your own eyes are seeing, and other times it can appear a little boosted. But maybe what you really want is just to direct the viewer’s eye with your exposure—in which case, you’ll have to use a third-party app to control that exposure. I do wish the default camera app had exposure compensation.

Working for your work of art

The big thing that you’ve probably noticed from shooting stuff back and forth between the camera and the iPhone is just how much more effort and thought needs to go into your camera shots. Even if you’re using a third party iOS camera app, it’s still doing a lot of the heavy lifting for you. Your camera, mostly as a product of its age, is going to be hella slow.

It also means having to manage a lot of settings that you weren’t previously thinking about, and dealing with files you didn’t have to bother with before.

Ask yourself if any of it would be improved if you changed something about that camera, like a different lens, or getting more familiar with settings and modes. Maybe it’s the circumstances you used it under. Was it daylight or night? Is it a crappy underwater camera that you should only use at the beach? Do you like everything about the lens and shooting experience but wish autofocus was snappier? There are solutions to these problems, sometimes even with relatively inexpensive used equipment from KEH, MPB, or the riskier eBay. Maybe just rent something from Lens Rentals.

Even if you put the camera back in the closet, I hope this experience has reminded you why the iPhone is such a popular camera, and why every year people clamor for even incremental advances in the ever-deepening camera module. Despite objections about “over-processing” images, it’s doing a lot of work you’d manually have to do. For being a camera for everyone, and every situation, it has to put on one hell of a show.

But sometimes it can just be a little fun to shoot with something else on the side.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]