By Joe Rosensteel

October 26, 2023 9:00 AM PT

Is there in communication no beauty?

With the new iteration of flagship smartphone cameras comes the new iteration of arguments about reality—and not the “fun” kind that you strap to your face. Google’s Pixel 8 and Pixel 8 Pro offer a new generation of AI editing tools, and Apple’s iPhone 15 and 15 Pro integrate the latest round of Apple’s Photonic Engine¹ and Deep Fusion to process every last detail.

What we should really be talking about when we talk about these cameras isn’t some representation of “reality,” but communication. What a person shoots with any camera is not truth, but a timed amount of light from a selected angle shining through a lens focused onto a recording medium. Even without any editing tools, you can produce a photo that is real but isn’t true entirely in camera at the time it was shot.

Take a selfie outside the entrance gates to Disneyland, posted to Instagram. The image communicates the narrative that the subject spent the day at Disneyland—but that’s not what the photo shows, it’s an inference communicated by the context. There are no fancy generative AI tools changing the background. It was all recorded in camera. But whether or not the inference is true isn’t recorded in the photo at all—it’s one viewers make based on trust.

A person at a landmark wants their picture taken, they hand the phone to someone and the person takes two photos with slightly different framing. One has just the person with the landmark, and the other has the person and a rando with the landmark. The rando isn’t what anyone wanted to communicate to the viewer, and choosing the photo without the rando isn’t deceit. The two photos, one framed with the rando, and one without, are both real, and both true, but only one of them clearly communicates what the person wanted. If they only had the one photo with the rando, is it deceit to crop, or use a generative fill tool to remove the stranger?

The ultimate computer

Now that we’ve established that we can have real photos that aren’t true, let’s discuss fancy computer-assisted editing tools.

A group of three friends is taking selfies with their smartphones. They take three shots. A different friend is blinking in each shot. Each of those photos is true, in that the light was recorded, and that person blinked, but it is also true that there were moments that the friends were there with the camera and none of them were blinking, but it just wasn’t recorded by a camera. Is it a lie to combine two of the photos to communicate something true that simply wasn’t recorded?

A person with an iPhone takes a shot of city lights at night. Green dots from the internal reflections of the lens elements dance across the image. It is true that the camera recorded those green dots, and they physically happened in the lens elements, but is that what this photo should communicate? Are the internal reflections what the photo is about?

Google swapping heads in a group photo isn’t quite the problem that people seem to think it is. Apple having no proper retouching tools at all in Photos on the iPhone is a problem. If it’s based on a philosophical argument about the nature of things “looking real,” then it’s misguided as iPhone photos are an idealized result of heavy computational work anyway.

Apple’s goal should be to help people communicate clearly.

Even when we’re not making some big artistic statement with our iPhones, we might want to retouch something. I’ve taken photos of things on my desk that, after Photonic Engine and Deep Fusion did their work, showed dramatic contrast and detail… of the specks of dust on the table I hadn’t noticed before taking the shot.

Me carefully wiping off all the dust and retaking the photo won’t communicate something more pure about that photo, but the dust is a weird distraction. I’ve used third-party apps on iOS for many years just to remove those specks of dust, or those green dots. It’s silly that there’s no tool in the iOS Photos app to do it.

We’ll always have Paris

My issue with Google is that they have gone so far into generative editing tools that can produce strange technical artifacts in the final image which distract from what the image is supposed to be communicating. In Marques Brownlee’s video on the Pixel 8 and 8 Pro, he does some very unconvincing sky replacements and other edits with the Magic Editor. The same sorts of artifacts are present in DPReview’s article, where pieces of hair get chewed up, and in one case where a woman’s shoulder is replaced/caved in.

This is very similar to the problems introduced by Google and Apple’s first forays into post-processed blur, like floating liquid in wine glasses, or chunks of ears or eyeglasses missing. Those errors have become less glaring over time, but there are still errors. The problem here isn’t that people can alter their images, but that the alterations are sloppy. The photographer doesn’t have the skill or the eye to know there’s a problem in the final result they’re sharing.

The ones who will really be able to take advantage of the tools are the ones that will understand the situations in which the tools do and don’t work. Like framing a shot so a green dot from a bright light is over a solid-colored area of the frame, and not in someone’s hair, or an intricate pattern or edge. Framing a portrait mode shot to minimize any tangents with background objects and foreground objects. Then it really is an editing tool, and not just generative mush.

There is a concern that the general public will heavily edit all of their photos if the tools are easy enough to access. This is perhaps the least concerning part. People want to remember things that actually happened in their lives, and the desire to heavily edit everything is unappealing. They want to remember the ups and downs of their vacation to Paris. They want to remember loved ones as they were, or a night time stroll by the Louvre.

Context is for kings

There is a long, long history of altering photos and videos that goes back to the beginning of celluloid, even before we get to computers, digital cameras, or smartphones. Democratizing these tools, and making it so that technical errors are less obvious, consistently scares people that wish to assume everything they see is some kind of legal promise. We need to consider the context: the sources of the photos we see, and who’s sharing them with us.

Not all deception is malicious, either. As a visual effects artist, I’m a professional deceiver for entertainment purposes (which is a completely acceptable form of communication). The context is that you’re watching something categorized as entertainment. We don’t treat TV and films like historical documents, and we shouldn’t treat social media posts that way either.

The flood of malicious misinformation, with real consequences, is perpetrated by people that don’t need to use a Pixel 8 Pro, and aren’t stymied by the iPhone 15 Pro’s lack of retouching tools. If people are more aware of the tools that can edit photos, because they have them in their own hands, perhaps they will also be less likely to fall victim to deception.

We should all be receptive to what someone wants to communicate while knowing the context for that person, that medium, etc. This is especially true when something is inflammatory, and not just “look how great my life is” posts on Instagram. Not everyone is lying, but if you’re getting very emotional about something, seek out more information from trusted sources. Sharing something because of an emotional reaction to a photo without understanding if it’s true isn’t the fault of the image—that’s on the person sharing it. The sharer’s act of communication is spreading a lie. The editing tools didn’t lie.

A photo can communicate something that really happened, but the context for deciding whether or not it does exists beyond the photograph itself.

[Joe Rosensteel is a VFX artist and writer based in Los Angeles.]