The killer smartphone camera feature that we all want, but may never get

I’ve been a longtime fan of shooting in portrait mode. It was popularised by the iPhone 7 Plus all the way back in 2016, but it was with the Huawei Mate 20 Pro a few years later that I first got to play with it – and I’ve been hooked ever since.

To be clear, I’m specifically talking about the artificial background blurring (bokeh) effect that’s applied to photos when shooting in this mode. These days, all kinds of features are lumped into portrait mode, but for the purposes of this article, I’m just focusing on the shallow depth of field effect.

This nifty feature makes your photos look like they were taken on a camera with a much larger sensor, like a mirrorless camera, rather than a teensy smartphone camera. And while we’re starting to see larger sensors on smartphones like the Vivo X90 Pro and Xiaomi 13 Pro, the sensor size required to take the photos that portrait mode emulates just wouldn’t fit into a pocketable smartphone.

It’s an ingenious solution to the problem, using depth calculation to separate the foreground from the background, and then applying an effect that simulates natural lens blur so accurately that it’s almost impossible to tell apart from professional camera snaps at a glance.

That is, when it gets the depth calculation right. When the subject cut-out is slightly off, the effect is quickly ruined, and this happens a lot more often than I’d like.

This could be down to challenging lighting, the phone being unsure of which people we want in focus or the image processing algorithms just not being optimised for the type of subject that you’re shooting.

Of course, some phones have better subject detection than others, but all systems have their strengths and weaknesses. For example, Google Pixel phones seem to have a hard time separating my glasses from the background, whereas Vivo phones do a great job with my glasses, but can’t ever get my headphones right. The spokes of a bicycle seem to be impossible for every phone that I’ve tested.

None of this would be an issue if you could go into the image and edit the subject cutout, but I’ve never come across a phone that will let you do this. Some will let you adjust the amount of blur after the fact, and some will even let you shift the focal plane, whereas others will just bake the effect into a JPEG and you’re stuck with whatever settings you chose before you pressed the shutter.

A particularly bad example of subject separation from the old Huawei Mate 20 Pro

This makes shooting in portrait mode inherently risky, if you take a once-in-a-lifetime snap and the phone messes up the subject separation, that picture is potentially ruined for good.

Implementing an editing tool that lets you refine the edges of your subject wouldn’t be hard to do, there are already much more advanced editing options available on smartphones – like Google’s Magic Eraser, for example. But, I don’t think we’ll be seeing it anytime soon.

The reason we haven’t seen this feature yet, at least in my opinion, is that adding it would mean that a smartphone manufacturer is admitting that their subject-detection algorithms are less than perfect – and no one wants to say that.

Instead, I believe we’ll just have to wait for the algorithms to get better, until they’re so reliable that this issue becomes essentially non-existent. With the recent advancements in AI and machine learning, it’s not unrealistic to think this will happen in the near future. On the other hand, with smartphone sensors getting larger and larger, maybe hardware advancements will mean that we just start carrying professional cameras in our pockets.

Time will tell how it all plays out, but the frustrating part is that it could be so much better right now, and all it would take is for one brave smartphone manufacturer to add the ability to adjust the subject cut-out.