Blurring the background: Understanding focus, defocus, refocus and Ufocus

Your HTC One M8 isn't a Lytro, but that doesn't mean you can't create some neat-looking effects

Background defocus effects are the hot new feature for smartphone cameras this year. Sony, Nokia, LG, Samsung and HTC are all offering DSLR-like bokeh effects in photos through software trickery. These effects have even been likened to the Lytro, the first mainstream light-field camera, which lets users refocus their photos after capture.

Instant after-the-fact refocusing from a smartphone camera isn't truly upon us just yet, however, and each manufacturer's approach to defocusing and refocusing shots after taking them has its individual quirks. That includes the HTC One M8's Ufocus feature, which while capable of producing some striking depth-of-field effects, has its limitations. And the same goes for single-camera refocusing effects from competitors like the Samsung Galaxy S5, LG G Pro 2 and Sony Xperia Z2.

Let's take a closer look after the break.

First of all, it's worth underscoring the differences between traditional digital cameras like the one in your smartphone and a light-field camera like the Lytro. Whereas most cameras capture the color and intensity of light at given points, the Lytro is also able to measure the direction of light at a given point. It does this using a matrix of microlenses over its sensor, and from this the device's onboard processor can generate images with different things in focus. Needless to say, that's way beyond what's currently possible in a smartphone camera.

If you want to emulate this kind of effect on a phone, you need to work around the fact that you've only got a traditional lens-and-sensor combo — not to mention a very small amount of space in which to work. The latter eliminates the possibility of creating DSLR-like bokeh effects the old-fashioned way. But fortunately what a high-end smartphone lacks in fancy optics it usually makes up for in processing grunt, meaning computational photography can pick up some of the slack.

The most common approach to defocusing or refocusing photos on a phone is to combine exposures with different focus points between the foreground and background — and this is what Sony, Samsung and Nokia do. (It's similar to the way that HDR shots combine longer and shorter exposures into the same image.) The camera app then does some number crunching and combines these far and near-focused exposures into a single interactive image, while perhaps overlaying a tasteful blurring effect over areas of the photo deemed to be out of focus.

Sony's Background Defocus app is the most limited of the bunch, doing exactly what the name suggests and keeping the foreground in focus — from there, you can add various levels of blur to the background. Samsung's selective focus mode on the Galaxy S5 is — well — more selective. After rapid-firing a bunch of exposures, the GS5 lets you choose to focus on the background or foreground, or enable pan focus, which aims to keep the entire shot in focus.

Meanwhile Nokia has a more Lytro-like setup, at least from the perspective of the images it produces. Like Samsung and Sony, the Nokia Refocus camera app for PureView devices captures images with varying focus points over a couple of seconds, then lets you tap anywhere on the image to "refocus" it. (Or tap "all in focus" to bring everything back in focus.) You're still somewhat limited by the finite number of exposures that are captured, but it does give you more fine control over the focus point than Samsung or Sony's alternatives. The "color pop" feature also lets you highlight certain objects while desaturating the background.

What's more, Nokia is the only smartphone maker offering an easy way to share "refocusable" photos on the web and through social networks. (Though it'll be interesting to see whether HTC's Zoe cloud sharing app includes anything like this when it launches.)

The Nokia-Samsung-Sony method has its advantages — you get a realistic blurring effect because defocused areas are based on exposures that are actually out of focus. But the main disadvantage is that it's painfully slow compared to normal smartphone photography — taking multiple shots with different focus points and then waiting for them to be processed into one image takes time. These kinds of shots require patience, are highly sensitive to movement of any kind.

That's where the HTC One M8's Duo Camera and Ufocus effect comes in. Instead of waiting around to combine multiple exposures into a single image, standard shots on the M8 incorporate an image from the main camera with depth information from the second camera. (Since it's got two cameras, it can easily judge depth.) You don't need to do anything special to get that depth information — if you're in Auto mode, it's always captured and saved elsewhere in the JPEG file.

There's actually nothing out of the ordinary about the hardware of the M8's secondary camera, in fact Anand Lal Shimpi of AnandTech reports that it's basically just the front-facing camera from the old HTC One (M7). (Which would explain why Duo Effects aren't available in some situations, like low-light and macro shots.)

In any case, with two cameras working in tandem it's possible to determine how far away each point in the photo is, and apply effects to what you calculate is either the foreground or background — and that's what happens when you use Ufocus, Foregrounder or any of the M8's other Duo effects. With Ufocus, for instance, HTC isn't combining in-focus and out-of-focus areas, just applying a blurring filter over certain areas of the photo.

HTC's Duo Camera setup is ridiculously fast, and lets you apply artistic, depth-sensitive effects later with no added inconvenience. But it too has its weaknesses. As we've mentioned, Duo effects don't work in all situations, and even when the second camera does capture depth information, the edges it calculates aren't always 100 percent accurate. And because you've only got one proper exposure to work from, you can't bring an out-of-focus object into focus. If an object is blurry in the original photo, it's blurry forever. For this reason you're best off using Ufocus and other Duo effects in shots with a clear foreground and background, and not much in between the two. Portraits, for example, are a great use case.

Smartphone photography is one of the areas where mobile tech still has plenty of room for advancement, and the recent trend towards refocus effects is just one way in which manufacturers are using new hardware and software to work around the challenges of getting great photos from a super-thin device that fits in your pocket.

And while Lytro-like wizardry is outside the grasp of mobile photographers for the moment, that doesn't mean we can't blur, shade and filter to our hearts' content.

More:

• Hands-on with Sony's background defocus camera app
• Hands-on with Nokia Refocus
• Hands-on with the Samsung Galaxy S5 camera
• Zoes, video highlights and Duo effects on the HTC One M8