Between all their various space, cost and computation constraints, smartphone cameras are hard. Producing great photos from a tiny lens and sensor in a pocket-sized device is a serious challenge, even at the high end.
Essential, makers of the Essential Phone is one of the many manufacturers adopting a dual rear camera system in its handset, with one sensor capturing color detail and another capturing monochrome images for improved clarity.
Yazhu Ling, lead image quality engineer at the Andy Rubin-led startup, walks through exactly how this dual camera setup works in a new post on the Essential company blog:
The first rear-camera is designed for color, and like most cameras, it applies a red, green, or blue color filter at different pixel locations, and then assigns that pixel a value. As a result, the camera must interpolate the neighboring pixels to produce the final image. What does this mean? If only some of the pixels are assigned color values, the camera must infer what the rest of the image should look like, and this often leads to less-than-ideal resolution. That's why we made our second rear camera a true monochrome camera, which does not require any color filter. The lack of a color filter means that no interpretation is necessary—every pixel is assigned a true black or white value, which enables the camera to produce images with much less noise and much higher resolution, no matter the lighting conditions.
That's all well and good, but combining data from the monochrome and RGB sensors into a pleasing photo is another challenge altogether.
As an image quality engineer, all of this is a lead up to the main thrust of my work. To convert imaging information from the sensor data into a final image, the camera must employ a complex Image Signal Processing (ISP) pipeline. Getting this right requires months of tuning, and it has been the focus of my work since October of last year.
The blog post itself is well worth a read if you're interested in how you go from photons hitting a sensor to JPEGs in your Gallery app. Ling also offers an interesting comparison of the same image captured throughout the different stages of camera tuning — from March through until present.
The Essential Phone is due for release in the next few weeks, having missed its original target window of July 2017.
