The Pixel 4 does not have a fingerprint sensor and relies on using your facial features for biometrics — that means unlocking the phone and accessing apps that depend on you proving your identity. A lot of folks aren't happy with the change, but the powers that be at Google want this for its future phones, so that's what we get.
Besides the back and forth about which method is better, the biggest question about face unlock is how it works. While I can't come up with any real answers to why Google dropped the fingerprint sensor, I can tackle this one: face unlock works by using infrared light to make a 3D map of your facial features. And believe it or not, it's not as complicated as you might think.
The hardware
There is a lot of stuff in the Pixel 4's top bezel. Too much to hide in a slim bezel or a notch, which is why there isn't one. And it's all important stuff if you want to securely unlock a phone using a face.
The Pixel 4's bezel isn't there to look pretty. It's a necessity.
The render above from Google lists the hardware involved, and outside of the front-facing camera and the audio port, all of it is needed for face unlock. Here's how it plays out.
The Soli radar chip notices that a hand is approaching the phone, which "primes" the display and wakes it up. When it's picked up, the phone detects that, too, using its accelerometer. This is a signal to turn on the rest of the hardware and get ready to scan and make a face unlock attempt. This is a big reason why face Unlock is so fast: it's ready and already working before it gets close to your face.
More: Here's how the Pixel 4's Soli radar works and why Motion Sense has so much potential
Once the phone is in position, the Face unlock flood illuminator turns on. This is an infrared LED that's pretty bright (that's a spectrum of light greater than the red end of the visible spectrum and we can't see it with our eyes). Its purpose is to flood your face in light so that all of it is illuminated. Since we can't see it, our features won't react to the bright light.
Our eyes can't process IR light no matter how "bright" it is.
While the flood illuminator is turned on, the Face unlock dot projector is engaged. This is also an infrared LED but it has two differences compared to the flood illuminator: it uses a contrasting color (infrared light also has numerous colors even though we can't see them) and instead of a single wide beam it's broken up into thousands of small points. This creates a matrix of dots on your face that are used to define the shape.
With your face illuminated and covered with dots using infrared light, the Face unlock IR cameras grab an image. These cameras have been specially tuned to get a good set of data about how and where the IR dots contrast with the flood illumination. Technically, a "regular" camera can be used to do this but it would be very slow and/or very insecure.
Software magic
Once a good "image" of your face and the IR lighting is acquired, software magic comes into play. The IR dots projected on your face are not uniform. They are when they exit the projector, but once they meet your face they are skewed and misshapen based on the surface and movement of your facial features. Using this information, the brains inside the Pixel 4 can collect data about the shape of your face and create a token during face unlock's setup. Think of this as a number between one and infinity and you'll have an idea of how it works.
More: How to optimize Face unlock on the Pixel 4
This token is stored inside the phone on the Titan Security Module. It never leaves your phone and nothing can access the data used to create it or the token itself. Every time you use face unlock, a token is created and sent to the Titan Security Module, where it is compared with the original. If the two are a close enough match, face unlock passes the check and your phone or app can be unlocked.
The token itself and the data used to create it are not available to any software.
How closely the two must match is based on a couple of things. The closer the match must be the slower it will be. The faster the operation is, the less secure it is. What Google had to do was fine-tune the software algorithm so it was fast enough and secure enough. That's where the real work is.
The process of gathering the data by means of IR light and sensors is actually pretty common. A lot of the things you use and eat every day have been sorted or graded using this type of technology, but when you add in a security factor, things get a lot more complicated. Because most of that process is software-based, we can expect face unlock to get even better over time.
