This is How Your iPhone’s Facial Recognition Works

Key Points

• Face ID is primarily used to unlock your iPhone and other Apple devices.
• This technology uses the face of the user to unlock and authenticate devices.
• The technology was advertised as Apple’s replacement for Touch ID.

As we continue to develop our technological systems, we rely on computer automation more often to do the tasks we don’t have the time or knowledge to do ourselves.

A perfect example of this is Face ID on Apple devices, which uses neural networks to identify someone based on facial recognition instead of password protection.

In the past few years, Apple has rapidly developed its facial recognition feature in the iPhone X and, now, the latest iteration of the iPhone, the iPhone 13.

The facial recognition feature allows users to unlock their phones by simply looking at them instead of entering a PIN or password, which has been the norm since cell phones came into existence.

So, how does your iPhone’s facial recognition feature actually work? Let’s take a look!

What is Face ID?

Face ID is a facial recognition system designed and developed by Apple, Inc. for the iPhone and iPad Pro. This technology uses the face of the user to unlock and authenticate devices.

Face ID is a special case of biometric authentication, where the enrolled user’s face replaces the traditional password and PIN combination as the sole factor in unlocking the device.

It scans 30,000 points on your face to create a three-dimensional map of your features, which are then used to generate a mathematical representation called a template.

The template mathematically represents the shape of the person’s face with remarkable accuracy. To unlock, Face ID simply reads the facial map by projecting light with specialized technology onto the user’s face and captures an infrared image. Then, it compares that data to the one stored in its secure memory. If the two match up, it unlocks.

That said, if you’re someone who changes their hairstyle often or has worn makeup lately, or if there are drastic changes to your appearance, you may need to enter a passcode.

And, while Apple says that their neural networks learn over time, you may not need to reenter your passcode as much after some time has passed (though this will depend on how often you use Face ID).

A Brief History of Apple Face ID

Apple announced Face ID during the unveiling of the iPhone X on September 12, 2017. The technology was advertised as Apple’s replacement for Touch ID, a fingerprint-based authentication system built into the home button of the iPhone 8, as well as prior models including the second and third-generation iPhone SE.

On September 12, 2018, Apple introduced the iPhone XS and XR with faster neural network processing speeds, providing a significant speed increase to Face ID. On October 30, 2018, Apple introduced the third generation iPad Pro, which brings Face ID to the iPad and allows face recognition in any orientation. iOS 13 included an upgraded version of Face ID which is up to 30% faster than Face ID on previous versions.

How Does Face ID Work?

Facial recognition works by scanning the user’s face to create a 3D map of their face. From there, the phone identifies key points on the face, such as where the eyes are positioned, and uses that information to calculate a mathematical representation of the person’s face. The system then compares this representation to other facial representations that have been calculated using an in-depth scan of an individual’s face.

The idea is that no two faces should look exactly alike. If Face ID finds a match between the mathematical representation and any stored image, it unlocks your device with just one touch or swipe. Face ID will prompt you for your passcode or show you different authentication options if it doesn’t find a match.

The success of biometric recognition is partly dependent on being prepared for the authentication process. Those using an iPhone X or newer iPhone models will need to prepare their face in the same manner that smartphone users are now accustomed to scrubbing a greasy or wet finger before attempting to unlock their device.

The process can be described in three steps, namely:

• Image capture
• Face recognition
• Authentication accuracy

Let’s explore each in more detail.

Image Capture

Face ID captures multiple photos of your facial characteristics using a mix of light projectors and sensors. Apple refers to these technologies as the ‘TrueDepth camera system’ and says they work together to create a ‘detailed depth map of your face to recognize you instantly.’

To activate Face ID, follow the on-screen instructions, which include turning your head in a circle so that a camera may capture several images of your face for a 3D map. It works day or night, outside or inside, and employs infrared (IR) light to highlight your face while shooting photographs.

Because infrared radiation (IR) spans wavelengths of electromagnetic energy (commonly known as ‘light’) just beyond the visible spectrum, the iPhone X’s display will not dazzle you in the dark.

Here is how an image is captured:

• The proximity and ambient light sensors assist the TrueDepth camera system to determine how much illumination is required for face recognition.
• To illuminate your face, the flood illuminator emits infrared (IR) light, a portion of the electromagnetic spectrum that is invisible to the naked eye.
• The dot projector generates over 30,000 dots of invisible infrared light to create a three-dimensional map (area and depth) of your facial landscape.
• The infrared camera records photographs of the dot pattern and the IR light (a heat signature) reflected back from your face.

Face Recognition

Your face is a ‘biometric,’ which means it is a measured biological property. Fingerprints, voice, and eye irises are other biometrics utilized in security.

Every biometric authentication system compares two complex patterns and determines their similarities. These patterns could be the waveforms in your voice, the ridges on your fingertip, the colored structures in your iris, or the topography of your facial characteristics.

When a biometric system is installed, a computer, such as a processor in your smartphone, captures and stores the reference patterns, which are referred to as a template or enrolment image. When you wish to access a gadget (for example, unlock your phone), you show the computer a verification image.

Because the enrolment and verification photographs will not be identical due to variances in capture conditions, your phone employs a threshold to assess whether they differ considerably. For example, a comparative score of 0.7 may be close enough in some situations, and that minimal score is not a fixed value.

Face recognition occurs as follows:

• The IR images are transferred from the camera to the “Neural Engine” computer processor in the iPhone, which creates a 3D mathematical model (map) of your face.
• The 3D model, also known as the verification image, is provided to the computer’s algorithms and compared to your saved template, also known as the “enrolment image.”
• Based on a comparison score of similarity between your photographs, the processor determines whether the verification and enrolment images match.
• If the comparison score exceeds a particular threshold number, the phone authenticates your identity and unlocks (or authorizes a payment).

Authentication Accuracy

Pose, illumination, and expression (PIE) are three factors that determine the accuracy of face recognition.

When your face is not concealed, recognition is obviously more accurate. After the three PIE variables, face occlusion is the fourth parameter. That indicates you don’t have anything covering your face, such as sunglasses or a mask.

Face recognition has so far needed your complete cooperation. You must provide photographs that adhere to specified requirements for all current government applications, including those for a passport, visa, mugshot, and driver’s license.

For a passport photo, you cannot be smiling or laughing. You are asked to remove any facial jewelry; the background is flat with indoor lighting.

Face ID seems to overcome many of the above restrictions, as Apple claims that its system can recognize you despite changes in your physical appearance like when you wear glasses or a hat, cut your hair or grow a beard.

Is Face ID Secure?

Face ID uses strong mathematical algorithms and sophisticated signal processing to ensure that you’re looking at your phone when it unlocks. It also ensures that your face is consistently recognized by the device—even when there are changes in lighting when you’re in a different room or even if you’re wearing sunglasses.

Face ID is the most secure biometric authentication method available on the iPhone, and Apple made it even more secure with their proprietary Secure Enclave hardware.

The Secure Enclave is a dedicated processor that handles face recognition and privacy while protecting data from being accessed by other apps running in the background. It’s an impressive technology that makes your data safer than ever.

When Face ID identifies a close match but requires a passcode to open the device, it will also update this data. Face ID data never leaves your smartphone and is never saved to iCloud or anywhere else.

The iPhone and iPad Pro, as well as the TrueDepth camera technology, have been rigorously tested and comply with international safety requirements. Under typical operating conditions, the TrueDepth camera system is safe to use. Because of its low output, the device will not injure your eyes or skin.

Face ID also requires your attention to unlock, so if your eyes are closed or looking away, it will not unlock. According to Apple, the chances of someone unlocking your iPhone X (or newer models) using Face ID are one in one million, which is an improvement above Touch ID, which has a false-positive rate of one in 50,000.

Uses of Face ID

Face ID is primarily used to unlock your iPhone and other Apple devices. When you look at your iPhone, it detects your face, and you don’t have to enter an Apple password or passcode. Besides unlocking your iPhone, Face ID is used to:

• Authorize Apple Pay
• Approve purchases in the App Store, iTunes, and iBooks
• Auto-fill passwords in Safari
• Sign in to third-party apps, such as banking or password manager apps

Summary

Apple’s Face ID uses a TrueDepth sensor that consists of an infrared camera, flood illuminator, proximity sensor, and dot projector. The TrueDepth sensor detects infrared light to see the geometry of your face and sends it to the A11, A12, A13, A14, or A15 Bionic chip. That chip creates a mathematical model of your face, which it stores for reference.

When you hold your phone up to your face and the phone recognizes you, it unlocks instantly. If someone else tries to unlock your phone with their face, they won’t be able to do so because they don’t have access to the mathematical model stored on their device.

While facial recognition may seem like a simple security feature, the technology is actually working behind the scenes to create a more secure and easy-to-use iPhone. It’s a win-win for users who don’t have to worry about passwords or fingerprints, and app developers, who have endless possibilities for facial recognition.

Up Next

Check out some of our other articles:

• What is Facial Recognition, and How Does It Work?
• 10 Incredible Search Engines That Use Facial Recognition To Search Faces
• 6 Reasons Why Face ID May Not Be Working