Key Concepts

Before you learn about the use cases and implementation of Face Analytics, it’s important to understand its fundamental concepts in detail.

Facial Landmark Localization

Facial landmarking is the process of detecting and localizing specific key point characteristics on the face. Key points like corners of the eyes, endpoints of eyebrow arcs, endpoints and arc of the lip, tips of the nose, position of the cheek contours, and nostrils are commonly considered while processing age detection, gender detection, and emotion recognition.

Zia Face Analytics first provides the general coordinates of the location of the face in an image. Additionally, the coordinates of localized landmarks are also provided in three landmark localization modes:

• Basic: This is a 0-point landmark detector that only detects the coordinates of the face [x1, y1, x2, y2].
  
  • x1, y1: Top left corner point of the face/face attribute/object
  • x2, y2: Bottom right corner point of the face/face attribute/object
• Moderate: This is a 5-point landmark detector that detects the following:
  
  • Eyes: The center of both eyes
  • Nose: Nose tip
  • Lips: The center of both lips
• Advanced: This is a 68-point landmark detector that detects the following:
  
  • Jawline: Face boundary
  • Eyebrows: Left and right eyebrow
  • Eyes: Left and right eye
  • Nose bridge
  • Nostril line
  • Upper lips: Upper and lower edge
  • Lower lips: Upper and lower edge

These details are provided for each detected face in the image individually in the response. Face Analytics also provides the confidence score of the determined face coordinates in the response.

Facial Emotion Detection

Facial emotion detection is the process of detecting human emotions based on facial expressions. Facial landmarking plays an important role in facial emotion detection. Key points such as the eye corners, mouth corners, eyebrows are considered to play a primary role when compared to secondary landmarks such as the chin or cheek contours, in detecting facial expressions and classifying them into an emotion class. The facial emotion detection technology is designed using deep learning methods, such as Convolutional Neural Network (CNN), that analyzes visual imagery.

Zia performs an in-depth analysis on a detected face and determine the presence of a smile on the face. The response contains the confidence scores for the aspects of smiling and not_smiling out of 1. Based on the aspect with the highest value, Face Analytics makes the prediction of the presence or absence of a smile.

Age Detection

Face Analytics predicts the age range of a face detected in an image. Age detection technology implements a similar approach of machine learning methods and AI algorithms to identify a series of interest points on a face and analyze the localized landmarks for particular signs applicable to specific age groups.

Face Analytics detects the age of a face as one of the following age ranges: 0-2, 3-9, 10-19, 20-29, 30-39, 40-49, 50-59, 60-69, more than 70 years old.

The response contains the confidence scores for each age range out of 1. Based on the age range with the highest confidence score, Face Analytics makes the final prediction.

Gender Detection

Face Analytics detects the gender of the faces by implementing deep learning algorithms to identify key characteristics associated with each specific gender. Certain localized points of interest in the face play a crucial role in determining the gender of a human. Zia compares the analyzed results with a vast array of datasets to arrive at a prediction of the gender.

Similar to the other attributes, Face Analytics contains the confidence scores for male and female out of 1. Based on the identified gender with the highest confidence score, Face Analytics makes the final prediction.

Input Format

Zia Face Analytics performs facial detection and attributes recognition by analyzing image files. Face Analytics supports the following input file formats:

• .jpg/.jpeg
• .png

You can code the Catalyst application to use the end user device’s camera to capture photos and process the images as the input files. You could also provide a space for the users to upload image files from the device’s memory to the Catalyst application, to generate the results.

The input provided using the API request contains the input image file, value for the landmark localization mode, and boolean values to specify if the emotion, age, and gender detection need to be performed. If you don’t specify the mode or the required attributes to be detected, all attributes will be detected and the advanced mode will be implemented by default.

You can check the request format from the API documentation.

The user must follow these guidelines while providing the input, for better results:

• Avoid providing blurred or corrupted images.
• Ensure that the faces in the image are clear, visible, and distinct.
• Do not upload images that have partial faces, silhouettes, side profiles, or other unrecognizable angles of the faces. The entire face must be visible in the image for better predictions. Face Analytics cannot detect faces that are tilted or oriented beyond a certain level.
• Ensure that there are no textual content or watermarks present over the faces in the image.
• The file size must not exceed 10 MB.
  

Response Format

Zia Face Analytics returns the response in the following ways:

• In the console: When you upload a sample image with faces in the console, it will return the decoded data in two response formats:
  

  • Textual format: The textual response contains the presence of the face, smile, the detected gender, and the detected age range of each face with their confidence levels as percentage values.
  • JSON format: The JSON response contains the general coordinates of the faces, the coordinates of their localized landmarks, the confidence score for each aspect of the detected smile, age range, and gender of the faces, along with the predictions for each attribute. The confidence score of 0 to 1 can be equated to percentage values as follows:
    

  Confidence Level in percentage	Confidence Score of values between 0 and 1
  0-9	0.0
  3-9	0.0
  10-19	0.0
  20-29	0.0
  30-39	0.0
  40-49	0.01
  50-59	0.12
  60-69	0.23
  <70	0.63

• Using the SDKs: When you send an image file using an API request, you will receive only a JSON response containing the results in the format specified above.

  You can check the JSON response format from the API documentation.