Predicting Face Expressions with Convolutional Neural Network

* Please allow the web browser to use your camera for this page *

Summary

WHAT

Face Classification
Using pretrained Convolutional Neural Network models

HOW

99.38% of Accuracy

of Expression Classification

RESULTS

It shows emotions probabilities
listing and ranking expressions by their probabilities

TECHNOLOGIES

VS Code

TypeScript

React

Next.js

Face-Api

Introduction

When customers interact with products and services, buying them or consuming their publicity, they feel excited, happy, sad, angry, or inspired by what they see. These emotions can create a desire or rejection to make an initial purchase or continue buying them. Thus, identifying emotions during interactions could be key to get feedback about how fitted the company vision of customer thoughts is.

In this project I made an application to detect faces, through your computer camera, and to classify your emotions on real-time using a Convolutional Neural Network pretrained model.

Methods

I implemented Face-Api as a real-time expression recognition. Face-Api is built in a Tensorflowjs library, with several pretrained models from where I used:

Mobilenetv1 SSD: for detecting faces and to make boxes over camera images.
FaceLandmark68Net: used to extract position, and shape from detected faces.
FaceRecognitionNet: used to compare the data extracted from faces with the existing data, making an expression prediction.

The expression prediction has a 99.38% of Accuracy using the dataset Labeled Faces in the Wild.

This application was coded using Typescript in React Next.js.

Results

In order to use it, you need to allow your web browser to use your computer camera (only for this page).

The application shows boundaries of every face detected, and below that we can see the different labels predicted with its respective probabilities (from 0 to 1). This means that the model could detect more than 1 expression and ranking them from the higher to lower probability.

Expression Detection

Conclusion

With this expression recognition solution commercial markets can build indexes to evaluate the customer services quality, or to rank emotions related to product in showcases. This could lead to addressing issues related to negative emotions in real-time or implement new ones searching for the improve the emotional performance of some products.

This emotional quantification open doors to use human natural (and mainly unconscious) responses to connect better with its preferences, and therefore with the best products or services we can offer.