What happened to emotion-detection APIs from facial expressions?

A story from eight years ago

Years ago I ended up on a reckless project: take a face photo from a camera, infer the person’s emotion from the expression, and score it.

At the time I was told to use an external library and, for some reason, what I got was a C library. It had been compiled for embedded use, so it did not run on the server. The test server did not even have a C compiler. I had to build on a separate machine and move artifacts around. Just getting the environment ready took days.

Then I had a simple thought: if I did this now, could I just call Google or Amazon?

The answer turned out to be yes, but the landscape has changed more than I expected.

Major cloud APIs

Google Cloud Vision API

Google exposes emotion signals as part of face detection.

Detected emotions

• Joy
• Sorrow
• Anger
• Surprise

Output format

Each emotion is returned as one of five levels: VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, or VERY_LIKELY.

That is easy to read, but less convenient if you want numeric thresholds. The old library I used returned numbers, which were simpler to process.

Pricing

• First 1,000 requests per month are free
• Then $1.50 per 1,000 requests
• New users get $300 in credits for 90 days

const vision = require('@google-cloud/vision');
const client = new vision.ImageAnnotatorClient();

async function detectEmotion(imagePath) {
  const [result] = await client.faceDetection(imagePath);
  const faces = result.faceAnnotations;

  faces.forEach((face, i) => {
    console.log(`Face ${i + 1}:`);
    console.log(`  Joy: ${face.joyLikelihood}`);
    console.log(`  Sorrow: ${face.sorrowLikelihood}`);
    console.log(`  Anger: ${face.angerLikelihood}`);
    console.log(`  Surprise: ${face.surpriseLikelihood}`);
  });
}

It is straightforward. Setup is basically just creating a GCP project and issuing credentials.

Amazon Rekognition

AWS can detect emotions through the DetectFaces API.

Detected emotions

• Happy
• Sad
• Angry
• Surprised
• Disgusted
• Calm
• Confused
• Fear

This is broader than Google’s set and includes states like calm or confused.

Output format

Each emotion comes with a 0-100 confidence score. That number is not emotional intensity. It means confidence that the face matches that emotion.

Pricing

• Up to 1,000 requests per month free during the first year
• Then usage-based pricing, depending on region

import boto3

def detect_emotion(image_path):
    client = boto3.client('rekognition')

    with open(image_path, 'rb') as image:
        response = client.detect_faces(
            Image={'Bytes': image.read()},
            Attributes=['ALL']
        )

    for face in response['FaceDetails']:
        print('Emotions:')
        for emotion in face['Emotions']:
            print(f"  {emotion['Type']}: {emotion['Confidence']:.1f}%")

If you are already in AWS, this is probably the smoothest option.

Azure Face API

Microsoft used to support emotion detection as well.

It no longer does.

Starting around 2023, Microsoft gradually removed the feature. According to the company’s explanation, inferring emotions from faces can reinforce stereotypes or discrimination, and research also shows that expressions do not necessarily map cleanly to internal emotion.

What remains includes things like:

• age estimation
• smile detection
• face orientation
• makeup detection

So the attempt to compare three cloud vendors ends with one of them having exited the feature entirely.

Free options

Cloud APIs are billed per use, so they are not ideal for large batch jobs or offline environments. For that, local libraries are still relevant.

DeepFace

DeepFace is an open-source face analysis library that runs locally.

Detected emotions

• angry
• disgust
• fear
• happy
• sad
• surprise
• neutral

from deepface import DeepFace

result = DeepFace.analyze(
    img_path='./photo.jpg',
    actions=['emotion']
)

print(result[0]['emotion'])
print(result[0]['dominant_emotion'])

Installation is just pip install deepface, though the first run downloads model files.

face-api.js

If you want in-browser emotion recognition, face-api.js is an option. It runs client-side on top of TensorFlow.js and only needs a model a few megabytes in size.

The catch is maintenance. The project has been effectively dormant since around 2020. It still works, but it is hard to recommend for new greenfield work.

Comparison table

Item	Google Vision	Amazon Rekognition	Azure Face	DeepFace
Emotion categories	4	8	Removed	7
Free tier	1,000 per month	1,000 per month in year one	-	Unlimited
Usage billing	$1.50 / 1K	Usage-based	-	None
Offline	No	No	-	Yes
Setup	Easy	Easy	-	Slightly more work

Summary

• For quick experiments: Google Cloud Vision
• For production in AWS: Amazon Rekognition
• For zero cost and local processing: DeepFace
• Azure: no longer part of the conversation

Compared with the painful C-library setup from years ago, this is dramatically easier now.

Still, Microsoft’s reason for dropping the feature is worth remembering. These APIs are recognizing expression patterns, not reading minds. A smile does not always mean happiness, and a neutral face does not mean emotional absence. That limitation matters when you design anything around these outputs.