Meta patents an AI system that can track user’s mood by listening to voice

Meta has been granted a patent for an AI system that continuously records a user’s voice, transcribes it, and feeds the audio through a machine learning model to detect their emotional state. The patent describes a device that listens for “audible communications”—including sighs, laughter, and vocal tone—and combines them with “contextual factors” like time of day, location, activity, and medication timing. The system builds a persistent mood log from all of this.

The patent frames the technology as a fitness tool.

Meta explains that the system is designed for “continuous emotional monitoring on everyday devices,” using what the patent calls “multimodal sensor inputs on synchronized timelines.” It listens at “predefined times” for tone, pace, pauses, and breathing, then quantifies the user’s emotional state. It can correlate mood with medication schedules, producing summaries like “a happier emotional state associated with a particular time of day or at a time when medication is taken.”

As part of its argument, Meta says that AI-powered emotional coaching could correct workout posture and adjust guidance in ways a human personal trainer cannot.

It is worth noting that Amazon tried something similar. In 2020, it launched the Halo Band, a fitness wearable with a built-in microphone that performed “tone of voice analysis.”

After public backlash however, Amazon removed the microphones from the next-generation model in 2021 and discontinued the product line entirely in 2023.

Meta’s smart glasses are already the subject of a privacy crisis over covert recording, with seven million pairs sold and two US lawsuits alleging the company misled consumers about how footage was handled. Adding persistent ambient voice recording to the product line would extend the surveillance from what users see to how they feel.