Big tech is quietly patenting facial emotion tracking—and your everyday desktop could be the next testing ground for workplace AI surveillance.
If you work for a large company in the U.S., odds are your employer is watching you work—even when you’re at home.
According to research firm Gartner, 60 percent of large employers track their workers using “bossware,” technology that surveils employees’ keystrokes, desktop activity, and location data. Spurred by the remote-work shift of the pandemic and the explosive growth of AI, this surveillance has become an unavoidable reality for millions of Americans.
But could the next frontier of bossware be monitoring an employee’s facial expressions?
Some of the world’s most powerful tech companies are actively investigating the idea. According to a recent *Financial Times* opinion piece by Elaine Moore, Microsoft, Amazon, Nvidia, and Meta have all filed patents claiming the ability to track a user’s emotions by analyzing tone of voice, body language, and facial expressions. Multiple outlets have criticized the technology as deeply intrusive; *Futurism* described Meta’s patent as a product that “listens to you and everything you do, combines it with other data, and feeds it all to a mood-predicting AI.”
These technologies have direct, if unsettling, commercial implications. The Amazon patent, for instance, mentions the ability to detect when a user is ill and subsequently have Alexa recommend flu medicine. However, in a corporate setting, this tracking could be weaponized by employers as yet another data metric to quantify and evaluate performance.
For some, facial monitoring is already a reality. Truck drivers have served as the testing ground for this technology, with companies like Visage Technologies offering software that monitors drivers’ faces for signs of distraction or drowsiness. Although driver-facing cameras are increasingly being rolled out in fleets across the country, the American Transportation Research Institute notes they remain highly unpopular among drivers due to privacy concerns—particularly among female drivers.
In fact, most Americans are staunchly opposed to having their expressions monitored throughout the workday. A Pew Research Center poll found that 70 percent of Americans oppose the use of facial recognition technology by employers to analyze employee expressions. Furthermore, when the non-profit news site *The Conversation* surveyed readers on emotion AI in the workplace, the most common concern was the loss of privacy, which can trigger significant negative psychological effects like stress and paranoia.
According to Tara Behrend, a psychology professor at Purdue University and an expert on bossware, digital surveillance also drives increased burnout. Speaking on the American Psychological Association’s podcast, Behrend explained that when employees are constantly worried about how they are perceived by surveillance technology—forcing themselves to constantly “put forward the right image or the right behavior”—it leads to severe distraction and anxiety.
As machine learning continues to improve, so will AI's ability to track human emotions through visual and auditory cues. Companies may be tempted to use this technology to monitor their workforce, attempting to turn happiness into a KPI that can be easily displayed on a quarterly chart.
However, adopting this technology in the workplace will almost certainly have the opposite effect. Instead of boosting morale, it will turn the focus of employees away from performing their jobs and toward performing for the AI panopticon. In many workplaces across the country, everything employees do and say is already surveilled.
Will how they *feel* be next?