Many of us who have worked remotely have experienced how exhausting it can be to manage how we look on a video call while simultaneously trying to be engaged in the meeting itself. In this blogpost, we explore how Novecs, a tool that displays personalized feedback on one’s own body language and facial cues during an online meeting, could potentially help tackle this challenge.

Non-verbal Cues in Online Meetings

Have you ever found yourself nodding and smiling a lot more than usual in your online work meetings? Or, conversely, have you engaged in an important work discussion, only to realize at the end of the online meeting that you were slouching the entire time? What about becoming overly aware, or even anxious, about how you look in your self-view? And feeling exhausted afterwards, experiencing what has been popularized as “Zoom fatigue”?

While non-verbal cues are crucial for effective communication by signaling engagement and rapport, these scenarios illustrate how such cues can be challenging to manage in online work meetings. Relying solely on your self-view requires constant monitoring, which can be exhausting or even impossible during cognitively demanding work meetings. It also doesn’t help many of us who have the desire to improve their non-verbal behaviour but might lack the self-awareness and knowledge of how to do so.

Thus, we designed and implemented Novecs, short for Non-verbal cue system, which leverages real-time detection and personalized, in-meeting feedback on the user’s own non-verbal cues. Novecs consists of a sidebar that displays feedback next to the videoconferencing window during a meeting, along with a summary window that appears after a meeting is over that displays statistics and a timeline visualization. Novecs targets four types of non-verbal cues, including both body language and facial expressions – posture, gaze, smiling and nodding.

We designed Novecs to be privacy-preserving, glanceable, non-normative, and personalizable. All data was processed locally on users’ machines. In-meeting feedback was designed to be ambient and glanceable to not distract from workers’ meetings. We avoided being overly normative to not cause users to feel judged, but encourage self-reflection. For example, we used more neutral colours like blue and grey to signal feedback, rather than more noticeable but potentially biased colours like red or green. Users were asked to personalize their feedback by defining their own target non-verbal behaviour.

Ultimately, we see Novecs as an individual self-reflection tool for helping workers who may want additional support in their body language and facial expressions during online meetings, recognizing that not all workers may feel the need to use Novecs. We designed Novecs so that it could empower the user, rather than be used against them.

Eighteen knowledge workers used Novecs in a multi-week field study with their regular online work meetings. We collected data from short post-meeting surveys, questionnaires at various time points throughout the study, as well as a semi-structured exit interview.

Not All Meetings are Created Equal

Most participants shared that Novecs helped increase their awareness of their non-verbal cues, without distracting from their meetings. Rather than simply being broadly aware of their non-verbal cues, Novecs encouraged reflection on whether or not their non-verbal cues were appropriate in-the-moment of the meeting. For example, this led some workers to realize they should smile less, as overcompensating with excessive smiling and nodding was common in online work meetings:

“I reduced [smiling] a bit, like where it’s more appropriate to do that, matching more towards the overall atmosphere of the meeting.” (P9)

Not all meetings are created equal, as they vary in factors like meeting size, the importance of the meeting, and familiarity with other meeting attendees. All these factors impacted our participants’ perceived usefulness of Novecs. Workers shared that Novecs’ in-meeting non-verbal cue feedback was most useful in important, small-to-medium sized meetings with unfamiliar meeting attendees. 

In one-on-one meetings, workers felt like the rapid, back-and-forth conversational pace was a sufficient signal of engagement, making non-verbal cues less important to manage. Large meetings meant that workers’ own non-verbal cues were less important as they were just “one of many small pictures on the screen” (P7), making small-to-medium the ideal size for Novecs. Important, more formal meetings with strangers required a level of professionalism where support from Novecs was especially welcome.

To Be Authentic is To Be Human

Participants reflected and shared on tensions around authenticity at work when using Novecs. Workers recognized that there were some meetings (again, not all meetings are created equal!) where their non-verbal behaviour had to express something different from what they were feeling internally – to “put on that customer service vibe.” (P9)

However, workers strongly emphasized the importance of authenticity, including, at times, embracing imperfect non-verbal cues as being human. By designing Novecs in a non-normative, personalizable manner, we found that it could support workers’ efforts to be authentic, by simply reminding them of their behaviour and giving them agency to decide the natural and appropriate timing to change, if at all:

“I wouldn’t correct it unless it felt right to do it. And that’s why I just kept my body language and gaze the same because that was authentic to me (…) It didn’t feel like I had to go out of my way to smile extra or not [smile] extra. It was just a little reminder of ‘ohh, you haven’t [smiled] in a while.’” (P8) 

Personalized, in-meeting feedback through Novecs can help workers become aware of, reflect on, and navigate tensions around authentic non-verbal behaviour during online work meetings, without being distracting. Critically, Novecs was able to respect workers’ agency and desire for authenticity and “humanness” in meetings.

To find out more about the design and implementation of Novecs, its impact on workers’ online meeting experiences, and additional design opportunities for similar systems, please check out the pre-print of our publication.