Abstract

(57) ABSTRACT A system configured to perform style-aware listener animation. By representing different listening styles (eg, facial expressions) using an embedding space, a single model can be trained to generate unique facial animations for a number of distinct listeners. Thus, individual listening styles can be associated with a listener identifier, enabling the system to (i) animate a plurality of different listeners with unique nonverbal behavior and/or (ii) select a particular listener identifier or desired type of listener style with which to animate. This enables the model to be generalized to new listeners to generate additional listener facial responses without needing training data for each new listener. The model may process a listener representation style or listener identifier, along with input data corresponding to a speaker talking, to generate unique facial animation responsive to the speech.

Date

March 18, 2025

Authors

G Datta, V Yadav, Y Wu, A Jaiswal, RM Reddy, P Singhal, ...

Inventors

Gourav Datta, Vivek Yadav, Yue Wu, Ayush Jaiswal, Rajiv M Reddy, Prateek Singhal, Karthik Ramakrishnan, Premkumar Natarajan

Patent_office

US

Patent_number

12254548

Application_number

18082709