The research explores the intriguing intersection of Cognitive Science, Real time interaction and affective computing through the lens of human mimicry. At its core lies a powerful idea : when avatars reflect our facial expressions, they can forge deeper, more natural connections. Using Blender for 3D modelling and Unity 3D for immersive environments, the design of expressive avatars capable of dynamic, real-time responses can be obtained. By integrating Mediapipe' s facial landmark detection, the system captures users facial expressions on the fly, enabling avatars to subtly mirror them.
Grounded in the Theory of Event Coding and Social Resonance Theory, this work decodes how facial expressions are obtained in shared virtual space to understand and enhance natural human connection with lifelike avatars.
A digital avatar could not only mirror your expressions in real time but resonate as well. This research explores the measurement, prediction and impact of human mimicry during live human-avatar interactions, grounded in cognitive and dynamics of social resonance. At the core, an integration of Unity 3D for real time avatar rendering with Mediapipe' s Facial Landmark Detection to track user expressions via webcam. Using Facial Action Units, the system captures and analyses specific muscle movements that produce facial expressions, enabling rich, two-way engagement between user and avatar.
Developing this system includes real time facial expression synchronization, seamless Unity 3D - Mediapipe integration, networking and usage of already working Me Other Overlap Task Experiment reference. Beyond technical enhancement, the research contributes to affective computing and human-computer interaction offering a powerful tool for studying human facial expressions and its resulting spontaneous response of avatar. This leads to integrate a powerful digital platform for studying how social signals shape digital communication and make sure that the research does not stop at this juncture but visualize how avatars can mimic us but how the avatars can meaningfully connect with humans.
