Creating convincing synthetic human beings is a notoriously difficult task. The "uncanny valley" phenomenon is a hypothesis that as you try to increase the realism of your human-like creations, there is a point behind which improvements actually become negative, causing a sense of unease or repulsion.
Something flips in the mind of the observer, and the creature starts to be seen as a “human with which something seems to be wrong” instead of “human-like”, a doll becomes a corpse. It takes a lot more effort to climb out of the valley, every little detail that was abstracted away comes back into play.
We created the Uncanny Valley project to explore these concepts. It is an interactive webpage experiment with three animated virtual human heads reacting to the user's mouse movements, accompanied by three songs, featuring light sources synchronized to the music. The experiment is trying to find out what can be achieved today on the web, with very limited resources, on a constrained platform.
Uncanny Valley features real-time skin and eye shading with a simple subsurface scattering approximation, under dynamic lighting, running on a home-brew WebGL rendering engine called XG.
The three electronic music pieces, which were written specifically for this project, tie music and lighting intrinsically, using a combination of automatic sound frequency analysis and manual beat annotations for modulating light intensities and positions. Synthetic humanoid voices aim to enhance the eeriness of virtual humans visuals.
We used a simple two-lights rig emulating photography studio lighting and realistic virtual camera with subtle post-processing effects (bokeh depth-of-field, tone mapping, color grading), trying to cheat our way out of uncanny valley by presenting synthetic human beings in a tried-and-tested setting and framing, with a dramatic and flattering presentation.
Special attention was paid to the eyes, both technically and stylistically. A two-layer material is used, with simple approximation of light scattering, refraction, and caustics. Pupils are dynamic, with physiologically-inspired animation depending on the amount of incoming light. Having eyes locked in eye contact with the observer turned out to be a simple yet effective way to induce the illusion of ”somebody being there“. As an Easter egg we also included several non-human “android” eyes variants.
Branislav Ulicny / AlteredQualia (SK) likes to explore possibilities of real-time computer graphics on the web. Before succumbing to the temptations of dark arts of rendering, he dabbled in academic research of artificial life, crowd simulations, and bioinformatics. He is an alumnus of a popular open source project three.js and holds a doctorate in computer science from EPFL.
Uncanny Valley is the result of a collaboration between the computer graphics programmer AlteredQualia, and the platform Fractal Fantasy.
Visuals, code and rendering: Branislav Ulicny, AlteredQualia
Fractal Fantasy, Songs by: Sinjin Hawke, Martyn Bootyspoon, Zora Jones