Dodging the uncanny valley
The phrase ‘The Uncanny Valley’ entered the popular imagination in 1970 when Japanese roboticist Masahiro Mori published an essay of the same name in the journal Energy. While there is some debate about whether Mori invented the term—it appeared in various, similar forms in 1906 and 1919—it is accepted that Mori’s idea of ‘the uncanny valley’ was his own.
With computer games and VR experiences, navigating out of ‘The Uncanny Valley’ is a bigger task in 2019 than it was in twenty years ago. Back then, in my gaming days of Duke Nukem 3D, Quake, Star Wars: Jedi Knight, and Shadow Warrior, there was no expectation that a generated character (or avatar) would pass muster as human being. The expectation instead was that they would only a recognizably-human form.
In VR, though, the stakes are different. Those entering a virtual world such as MATERIA.ONE are expecting a proximation of reality. That’s because the objective of the VR environment is to provide an alternative to the real world, not as—in the case of a computer game—to present a 2D challenge to be overcome or puzzle to be solved.
At this point, it might be helpful to take a step back and look at what ‘The Uncanny Valley’ phenomenon actually is. When his work was first published in Energy in 1970, the reaction was one of polite almost-indifference. It is only in the nearly-fifty years since that the concept has gained prominence. This means that there is often some confusion over what it actually is.
Mori posited in his essay that human empathy with technology in a humanoid form increased as the latter came closer and closer to resembling actual humans. However, there was a ‘dip’ in this level of empathy the closer to human the technology became, before rising again to a level higher than before. This ‘dip’ in the emotional response was what Mori coined ‘The Uncanny Valley’.
Or, as one interpreter of the original report would later write: “[Mori] hypothesized that a person’s response to a humanlike robot would abruptly shift from empathy to revulsion as it approached, but failed to attain, a lifelike appearance.”
Mori himself gave the example of a prosthetic hand, writing that it had already resembled a human hand to the same extent as a pair of false teeth. “However,” he added, “once we realize that the hand that looked real at first sight it is actually artificial, we experience an eerie sensation.”
‘The Uncanny Valley’ effect has had multiple direct and inadvertent applications on computer-generated characters. The Star Wars universe’s Rogue One was criticized for its ham-fisted depictions of Peter Cushing and Carrie Fisher(1), computer animation of the original’s stars in 1977 that looked like half-animate mannequins. The barbs thrown were that their inclusion did nothing more than cause an audience to recoil from the screen. Conversely, the difficulty in animating human aspects such as hair and skin texture, however, directly gave us Toy Story (2) as the creators at Pixar realized it would be easier to render the smooth plastic surfaces of its plaything cast.
Technology has been coming closer and closer to bridging the gap. As Ciaran Foley, CEO of Ukledo and Immersive Entertainment wrote on Medium(3) last year: “Countless advancements in 3D modelling, dynamic lighting and processing power available to the general public, means that this issue is eroding over time, but to date this issue can pervade in even the latest game releases boasting the best graphics.”
This is not to opine that ‘The Uncanny Valley’ is always a bad thing. It can have its uses. At the end of February, in the UK, there was a hoax/moral panic about Momo(4). The ensuing and accompanying articles, reported on sites such as the BBC, used a picture of a sculpture by Japanese artist Keisuke Aisawa called ‘Mother Bird’. The original sculpture is a representation of an ubume, an evil figure in Japanese mythology. Pictures of the sculpture, which is probably still in Japan or lying in a strong box at the bottom of a very deep sea (and hopefully not behind me), are unnerving. This is because the eyes are too big and bulging, the face too angular, the smile a contorted rictus. If it was supposed to be an artificial human, it would be a failure. But it’s the open use and acknowledgment that ‘The Uncanny Valley’ exists that makes it so effective.
Despite being humanoid (and pretty convincing), the ‘Mother Bird’ figure falls pretty deep into ‘The Uncanny Valley’. In fact, it is a good candidate for a horror film or game—this is where ‘The Uncanny Valley’ is used well.
In developing our innovative VR products at Staramba, we are acutely aware of how ‘The Uncanny Valley’ can impact our users’ enjoyment of their experience. We believe that when a user puts on a VR headset, the stakes are even higher as ours is a more-immersive, complete world than that of a 2D game. They have the benefit of being recognizably separate from the world of the user. But VR experiences are different as we are producing an approximation of the real world. After all, the world ‘reality’ makes up 50 per cent of the term ‘virtual reality’.
Technically, there are a few things that we keep in mind when developing VR characters. Firstly, we need to keep a frame rate of 90. And, secondly, we cannot assume that all of our users own high-end computers. This means that we have different requirements for creating high-poly characters as opposed to somewhere like a movie studio. Adding to the level of difficulty is that our users can interact with characters in real-time. These are all things that we need to bear in mind, along with following the engine-and-performance rule that more polygons mean not only a better character but more of a drain on a computer’s processing power. However, we have the backing and technical skill to pull this off and believe that we have one of the best teams for the creation of characters.
It is also a far-more immersive experience than previous entertainment forms. There is a direct lineage from books, which were the first form of entertainment, through film and television to computer games and now VR. An example of how we are more immersed in VR worlds than we are with games or written material is that the degree for succeeding at audio-visual is so narrow that to miss it can cause motion sickness within users.
Surpassing the Uncanny Valley is vital to the success of VR. As Michael Abrash, chief scientist at Oculus(5), said, “In the long run, once virtual humans are as individually quirky and recognizable as real humans, VR will be the most social experience ever, allowing people anywhere on the planet to share virtually any imaginable experience.”
We are working hard at Staramba to make sure that our creations do not fall into ‘The Uncanny Valley’. After all, we want to ensure that our users have the best, most-immersive VR experience.
As you can see from this picture of Timo Boll, there was some work to be done on this avatar’s appearance. When shown to the Staramba marketing department, the first thing most mentioned was The Walking Dead. The lack of eye colour, a skin texture that did not look ‘natural’, and an elbow that looks as if it has been dislocated then roughly stuck back into place shifts Boll’s avatar away from looking perfectly human. And that shift, just a few degrees and fixable by adding eye colour and improving the look of the skin, is enough to pitch the character straight into ‘The Uncanny Valley’.
This is why we are now collecting feedback from our beta test of MATERIA.ONE. In February, we sent out 34 keys via Steam for users to look at the world we have created and give us their opinion on where we can best deploy our resources.
One of our most-recent ventures has been the addition of MATERIA.ONE artwork to our Instagram page. Not only does it present a good idea of how MATERIA.ONE will look when fully completed, we have put that imagery there so we can gauge, from the feedback, whether we are heading in the right direction. So please check it out, take a look, and let us know what you think.
Unfortunately, the Instagram feed only gives a 2D view of MATERIA.ONE. But very soon we will start allowing users properly into MATERIA.ONE, and look forward to hearing their—and your—feedback about the world being created.
(1) Collider.com ('Rogue One': The Problem with That CGI Character)
(2) The Guardian (Artists climb the uncanny valley)
(3) VU Token (Have we crossed the Uncanny Valley?)
(4) BBC News (Momo challenge: The anatomy of a hoax)
(5) Upload (Oculus' Chief Scientists Hints a What's Next For VR)