Professor Charles Fernyhough from our Department of Psychology explains about his research on the inner experience and why he believes it has the power to transform the future of video games.
Video games are big business. The value of the global market is pushing the US$200 billion mark (£158 billion): bigger than the music industry and Hollywood combined. But the gaming industry has also been facing challenges. The market is shrinking from its peak at the height of the pandemic, and there has been a rash of layoffs and studio closures.
In this uncertain environment, game developers are, more than ever, looking to create experiences that stay with their players. A great deal of effort goes into visual elements, including the appearance and overt behaviour of characters, for example. But at the level of subjective experience, the inner worlds of gaming characters are not so often explored.
This is where the science comes in. My research over the last 30 years has been about inner experience: the things in our minds that we are conscious of, such as thoughts, memories, inner dialogue, visual imagery, feelings and emotions. Traditionally considered impossible to study because of its private nature, inner experience is becoming established as an important field of cognitive science. And it has the potential to transform gaming.
In so many ways, video games just keep getting better and better. With the shift toward mobile gaming, the question for studios has become less about how to persuade people to buy this product (they may be downloading it for free) but to keep coming back to it.
Graphics are becoming ever more vivid and lifelike. What studios are doing with audio design is stunning. The player feels they are actually there in that medieval village or that rainforest or on that spaceship travelling between galaxies. And yet a common criticism of many games is – still – that they are just like watching a film.
That’s a puzzle, because of how gaming, more than other media, creates so much potential for a truly interactive experience. You are not just watching that ship sailing across the galaxy; you are captaining it. You can choose your body shape and physical skills, and see them depicted there on the screen in astonishingly lifelike detail. But these qualities are mostly on the surface, at the level of appearance and overt behaviour.
How could gaming go deeper into inner experience? Here’s one example. Many people report having a silent, internal conversation with themselves for much of the time. Our research has shown that inner speech comes in several different forms and has varied functions in thinking, planning and emotion regulation. But when inner speech is depicted in video games, it tends to lack the qualities and variety that make the experience so different between people.
Another example is the kind of memory we have for the events of our own lives. Autobiographical memory can take different perspectives, vary in vividness and show a range of multisensory qualities. Memory does not work like a video camera, but instead brings together sights, sounds, smells and other kinds of information in a dynamic, endlessly shifting way. We are even beginning to understand how these different qualities of memory are realised in the brain.
Making a game is fundamentally about creating an experience – seeding an experience in the mind of its player.
When we do get to share a game character’s inner experience, it tends to lack the variety and nuance that the science tells us is there: the different qualities of inner speech, the various features of memory and visual imagination. The inside of a game character’s mind is often a black box.
There are of course exceptions to this rule. In Disco Elysium, for example, you can play with the main character’s thoughts and mental attributes in an unusual way. This text-based game was, however, limited in what it could do to depict the subjective qualities of inner experience – the colourful pageant of the everyday mind.
To give an example of a game I consulted on, Hellblade is the multi-award-winning story of an eighth-century Pictish warrior, Senua, whose inner experience is distinctive in that she hears voices and has other unusual perceptions and beliefs. In the first game, Senua’s Sacrifice, and in its just-about-to-be-released sequel, Senua’s Saga, Senua experiences psychosis, and you as player share these experiences with her.
I think the gaming industry can only benefit from understanding inner experience better. What I hope we’ll see is a more fluid, realistic, immersive gaming experience where the inner worlds of gaming characters live and breathe as much as their actions in the game world.
As well as having the potential to create new, interesting and memorable takes on the gaming experience, this work has real-world implications for accessibility, mental health, neurodiversity and sensory inclusion.
In fact, our starting point is that there is no such thing as a “normal” mind – we are all different, and our own minds differ from moment to moment. Rather than just taking your own mind into the gaming experience, gaming provides an exhilarating opportunity to experience a different kind of mind while you are there.
Gaming has incredible potential to work for good. Most of all, it’s not about some worthy educational ideal. Games are – and should be – about having fun. Beyond opening up creative possibilities for game developers and players, I believe that knowing more about our own inner experience can be helpful, restorative, even therapeutic. Gaming is a powerful way to push that ideal forward.
Our Department of Psychology prides itself on being a collegiate and welcoming department, committed to success for all who work here. Through dedicated teaching sessions, an optional placement module, external speakers and even an employability retreat – we actively prepare students for working life.
We are proud to be ranked in the Top 10 in the UK for Graduate Prospects by The Times and Sunday Times Good University Guide 2023 and in the World Top 100 in the QS World University Rankings by Subject 2024.
Feeling inspired? Visit our Psychology webpages to learn more about our postgraduate and undergraduate programmes.
Durham University is a top 100 world university. In the QS World University Rankings 2024, we were ranked 78th globally.
/prod01/prodbucket01/media/durham-university/research-/73895.webp)