Dissertation Project

In completing my Sound Engineering degree, I wrote a dissertation exploring how varying degrees of physical accuracy influence sound localisation ability in virtual environments. Three degrees of accuracy were used, from the built-in Unity spatialiser to Microsoft’s Project Acoustics, which is a wave acoustics engine that simulates effects such as occlusion, reverberation, and portalling.

A testing environment was constructed in Unity, where participants were tasked with locating a series of sounds played back to them in sequence. Results showed that while increased accuracy tended to correlate with more successful localisation, physical accuracy was a drawback for overly-reverberant situations such as caves, indicating that design decisions need to be taken in finding a desired balance between realism and ease of use, though external elements such as user interface indicators could help.

Visualisation of Project Acoustics simulating a sound source

Environment and source positions used in testing

The dissertation is titled: “Assessing the impact of acoustic simulation on sound localisation within virtual environments”.

An academic poster providing an overview of the project, as well as a PDF of the full dissertation, can be accessed using the buttons below.

Screenshot of participants’ view

View Poster

View Dissertation

Research

Adaptive Audio Cues for Dynamic Difficulty Adjustment

Dynamic Difficulty Adjustment (DDA) is a real-time approach to adaptive difficulty in which game systems adapt to a player’s performance, to the effect of maintaining their engagement and presenting an optimally challenging experience. DDA is well-researched, but focuses heavily on the modification of gameplay parameters such as enemy damage or resource availability.

Exploration of applying DDA through audio is limited but shows potential; audio immerses a player within a game’s context and can be used to support key gameplay information, both in a non-diegetic (UI sounds) and diegetic (weapon reloads, enemy ‘barks’) manner. Audio can also play a strong role in gameplay, such as listening for enemy footsteps in tactical shooter games, but is often not actively thought about during the gameplay, instead playing a more supportive role. Adjusting audio could therefore be a subtle, un-noticeable effect in contrast to more drastic challenge adaptation implementations that players may perceive as unfair.

To test the feasibility of this idea, a simple first-person stealth game in which audio plays a key gameplay role was made, with audio cues adjusted in real-time based on player performance as a form of DDA. The goal of the project was to test if this dynamic audio cue adjustment could increase player enjoyment by better matching the game’s difficulty to their abilities.

Screenshot of tutorial area from the game

The game tasks players with navigating a maze-like castle environment while avoiding patrolling enemies by listening for the sounds they emit. Enemies are invisible, becoming translucent at longer distances outside the range of their sound attenuations. Players primarily listen to footsteps to identify, for example, when an enemy crosses from a stone path to a grass patch, or goes silent for a moment before turning in a different direction. Keeping track of multiple enemies with increasingly complex movement patterns serves as a natural challenge progression. The player reaches checkpoints as they progress, and they have 2 lives which they can lose by being spotted by an enemy, sending them back to their last checkpoint. Losing both lives instead sends them to the next checkpoint, and crossing a checkpoint in any way resets all lives.

The player’s performance is measured through a ‘detection score’ which updates in real-time. As the player avoids enemies, their score improves, and when they are seen it worsens. Losing lives skews the score to be more negative, until the player can show improvement. This score then adapts the game’s audio to best suit current performance. Enemies’ sounds are filtered by high-pass and high-shelf filters to either increase or reduce clarity. Enemy barks’ chance to occur changes, alongside attenuation sphere size and the intensity of occlusion and obstruction effects for all sounds.

Screenshot from gameplay - enemy visible in the distance

Testing was conducted using a between-subjects design, where one group played the DDA version, and the other played a version without DDA, where audio didn’t change in any way over the course of the game. Upon passing a checkpoint, the player’s average score, time played, and lives lost are recorded into a .json file for analysis. After completing the game, the player would fill out a short questionnaire designed to gauge their subjective experience, querying points such as frustration, willingness to keep playing, and perceived difficulty to gain an overall understanding of their enjoyment.

Average detection score by group

Average detection score by checkpoint

Mean response to each question by group

Given that a lower detection score demonstrates superior performance, it can be seen that the non-DDA group performed marginally better. Looking at mean detection score by checkpoint, it can be seen that both groups’ scores worsened throughout gameplay. Statistics analysis shows that this effect was slightly stronger in the non-DDA group, which could suggest that DDA helped to balance challenge. Finally, question responses, measured 1-5 for least to most agreeable, showed there was not much deviation in perceived enjoyment between the two groups, though the non-DDA group scored higher in 4 out of 6 questions.

These results were, however, affected by a low sample size of 6 for each group, meaning the trends found are not reliable. Alongside increasing sample size, further work could investigate in detail the best approach to adapting the difficulty of audio cues, as the filtering approach was untested in this regard before being employed. Extending the game’s duration would also provide a clearer idea of how this DDA approach would apply to the target use case of a full game, alongside variations in genre and perspective. Overall, the study gave me great insight into adaptive difficulty, and made me think about both the practical and subconscious role audio plays in games, and how it can affect players in a number of ways.