Sound Analysis of SSX (2012). Remixing Sound While Snowboarding.

Featured Image Credit: EA Sports.

SSX: Deadly Descents (2012) is a PS3 and Xbox 360 snowboarding game developed by EA Canada and published by EA Sports. In the game, you follow ‘Team SXX,’ consisting of nine fictional characters, all with different skills in snowboarding as an extreme sport.

The team is on a world tour to try to conquer the world’s nine most dangerous mountain slopes as the first ones. But a former member of Team SSX (Griff), a skilled snowboarder, has decided that he will be the first in the world to conquer all nine peaks.

The player’s first task is to train each of the nine characters in specific disciplines ranging from tricks and speed to using parachutes and ice axes in their run.

The training runs, and the final nine deadly runs occur on realistic simulations of real mountains and mountain ranges on Earth. The goal is to beat Griff in all disciplines on all nine mountains.

The fight to be the first to conquer the nine most dangerous peaks in the world is played as a single player.

However, there is also a multiplayer part, where you can compete against other players online. This mostly occurs asynchronously against a “ghost,” which means you compete against a recording of another online player’s run.

You register for competition within a certain time frame, and when this period ends, it is determined where you rank in the overall standings among all the participants. One could say that several players are thus competing asynchronously within the same diegesis.

The game sequence I will analyze here is an asynchronous time course, so you can sometimes see the “ghost” of another online player’s run.

Sound and Genre Conventions in Sports Games

SSX is a sports game that is based on real-life snowboarding disciplines. The game explores off-piste and snow park traditions and combines them with competitive elements about being the best at doing tricks or getting from the top to the bottom of the mountain as quickly as possible. It is not a realistic game.

On the contrary, it seems based on a long tradition of arcade games, such as its colorful effects and unrealistic opportunities to perform wild tricks and jumps.

The mountains are designed as huge “playgrounds,” so you can largely choose your way down according to which jumps, rails, and pipes you prefer. Deep abysses naturally limit the player’s options.

Even though the mountainsides are largely open landscapes, it fundamentally comes down to getting from A to B. The gravity in the game ultimately always drives the player downwards.

This leads to a certain linearity, which makes the game comparable to, for example, car games. The linearity means that this type of game is well suited to popular music tracks.

Collins describes it as follows:

“Genres such as racing games are much better suited to linear music than many other genres since the player may be tied to a specific length of the track or a particular length of time, and therefore, the timings are more predictable than in other genres. For instance, a racing track may last at least three minutes, or a soccer half will last forty-five minutes, whereas a battle scene is far more temporally unpredictable”.

Karen Collins in Game Sound (2008:129)

It is perhaps not surprising that there is a long tradition of using popular music numbers within sports and car games. Over time, the game genres have even developed their musical genre conventions.

Car games largely use dance and hip-hop, which either get the adrenaline pumping or tie the game to a certain subculture (Ibid.).

In the popular Grand Theft Auto series (GTA), hip-hop music is heavily used, linking the game to the gang culture in the USA, and since GTA III (2001), the player can each time they get into a car, choose from several different radio stations with popular music tracks, DJs, advertisements, etc.

The developers behind SSX follow a similar trend, using high-tempo music to get the adrenaline running in the player’s body while trying to maintain some subcultural authenticity by avoiding overly popular mainstream numbers and instead exploring newer subcultural genres like dubstep.

But as we will see, how the linear music tracks accompany the neck-breaking rides down the mountainsides is anything but linear.

This is due, among other things, to the processing power of today’s game consoles enabling real-time manipulation of existing material from CDs. When I find it necessary to explain this type of manipulation of sound files to emphasize a point, I have included some ‘sound studio technical’ concepts.

A full list of music tracks in SSX: Deadly Descents can be seen at: http://en.wikipedia.org/wiki/SSX_(2012_video_game)#Songs

Heli-boarding: From the Game menu to Mountain top with Helicopter

Once the game is loaded, you reach a menu shaped like a globe, from where you have to choose a mountain range, a mountain, the discipline you want to compete in, and finally, your avatar.

Then the game starts loading the mountainside you will compete on. An underscore has been composed for these purposes, and the menus have associated system-like interface sounds (which can be interpreted as extra-fictional/extradiegetic and overlay interface).

As a transition between the game’s menu part and actual gameplay, the “loading” screen switches to a cutscene. Here we first see a short animation of a helicopter flying over a mountain massif, and we hear the sound of its rotors.

Next, the fictional camera position changes, so we are now inside the helicopter’s cargo bay with the chosen avatar (here: Edward ‘Eddie’ Wachowski). The music simultaneously switches to the track “Young Blood” by the New Zealand band “The Naked & Famous,” which is graphically explained via an overlay interface in the top left corner:

One might be led to believe that it is extra-diegetic music to function as a typical underscore if it weren’t for Eddie immediately starting to dance to the music. This points towards the music being diegetic, which seems supported by the music being frequency-wise “rolled off” with low- and hi-shelving filters.

As we also see Eddie in third person, I can’t help but get associations to situations where I can hear music from another person’s headphones in public. Or to earphones of dubious quality. I assume that there must be a pair of earphones somewhere in Eddie’s gigantic hair.

In addition to the music, we can still hear the helicopter’s rotors from the cargo bay. Then, I assume the female helicopter pilot (we can’t see her) starts to talk to Eddie and encourages him to show as many tricks as possible before he reaches the foot of the mountain.

That it is the pilot seems to be supported by it sounding like she is communicating with Eddie through a headset, as her voice frequency-wise seems to be limited to the typical voiceband (approx. 300 Hz to 3.4 kHz).

This seems to support that my avatar has a pair of earphones on, through which he can listen to music and simultaneously receive information from the helicopter pilot.

This is not an interface that the player can interact with by pressing different icons, as in StarCraft. However, it still carries gameplay-relevant information from the game’s gameworld to the player, from music tracks, points, combinations, etc.

The way it graphically lies “on top” of the game’s fictional world makes me categorize it as an overlay interface.

The voice of the helicopter pilot is initially acousmatic and diegetic. Still, at the same time, it carries gameplay-relevant information about the goal and, thereby, the game’s rules.

According to Jørgensen’s narrative terminology, this would be a trans-diegetic internal sound, as it both points ‘in’ towards the game’s diegesis and is anchored in the fiction, but at the same time out towards the player as part of the game’s interface (emphasized interface).

The sound of the helicopter and pilot follow us down the mountainside. We never see her but only the helicopter from the outside. In that sense, she is consistently “off-screen.” In addition, the aircraft cannot crash, which makes the pilot invincible.

When you snowboard down the mountainside, the helicopter moves in and out of the picture, partly due to which tricks the player does. And if you jump high enough, it is possible to hit the helicopter and use it as a springboard for your tricks.

The pilot provides information about the goal of the run, gives encouraging comments and hints to the player about how far there is left on the track, about good places for tricks ahead, etc. She thus has a kind of divine overview of the track, which the avatar does not possess.

It is possible to perceive the pilot as acousmatic (although still trans-diegetic). She is speaking via some form of wireless radio communication.

In that sense, the helicopter could be perceived as the diegetic ‘box’ from which her voice comes, just like a radio in a film (which can also be both on- and off-screen). I don’t have a definitive answer to this, but I can only conclude that her position can be understood in both ways to the diegesis in the game.

At some point, the helicopter reaches its goal, and you get a short glimpse down the mountainside, where an opponent’s ghost is ready.

With a press of a button on my PS3 controller, Eddie jumps out of the helicopter’s cargo bay. Eddie and I get the pilot’s enthusiastic ‘Let’s do this!’. The run is on.

On the way down Makalu: Filter-sweeps and intertextual tricks on the Great Wall of China

Eddie’s trip down Makalu in the Himalayas is enhanced with sound effects, which can perhaps best be described with onomatopoeia such as ‘whoosh’/’swoosh’ (when he turns) and ‘crunch’ (when he lands from a jump).

The sounds help to give Eddie’s movements dimensionality, physicality, and weight in the fictional world. In addition to this, one can faintly hear a wind that becomes more evident the higher one jumps. And, of course, the sound of the helicopter’s rotor blades, the female pilot’s voice over the radio, and the music.

The real-time manipulation of the music is revealed as soon as Eddie makes his first jump. A bandpass filter creates a sweep over a frequency range in time with the arc of the jump (which becomes even more evident the higher one jumps).

At the same moment, the music transforms from being exclusively diegetic to also conveying gameplay-relevant information.

From a narratological perspective, this is a manipulation of the sound that transforms it from diegetic to trans-diegetic internal, as it goes from something Eddie can hear in his headphones to suddenly informing the player about the height and the length of the avatar’s jump in the game world.

This is important information, as the player quickly learns to ‘listen’ to their jump, so they know when to stop doing tricks and straighten up again, so they can land safely and not lose their points.

The real-time manipulation of the music, in other words, helps the player to optimize their score in relation to the game’s rules and goals by informing the player about how far it is down to the fictitious mountainside in the simulation of Makalu.

I find it difficult precisely to place the bandpass-filtered music within Jørgensen’s categories for the interface.

At first glance, it could be understood as part of the game’s game frame and thus the category of metaphorical interface. But as was the case with the extradiegetic interpretation, this must be rejected as soon as Eddie starts to dance to the music.

The manipulation of the music is also so closely linked to the avatar’s movements in the game’s gameworld that it should, under all circumstances, be one of the categories tied to this.

One argument speaks for the real-time music manipulation as an iconic interface due to its seamless integration with the game’s game world.

But this does not seem to harmonize with the sound, also supposedly being a natural part of the diegesis. Another argument speaks for the music here could be categorized as an emphasized interface. This does not initially harmonize with a sound originating from a friendly NPC.

But if we choose to interpret this “friendly NPC” as a system-generated “helper” that can take any “form,” then the use of a bandpass filter sweep can be understood in this way. As an invisible helper, the real-time music manipulation becomes almost the ludological counterpart to l’acousmetre.

Perhaps the above is an example of sound material carrying gameplay-relevant information, which can be categorized as several interface types simultaneously.

It is a legitimate conclusion, although it raises the question of whether the greater differentiation for the categorization of the sound interface, which Jørgensen’s ludological approach offers, is always the most optimal to use (does it fall short here).

In my view, the concept of trans-diegesis, in this case, seems better able to explain the spatial integration and function of the sound material.

This again supports the importance of recognizing and including computer games’ fiction as an important source for analyzing their sound side and why a gameworld understood exclusively as a world built on rules is a wrong premise for such an analysis.

In other words, the above example clearly shows the continued strength of diegesis as an analytical concept in the analysis of the sound side in computer games.

Activating Tricky mode

Eddie performs a trick against the helicopter’s underside in the image above. Since I have accumulated enough points, the so-called “Tricky Mode” is simultaneously triggered, enabling several tricks.

This is graphically indicated by the letters “TRICKY” at the bottom of the screen, along with the sound of a male vocal, with an echo effect on the voice saying “Trick(y).”

Thus, There is a clear connection between the graphic overlay interface and the sound as an interface, which could be an argument for the sound being categorized as an overlay interface.

However, the voice does not naturally seem to originate from the game’s gameworld but has a more external relation to it, which is supported by the echo effect.

Since the voice also tells the player that there is now the opportunity to perform extra wild tricks and get even more points in the subsequent period, the voice can be compared to the enemy music in Dragon Age: Origins – but with the opposite sign.

Here, there is no danger (if we disregard various abysses and avalanches in the terrain), but rather an invitation to “go for it”.

If this interpretation is correct, the voice does not originate from the game’s gameworld but rather from the game’s game frame, so the male voice must be categorized as a metaphorical interface.

Like in the previous example, everything seems to suggest that the male voice can be said to reside within both categories simultaneously.

If we were to describe the voice as part of the game’s fiction, we could note, among other things, that we have not heard the man before, we cannot see him (off-screen), and his voice is not frequency-limited, so it sounds like a radio or telephone voices, as is the case with the helicopter pilot’s.

From a narratological perspective, the sound can therefore be described as trans-diegetic external since it says something about the game world but from an external position closely related to the extra-diegetic space.

Unlike the example with the bandpass filter, the ludological approach in the latter example seems to have more to offer, in my opinion. This could be related to the voice’s external affiliation to the game’s gameworld.

As part of the game’s gameframe, the sound seems to have more of an “interface character” than a “diegetic character”.

The prefix “trans-” thus seems to form a half-real/half-fictional two-way bridge between the game’s fiction and interface and thus between the narratological and ludological discourse.

In the image above, Eddie is in the process of performing a so-called “Frontside Boardslide.”

It is a trick where he spins 360 degrees around himself repeatedly while riding on an edge or a pipe down the mountain (here it is on the edge of the ruins of the Great Wall of China, which is buried in the snow). Eddie is a skilled snowboarder who can continue with the trick for a long time.

And after a while, the music suddenly starts to loop, like when a vinyl record or a CD gets “stuck” and repeatedly plays the same small sound bite.

It is as if the music has become a metaphor for Eddie’s spinning, and Eddie’s spinning is a metaphor for the music being played from a stuck phonogram. But this stuckness is not entirely random.

The game’s sound engine cuts some sound material in real-time after a certain number of seconds of trick time, from where the music has temporally and linearly reached.

This first loops briefly in quarter note triplets, then in eighth note triplets, which then transforms into sixteenth notes if one can keep balancing in the trick. You could say that the game’s sound engine makes it possible for the player – through gameplay – to almost remix the music in real-time.

This seems to serve the same function as the example with the bandpass filter since it is an indicator that one is performing a trick, which one must remember to finish properly before the edge one is sliding on ends.

This loop of the sound material also seems significant for the perception of time.

Even though game time continues unchanged, and Eddie in fictional time also continues his run down the mountain, there is an expectation of linear progression linked to the music, which then propagates into the fictional game space.

When the music suddenly loops, a discrepancy arises in the diegesis’s spacetime continuum.

It is as if the music here detaches itself from its diegetic position in the fictional time and transitions to a part of the interface more closely linked to game time.

Sound, which in the first place is anchored in the diegetic space, becomes the carrier of gameplay-relevant information and thus is also spatially anchored as part of the game’s interface. It is precisely this movement that the prefix “trans-” in my view can describe.

The image below shows that Eddie has accumulated enough points in Tricky Mode (without crashing) to activate Super Tricky Mode, which opens up even more tricks. This is indicated sonically using a fragment of a dubstep remix of the hip-hop group Run DMC’s song “It’s Tricky” from 1986.

The music’s function and interface are the same as the voice, with an echo effect that sounded when Eddie activated Tricky Mode. But here, there is also a clear intertextual and transmedial reference to an early hip-hop number on another medium.

The next example is also related to time.

Reverse time example

As shown in the image below, I mistakenly sent Eddie into a free fall off a cliff. Here I can choose to do two things: I can let Eddie continue his fall into the abyss, or I can rewind the fictitious time in the game by holding a button on the controller for as long as I want to rewind.

If I let Eddie continue his fall, the track does not count as completed. Instead, I am shown a menu where I can restart the track from the beginning. This also means that all data must be reloaded, which means a pause in playtime.

However, I can continue my game if I choose to rewind the fictitious time. For example, it’s an alternative to being given extra lives in a game, as is the case in many early arcade titles. If I choose the latter, the music sounds again as if it was recorded on a vinyl record.

The developers seem again inspired by a vinyl aesthetic, as the game’s sound engine is based on the music and, via real-time manipulation, stops it and plays it backward with the player rewinding the fictitious time.

When the player stops rewinding the fictitious time, the music starts up again, like when a gramophone starts playing with the pick-up resting on the vinyl.

While the playtime continues unchanged, the time in the game’s diegesis is rewound.

This again points out that sound as an interface seems to be tied to playtime, while sound as part of the diegesis is tied to fictitious time.

According to the narratological discourse, this would mean that the diegetic music Eddie has in his ears, through the rewinding of the fictitious time, gets a function as a carrier of gameplay-relevant information tied to the game’s gameworld. The music is then transformed to be categorized as trans-diegetic internal.

As a carrier of gameplay-relevant information, the rewound music can also be described as part of the game’s interface.

The creative use of a vinyl aesthetic approach to the interface in this example reminds us in many ways of both the use of bandpass filters at Eddie’s hopping and the loop at Eddie’s spinning around himself; the sound that follows the rewinding also starts with the music, which is already closely integrated into the game’s gameworld.

However, I still can’t help but perceive the music as it sounds when the fictitious time is rewound. It is more externally related to the game’s gameworld than the effects associated with Eddie’s tricks.

But even though there can be no doubt that the music here becomes part of the interface, I have difficulty fitting this into any of Jørgensen’s categories. Fiction or interface?

In Closing

I think SSX is an interesting example because the game uses linear music numbers that are not already designed for computer games’ nonlinear and interactive universes uniquely.

The processor power in even the PS3 enabled the player to remix the music in real-time through gameplay. This means that linear pop songs, usually just used as linear and extradiegetic underscore, suddenly get an interactive character and become part of the game’s interface.

And in SSX, there is such a transformation in real-time, as the music constantly swings between being a fictitious element to getting an interface character the moment the player performs tricks, where the music is real-time manipulated by the game’s sound engine.

To describe this transformation from diegesis to interface, the concept of trans-diegesis seems to have great potential, which I find the examples with real-time manipulation of the sound above show with all clarity.

Thus, something methodically constructive can be gained by starting from computer game fiction and then looking at how the music either already carries gameplay-relevant information or moves during the game between being spatially integrated as part of the game’s diegesis and interface.

Jørgensen’s interface categories can help to clarify further how sound is spatially integrated into the game. However, I must also note that the categories seem difficult to use in practice. They show some strength when the sound connects to, for example, the graphical interface.

This may be because Jørgensen, in developing her ideologically-founded terminology, also started from the graphical interface in the game examples she has chosen.

The sound analysis of SSX also led to a surprising connection for me between the sound’s spatial integration and time in computer games. The sound seems tied to fictitious time when it is part of the diegesis.

When sound functions and is spatially integrated into the game’s interface, it consistently seems tied to playtime.

In hindsight, this seems to be an obvious and logical connection, but I have still not seen it pointed out elsewhere. In my view, this adds an extra dimension to understanding sound integration in the fictitious player rooms.

Until next time, happy gaming!