Basics of Sound Design for

Video Games
Michael Cullen
About Me
- BFA in Film Production (Sound Design emphasis), Minor in Music
- Worked for Sony, Avid, NBCUniversal (TV show Grimm)
- Worked on over 35 short films/docs - nominated for Student Academy Award
- Finalist for Cinema Audio Societys Recognition Award for best sound mixing
- Performed / worked in over 80 live theater musicals/shows
- Just finished UCIs Sankofa video games first level!
Sound Designer Jobs
For our purposes, there are two separate jobs that work with sound while making
a video game:

- Sound Designer
- Sound Implementer

The creation and implementation of sounds can be handled by both positions but
for our purposes its easiest to divide the tasks into two separate jobs.
Sound Design
What is Sound Design?
Instead of defining sound design, its best to talk about what it can do:

- suggest a mood, evoke a feeling

- indicate a geographical locale
- define a character
- mirror or exaggerate how things sound in real life
- clarify the narrative

Although we may not think about the sounds we hear everyday, we recognize
them readily and have preconceptions about how things should sound.
Families
DX = Dialogue - any verbal speech in the game.
MX = Music - any non-diegetic music.
SFX = Sound Effects (Hard Effects) - any sound from an real-life object.
FOL = Foley - any sound effect that the player makes.
BG = Backgrounds (ambience) - noise from the environment.

Generally every video game should have every type of sound family represented.

Some sounds could fit into multiple categories.

Video Game Exercise 1 - Shadow of the Colossus
Lets watch this gameplay clip and imagine what you think you should hear:

(4:30)
What I thought I should hear:
- DX - player talking to the horse, breathing, grunts
- MX - some kind of epic music (orchestral), slow and steady
- SFX - all the horse sounds (breathing,footsteps on stone/grass/dirt)
- SFX - sound of colossus
- SFX - sound of sword bow being pull / tension on the string
- SFX - sound of rocks falling
- FOL - footsteps of player on stones/dirt
- FOL - rattle of cloth
- BG - light wind in the plains
- cave echoes?
Video Game Exercise 2 - Badlands
Lets watch this gameplay clip and imagine what you think you should hear:

(0:22)
What I thought I should hear:
- DX - maybe some noise from the creature
- MX - slow mystical music, dark and subdued
- FOL - flapping of the creature
- FOL - enlarging/shrinking sounds
- FOL - creature knocking into metal pipes / branches
- FOL - squish of creature dying
- SFX - sound of whooshing fans / gears
- BG - jungle sounds (bugs, leaves, wind)
Video Game Exercise 3
Lets watch this gameplay clip and imagine what you think you should hear:

(44:45)
What I thought I should hear:
- DX - breathing, grunts
- MX - some kind of epic music (orchestral), slow and steady
- FOL - sounds of flapping clothing
- FOL - player flying sounds
- FOL - collection of items
- SFX - flying kite fluttering
- SFX - water rising
- SFX - glittering walls
- BG - sandy wind
- SFX - hall echoes?
The Purposes of Sound in Game
In general, sounds tell the player:

[Link] state of their actions in the game.

- Points
- Level Up
- Health
2. The environment that they are immersed in.
- Windy, rainy, snowy
3. The mood of the game.
- Hostile, foreboding, adventurous, dangerous, calm, organized

Sounds are the easiest way to connect to your player emotionally.

Where do I need a sound effect?
To determine what does/does not need a sound effect:

- Does it move?
- Does it draw attention to itself?
- Does it tell the player something they cant see visually?
- Could it set a mood?

If the answer is yes to any of these questions, that object needs a sound.
Thematic Expectations
Picking a sound depends on the context. Although there is no right way to sound
design a game, there are certain expectations for different genres:

- Action-Adventure games: complex big music, big sound effects, a lot of DX

- Mobile games: big music, simple sound effects, some DX
- 8-bit games: simple music, simple sound effects, little DX

You dont have to follow the conventions, but knowing them will allow you to break
them in a smart way.
How do I make a sound effect?
Normally the best sounds are the ones that combine the realistic with the fictional:

- What is the object made out of?

- Are there expectations on how it should sound?
- What kind of mood should the sound evoke?

To make a sophisticated effect, combine multiple SFX into a new single sound. By
combining sounds from different frequencies, you will have a larger sound.
Human Hearing Frequency Spectrum

[Link]
Orchestral Instruments Frequency Spectrum

[Link]
Practical Example
Making the sounds of the creation of the universe.

- video example of universe breaking open from an egg

Practical Example
Making the sounds of the creation of the universe.

- Low frequency - low rumble

- Low frequency - canon gunshot
- Mid frequency - egg crack
- Mid frequency - bomb dropping explosion
- High frequency - thunder/lighting
Mono vs Stereo vs 5.1 surround
Sound effects are recorded with various microphone setups. Most of the time they
are recorded in:

- Mono = recorded with one microphone

- Stereo = recorded with two microphones
- 5.1 = recorded with six microphones (.1 is the bass channel)

When you are creating your own sounds:

- DX, FOL, SFX should be mono files.

- MX and BGs should be stereo files.

Ensure to use 48k sample rate, 24 bit depth for highest quality
Programs to Use
You can make the sounds in:

- Garageband (Apple only)

- Audacity (Free)
- Reaper
- Cubase
- Sony Acid
- Pro Tools
- Logic

Look for free DAWs (Digital Audio Workstation)

Audio plugins are called VSTs or RTAS

Where do I find sound effects?
[Link]

[Link]

[Link]

[Link]

[Link]

Record them yourselves with your phone!

Implementation
Audio Objects in Games
Three audio objects that you can add in a game:
- Audio Listener (the Cameras ears)
- Audio Sources (Speakers)
- Reverb Zones

The speakers placed across the map to wherever you want a sound to emanate
from. From that speaker the sound can be triggered to play at a specific time.

Reverb Zones can add additional effects to how the speakers play the sounds.

The listener doesnt have to be parented to the camera but it normally is.
Visual of Audio Sources (Unity)
Visual of Audio Sources (Unity)
Video Game Exercise 4 - Stalin vs. Martians
Before looking at good implementation, lets look what bad implementation is:

(4:30)
Some of the Bad Implementation Traits
- DX - no character DX (from soldiers)
- MX - has vocals
- MX - too loud
- BG - no ambient BGs whatsoever
- SFX - same russian lines while adding vehicles
- SFX - most SFX repetitive (vehicles moving and firing)
- SFX - chaotic, not responsive to players current actions
Video Game Exercise 5 - Motorbike
Before looking at good implementation, lets look what bad implementation is:

(1:54)
Some of the Bad Implementation Traits
- BG Music - drops out unexpectedly
- BG Music - too loud
- BG - no ambient sounds whatsoever (doesnt distinguish sand/mountain)
- SFX - motorcycle has no variety
- SFX - wood hits no variety
- SFX - no tire skids for dirt vs. metal vs. wood
- SFX - no human breaths/grunts
- SFX - no level up / success SFX after finishing level
Video Game Exercise 5 - Ride to Hell: Retribution
Before talking at good implementation, lets look what bad implementation is:

(8:06)
Some of the Bad Implementation Traits
- DX - no breathing, grunts, other character's yells
- MX - too loud
- BG - no ambient sounds
- FOL - no sound of player weapon hitting
- SFX - motorcycles have no variety
- SFX - wood hits no variety
- SFX - no tire skids for slides/spinouts
- SFX - no level up / success SFX after finishing level
Implementation Considerations
After designing sounds for an object, you have to decide how to implement the
sound to best tell the narrative. When doing so, youll to have to consider:

Repetition / Variation
Priority (Mixing Levels)
Trigger Areas (Optimization)
2D vs 3D sounds (Non-diegetic vs. Diegetic)
3D Sound settings
Platform and devices
Repetition / Variation
Most sounds in real life never repeat exactly the same. Unless a sound
corresponds to specific game mechanic, it should have many variations.

Example: footsteps

- Provide at least 6 different single footstep sounds per surface

- Randomize the order in which the SFX play
- Randomize the pitch +/- 20% from original source (80%-120% speed)

Variation prevents ear fatigue.

Audio Engine Priority (Mixing Levels)
In Unity and other audio software, you can provide a priority number to tell the
audio engine what is most important to hear. It will then mix the audio accordingly:

Unity (0-255)

- DX - 255
- MX - 200
- SFX ~ 128
- BGs ~ 128

When resources are low, then the most important sounds remain audible.
Trigger Areas
Having all the sounds play all the time is expensive resource-wise. Instead use
triggers to minimize load and avoid a wall of noise.

- SFX sounds should only trigger when player is in that area

- only BG sounds should trigger On Awake

Snapshots (Unity only)

- Write code to trigger change in audio mixer to allow certain sounds to play
louder than others

Unity has build compression settings for to minimize size of audio files in a build.
2D vs. 3D Audio Sources
For video games, there are two different types of audio sources: 2D and 3D.

2D sounds = (non-diegetic) emanate from player (like wearing headphones)

Examples: MX, FOL, most BGs, some DX

3D sounds = (diegetic) emanate from speakers placed around the scene

Examples: SFX, most DX

3D Sound Settings
Each speaker in :

- Output to a specific submaster

- Should have priority level
- Special blend slider changed to 3D or 1
- Should have low Max/min levels

To avoid chaos, it's best to have low

speaker radius so sounds are contained
to specific area in the map.
Platforms Limitations Considerations
Just like having visual constraints with different screen sizes/graphic cards, same
constraints with a variety of speaker sizes.

Speaker cone size <= 6 to fully reproduce fully frequency of human hearing

- Computers: headphones to larger speakers

- Tablets: headphones or small speakers
- Phones: headphones or tiny speakers

Knowing where your game is going to be played

will determine how you implement the sounds.
Implementation Software/Plugins
- Wwise (standalone)
- FMOD (standalone)
- Unity Audio Engine (Audio Mixer, Snapshots)
- Plenty of plugins on Unity Asset Store
- Fabric (Unity plugin)
- Two Big Ears (binaural plugin)
- Hand coded scripts (C# or Javascript)

The program you use doesnt matter. What matters is the intention behind the sound.
Practical Example
Sankofa is a action-adventure game made here at UCI.

Led by : Magda El Zarki and Patricia Seed

Platform: For PC made in Unity 5

Team: over 22 students / professionals

Scope: 2 years (6 months of intensive work)

Sankofa Sound Features
Sankofa is a action-adventure game made here at UCI.

- Submaster Busses
- Reverb Zones
In Conclusion
- Sounds inform the player about their status, surroundings, and progress.
- Combine multiple sounds from different frequencies to make fuller SFX.
- Add variety to SFX to promote realism.
- Centralize your sounds to avoid chaos and over-informing the player.
- Ensure platform can handle sounds to maximize what the user hears.
- Use sound to better tell your story.
Credits
Jory Prum - Konsoll 2013: Making a great sounding Unity game using Fabric
Lectures by Dan Pavelin (Chapman University)
Lectures by Harry Cheney (Chapman University)
Lectures by Roy Finch (Chapman University)
Lectures by Michael Kowalski (Chapman University)
Images from [Link]
Sankofa game made by UC Irvine
Questions?
Brainstorming Exercise for your Group
Please take this time to gather in your groups and start thinking about how you
can incorporate sounds into your game that better tells the story.

- How can sounds help our game?

- Which platform going to use to implement the sounds?
- Who will take the lead with sound designing/implementing the sound?

Ill be coming around to each group to hear your proposals and give additional
ideas for your game.