Diploma in Sound Engineering
Film and Game
Audio
Summary Notes
[Type here]
2
Contents
3 Lesson objectives
3 Introduction
4 ADR (Automated Dialogue Replacement)
6 Sound design, Foley, and FX
9 Bringing them together
10 Additional resources
11 References
SOUND ENGINEERING
3
Lesson objectives
By the end of this lesson, you should be able to:
o Learn the difference between ADR and foley
o Understand when to use foley
o Know how to record ADR
o Apply ADR and foley in a session
Introduction
The film, television and games industries have grown significantly in the past two decades. Not only in the quantities that
they are produced, but also in the speed of delivery. In the last lesson of module 3, we are going to look at recording dialogue
and sound effects for film and games. When we create soundscapes, it often happens that we use sounds made in studio,
and today we are going to explore how you can easily create your own sound effects as well as dialogue for film and games.
SOUND ENGINEERING
4
ADR (Automated dialogue replacement)
Location Dialogue
With our mobile devices such as phones, tablets and laptops becoming increasingly more powerful, more and more people
are using these devices to capture sound and video to make their own video content. Regardless of whether you are using
your phone, tablet, laptop, or portable recorder, you will often capture some unwanted sounds. Let’s look at location audio
and the issues that may arise during your location shoots.
To record location audio for film, we usually use portable recorders, rifle mics and boom poles. These tools are essential for
recording good quality location audio. But regardless of how good your gear is, if a plane flies overhead, a dog barks in
background or wind blows into the mic, your location audio won’t be usable for a production. Besides the noises in the
natural outside world, the reflections you get in some indoor environments will also often render your location audio
unusable for a full-scale production.
Most Hollywood films have the majority of the dialogue re-recorded in studio. We refer to this process as ADR, short for
automated dialogue replacement. ADR offers a variety of benefits compared to using the dialogue recorded on location.
Firstly, if the location audio has a lot of background noise, you’ll likely need to use ADR to get a clean recording. This way
you can avoid any sounds that would distract the viewer from the film and contribute to an added sense of immersion.
The second benefit of recording ADR is that you have much more control over the performance. If an actor on location didn’t
deliver a line as convincingly as the director wants, or mispronounced a word or line, ADR will be used to replace the poor
performance with a more convincing one recorded in studio. Because you are re-recording dialogue, you can try multiple
takes and styles of delivering the lines to get the best match for the scene.
The third benefit is that you have much more control over the tone and balance of the dialogue recordings. You can use
mixing and post-production techniques to enhance the timbre of the dialogue, add spatial elements and balance the levels
between different actors. Location audio can only be adjusted very little in terms of timbre, spatial elements are often fixed
and the levels between actors also can’t be adjusted – if you can and boost the levels of an actor, you will also cut and boost
all the ambient noise which will be extremely distracting to the viewer. The benefit of control over ADR is a big contribution
to the widespread use of it.
Cue Sheets
When you are about to record ADR, you will need quite a bit of info about the session, lines and where and how they need to
be delivered. This is the purpose of the cue sheet.
This is a typical cue sheet and I’ve added a link to the Pro Tools session and cue sheets in the summary notes so you can
explore and practice. A cue sheet is a document you will most likely receive from the producer of the film, series or show.
The cue sheet contains only the information you will need to complete the delivery of the ADR. It contains two sets of
information. The first set relates to the production and contains the name of the film, director, episode number, audio
supervisor’s name and the actors’ name.
The second set of information is what you need to complete the dialogue successfully such as character name, line number,
exact time stamp of the line and the words to be delivered. This is the information you will use to record the dialogue at the
correct scene and to identify the character. The location audio will give you a guide as to how the line was originally
delivered. The actor can then use the location audio as a guide track and perform the ADR in time with the original audio.
How to record ADR
Recording ADR is a fairly simple but time-consuming process that offers a great level of quality, clarity and control in a
production. To record it correctly is a simple process if you have the right workflow set up.
SOUND ENGINEERING
5
The first and most important aspect is that you need to have a DAW that supports video. Pro Tools, Nuendo, CuBase, Ableton,
Logic Pro and Adobe Audition supports video and allows you to record ADR and sync it to video within your DAW. Film editing
software is only capable of doing minor editing, so getting a DAW that is capable of advanced recording, audio editing and
processing will be really advantageous to you and your clients productions.
Recording ADR can make use of two approaches. To follow the video or to follow the location dialogue as a guide track. If
you decide to use the film as the guide, you need to pay special attention to the speed of the delivery to make sure that you
capture the dialogue in sync with the lip movements of the on-screen actors. Its sometimes a little tricky to record ADR this
way since you are relying on visual cues to sync audio with. If the actor is not experienced in recording ADR or dubbing, this
might be an exceptionally tedious and difficult task to complete on time.
The second option is to use the audio from the location recording as a guide track for the ADR recording session. Using the
location audio gives the actor a sense of the original delivery, how long each pause and line needs to be, as well as the
inflictions that were used during the shoot. This helps a great deal if some of the dialogue is off-screen and there’s no way
to tell whether the dialogue is in sync with the film or not.
It’s common that engineers use a combination of the two approaches and use the original dialogue in unison with onscreen
visuals for the actors to follow. This way you can make sure that the delivery is in sync with the visuals and that the inflictions
match the audio supervisor or directors’ expectations.
It’s best to use a separate room such as a vocal booth for the actor to record ADR since they might need some extra space to
move closer or further from the mic. You’ll need to mount a screen or a projector for the actor to follow the film scenes as
well as a pair of headphones to hear themselves and the guide track of the location audio.
SOUND ENGINEERING
6
Sound Design, Foley and FX
What is sound design
Sound design is the addition of audio elements in multimedia such as film and games. It involves recreating the ambient,
diegetic, and non-diegetic sounds. Diegetic sounds are the sounds that occur in the scenes. The best way to describe it is to
regard diegetic sound as the sound that onscreen characters can hear. Non-diegetic sounds are the sounds that the on-
screen characters cannot hear. This generally includes the music of a scene and the dialogue of a narrator.
Sound design revolves around creating a good balance between diegetic and non-diegetic sound. You can use non-diegetic
sounds to lead from one scene to the next and to make non-diegetic sounds become diegetic sounds as the scenes change.
When you are working with film, you often need to recreate the entire audio landscape. This includes everything from the
background ambience and small rustles and sound effects to the foreground sounds made by characters. The purpose of
sound design is either to add an additional element of immersion to a film or to replace existing location audio that was
poorly recorded or contained noises that rendered the takes unusable.
Sound design can be broken down into a few subsections:
• Ambient,
• Foley, and
• FX (effects)
Ambient sounds are those that make up the sounds of a certain environment, such as the sound of waves crashing on the
beach, the sound of cars driving on a busy highway, the sounds of a train station or just the subtle rustle of the wind in the
grass. All these types of ambient sounds give us a sense of where we are without the need to even see the onscreen visuals.
Foley is where you perform the actions of the on-screen visuals in studio and rerecord the sounds that you are supposed to
hear based on the onscreen interactions. These sounds include footsteps, fighting sounds for lunches and kicks, and the
rustle of clothing as a character is walking or running in a scene. You often need to be rather inventive with the ways in which
you create sounds, especially if those characters are fictional – such as robots, monsters, and aliens.
FX, short for effects, refer to all the sounds in a film or game that aren’t necessarily directly related to the interactions of the
characters. These sounds relate to larger elements in the film and can often be non-diegetic sounds that are purely added
for effect. It can also be sounds like explosions, earthquakes, storms, crashes and other types of sounds that would
contribute to the audio landscape in a significant way.
Foley
When you record ADR for a scene, you need to recreate the entire scene. Not just the dialogue, but everything from the
background ambience to all the other sounds that might occur due to the interactions onscreen.
There are many films where all the sounds in the scene are recreated in studio. The goal of foley is not to replace all the
sounds in a scene with the sound of the real thing, but rather to recreate the sound that will convince the viewer that what
they are seeing and hearing is true. It often happens that we need to create the sound of something that doesn’t exist in the
real world, such as cartoons, monsters, advanced robots, spaceships and aliens. In these cases its obviously impossible to
know what they would sound like if they were real. To create the sound for objects that don’t exist in the real world, you
need to use your existing sounds and create new sounds and combine and adjust them to create the sound for the fictional
element.
For objects and elements that do exist in the real world, we still often use sounds from other objects and repurpose them as
the real thing. As an example, the sound of rain could be the sound of sand falling on some cardboard, thunder could be
made by shaking a big steel sheet and a robot sound can be made by using an optical camera zoom. Recording foley is a
time-consuming process, but the result is a very authentic sound that doesn’t rely on Hollywood sample banks.
SOUND ENGINEERING
7
To record foley you have to follow the video extremely carefully. Every movement, rustle, and noise needs, to be perfectly
in sync with the video. If you are recording 2 or three foley elements at once, this can become increasingly more challenging.
Let’s consider the most common foley you’ll record, which are footsteps.
• Firstly, you need to consider what shoes the actor is wearing. If its boots, it will sound different to high heels, running
shoes or being bare feet.
• Secondly, what is the surface the actor is walking on? Is it a wooden surface, cement, gravel or tiles?
• Thirdly, you need to consider the environment. Is the person on screen walking outside, in a hall, corridor or
warehouse? This will influence the reverberation you choose for the footsteps.
• Fourth, you need to consider the clothes the actor is wearing. If the actor is wearing a bag or clothes that are loose,
you might need to add some cloth rustle or keys jingling while the actor is moving on screen.
• Fifth is the speed and intensity of the footsteps. Is the actor sneaking, walking, or running? This will impact the
overall level, intensity, and speed of your foley performance.
For footsteps, the considerations are:
• Footwear
• Surface
• Environment
• Outfit
• Pace speed and intensity
Game Audio
Game audio is an entire industry that is almost entirely separate from the rest of the audio engineering industry. If you are
interested in game audio, but have no idea where to start, in this section I’ll give you some guidance on how to approach
learning game audio and what the most significant differences are between game audio and sound for film and video.
Game audio has two elements to it, the first is to only create the sound and music. The second is to be part of the
development process and do game audio implementation in the game engine. Most likely, if you want to get into game
audio, you’ll need to develop some skills and understanding in audio implementation in game engines such as Unreal Engine
or Unity. These two are the main software used to develop games. If you are new to coding, I’d strongly recommend starting
learning with the Unreal Engine.
To make the audio implementation a bit more streamlined and easier to adjust, audio engineers often use middleware such
as Wwise. Wwise, also known as an API, short for application programming interface allows you to implement sound in the
game and adjust certain aspects of the sounds without needing to work in the game engine directly all the time. For those
who aren’t comfortable with coding in C, Wwise and Unreal Engine will be the most important tools in your learning
experience as a game audio engineer.
Now that you know where to start developing your game audio skills, let’s look at the main differences between game and
film audio. Because when you get to studio, there are some minor differences that will make a big difference in your
approach.
The first and most important is that in game audio you are limited to storage. In a film you can add as many sounds as you
like, because in the end all of it will bounced down to a 5- channel surround or 2-channel stereo track. In a game you are
limited to a storage budget. This means that a certain amount of storage is allocated to the audio assets of the game, and
your audio assets cannot exceed that storage limit. This means that you need to be aware of your files sizes constantly.
Having a 10 MB sound for a creaky door opening in a film is an average file size, but in game audio 10 mb for the sound of a
door opening is a huge file size. In this case, you’ll consider the frequency range of the sound, down-sample it to mainly
match the most prominent and relevant frequencies, and thereby cutting the file size in half. Usually 22 kHz sample rates
SOUND ENGINEERING
8
are used – and compared to film which could use sample rates of 48 -96 kHz, you can see that a single film sound could easily
be 4 times the size of a game audio file.
The second big difference is that in game audio sound is triggered at the control of the player, such as how the player moves
around, what they do in the game and the places and objects they handle. In film, there is a set sequence of events and every
scene sequentially follows one another. Because game audio is mostly player controlled, a single set or sequence of events
can occur as many times as a player wishes to trigger those events. To accompany the repetitive possibilities in games, a
huge amount of the audio assets can be looped indefinitely, such as music, footsteps, ambience, environmental elements
like water flowing or wind blowing. This means that, in film, a sound has a beginning, middle and end and doesn’t occur
more than once, but in game audio, sounds need to be edited so that they loop seamlessly without any noticeable silent
gaps.
SOUND ENGINEERING
9
Bringing them together
Session setup
To set up a session for ADR or foley, you’ll run the session a little differently to when you are recording music. You will need
a screen for the actor or foley engineer to follow the film as they perform their parts. You’ll also need to use a cardioid or rifle
condenser mic to capture all the finer details of the foley, as well as the dynamics in the dialogue. Check your gains and
make sure your sound at the maximum level doesn’t exceed -6 dB and averages around -18 dB. If you plan on recording
foley, make sure that you have a box that you can make a mess in. The last thing you want is fruit juice, sand or vegetable
pieces lying around your studio. Now that you’ve got the recording room organised, you can move over to your DAW.
First you need to make sure that the session sample rate is set at 48 kHz, 24-bit or 32-bit float. Second, you need to import
the video file. Depending on the DAW that you’re using, the required file format might differ, but MP4 videos usually work in
most DAW’s. Once you’ve got the session set up and video file imported, it’s time to set up the next part of the session.
When you record ADR or foley, you will focus on specific elements within a scene. Most likely you’ll need at two or three takes
for many of the dialogue and sound effects, and to make this recording process run faster, you should make use of looping
during the recording session. For ADR, follow the timecode on the cue sheet where the line needs to be delivered. Highlight
the section to be recorded with 2 seconds of room before and after the scene, turn the loop record feature on and start
recording.
Treating your ADR and Foley space
When you’re starting to decide on a foley space, consider turning a section of your live room or vocal booth into a foley stage.
To start out you’ll need to put some acoustic treatment on the walls. Once you’ve got that covered, you can begin
considering where you need to set up your foley stage. For the foley stage you’ll need to take some aspects into
consideration: Where can you put a box to make a mess in with gravel, sand and other messy things you might need for the
production. Remember, if you are going to make use of water, fruit, or other liquids, its best to keep them separate from the
foley sand box to avoid a muddy situation in studio. Just make sure that you can still see the screen you need to follow for
performing the foley.
The second aspect you’ll need to consider is the microphone that you’ll use. For foley you generally want to capture as much
of the direct sound and as little of the reverberation and early reflections as possible. To achieve this, you can use a rifle mic
for the best results. If you don’t have one, you can opt for a condenser mic with a cardioid polar pattern. This way you’ll still
capture more direct than reflected sound and will also minimise recording too much of the room tone. You want to capture
the sound as clean and dry as possible because you’ll likely repurpose these sounds and bus them out to a reverb that will
place the sounds in the right environment. Adding a natural sounding reverb to a sound that already has a bit of room tone
on it doesn’t always work because you not only add reverb to the original sound, but also to the room that it was in, which
means that you’re adding reverb on reverb – and this will more often than not create a sound that doesn’t fit the mix as well
as it could have.
The coolest tools in your toolbox
For the Additional resources section in the summary notes I’ve added loads of links, tools and tricks that will help you get
your career in film sound going. There are loads of software that addresses the tedious tasks of nudging syllables into place
to sync with video and other helpful plugins.
Things to get that will make your life easier:
• Music stand (cue sheets, lyrics, chords etc can be placed and doesn’t need to be waved around during
recording.
• Screen – you and the artist need to follow on screen ques all the time
• Foley box: Wooden box that is about 1 square meter wide and 30 cm deep
SOUND ENGINEERING
10
• Foley water box: 1 large container roughly the same size as foley box and 1 smaller container to contain
the water for splashes and swooshes. The thicker the container the better – you need this to be the
most waterproof object in the building. If water leaks in your studio, with all the cabling and electricity
around, it can easily turn into a hazard if your container leaks.
• Wooden chopping board: for foley sounds you’ll chop and squash more objects than you can imagine,
and it can get very messy very quickly. Make sure you cut over your foley water box to minimise the
mess
Additional resources
If you’ve never been on a film set, going out on a location shoot can be confusing to say the least. To know where to stand,
how to use the equipment and how to capture the best audio possible, Rode Microphones and Ric Viers delivered a crash
course video series on the best location audio practices.
This is a must watch!
• [Link]
Once you’ve completed your location shoot and its time for ADR, watch this video before you begin:
• [Link]
When you record ADR, you often need to manually nudge transients and syllables a few milliseconds to sync twith the
visuals. It can be a rather tedious and arduous task and can be costly for your clients. To simplify the sync process when you
are recording ADR, consider using this plug-in in conjunction with a guide dialogue track.
• [Link]
If you want to record foley, samples or just record some sounds around your house or neighbourhood, the video below will
help you identify the equipment and tools you need to make the most of the session.
• [Link]
mixers
SOUND ENGINEERING
11
References
• [Link]. 2021. Before you continue to YouTube. [online] Available at:
<[Link]
• Paul, M., 2021. [online] [Link]. Available at:
<[Link]
• [Link]. 2021. Revoice Pro 4 - Whats-new - Synchro Arts. [online] Available at:
<[Link]
• B&H Explora. 2021. Top 10 Essentials for Location Sound Mixers. [online] Available at:
<[Link]
sound-mixers>.
SOUND ENGINEERING