Fundamentals of Multimedia
Wolkite University - Department of Information Technology
2nd Edition 2014
Ze-Nian Li
Mark S. Drew
Jiangchuan Liu
Chapter 6 :
Basics of Digital Audio
1
❖Content:
6.1:Digitization of Sound.
6.2: MIDI: Musical Instrument Digital Interface.
2
❖Objectives:
This chapter explores audio information
since it’s crucial for multimedia
presentations.
So, in this chapter we introduce basic
concepts for sound in multimedia, and the
digitization of sound.
We begin with a discussion of just what makes up sound
information, then we go on to examine the use of MIDI as an
enabling technology to capture, store, and play back musical
notes.
3
What is sound?
Sound is a wave phenomenon like light, but is
macroscopic and involves molecules of air being
compressed and expanded under the action of some
physical device.
• For example, a speaker in an audio system vibrates
back and forth and produces a longitudinal pressure wave
that we perceive as sound.
Since sound is a pressure wave, it takes on continuous
values, as opposed to digitized ones with a finite range.
Without air there is no sound—for example, in space.
4
Sound Wave
5
• Even though such pressure waves are longitudinal,
they still have ordinary wave properties and behaviors,
such as reflection (bouncing), refraction (change of
angle when entering a medium with a different density)
and diffraction (bending around an obstacle).
• If we wish to use a digital version of sound
waves we must form digitized representations of
audio information.
9
Digitization
Digitization means conversion to a stream of
numbers, and preferably these numbers should be
integers for efficiency.
10
Digitization
An analog signal: continuous measurement of pressure wave.
• Sound is 1-dimensional (amplitude values depend on a 1D variable, time) as opposed to
images which are 2D (x,y)
11
The following graph has to be made digital in both time and
amplitude. To digitize, the signal must be sampled in each
dimension: in time, and in amplitude.
•Sampling means measuring the quantity we are interested in,
usually at evenly-spaced intervals.
•The first kind of sampling, using measurements only at evenly
spaced time intervals, is simply called, sampling. The rate at which
it is performed is called the sampling frequency.
•For audio, typical sampling rates are from 8 kHz (8,000 samples
per second) to 48 kHz. This range is determined by the Nyquist
theorem, discussed later.
•Sound is a continuous signal (measurement of pressure).
•Sampling in the amplitude or voltage dimension is called
quantization. We quantize so that we can represent the signal
as a discrete set of values. 12
Sampling and Quantization.
(a): Sampling the analog signal in the time dimension.
(b): Quantization is sampling the analog signal in the amplitude
dimension.
13
-Whereas frequency is an absolute measure, pitch is generally
relative — is a perceptual subjective quality of sound.
(a)Pitch and frequency are linked by setting the note A above
middle C to exactly 440 Hz.
(b) An octave above that note takes us to another A note. An
octave corresponds to doubling the frequency. Thus with the
middle “A” on a piano (“A4” or “A440”) set to 440 Hz, the next
“A” up is at 880 Hz, or one octave above. (A5)
(c) Harmonics: any series of musical tones whose frequencies are
integral multiples of the frequency of a fundamental tone: Fig in
next slide
(d) If we allow non-integer multiples of the base frequency, we
allow non-“A” notes and have a more complex resulting sound.
14
Signals can be
decomposed into
a weighted sum
of
sinusoids:
Building up a
complex signal
by superposing
sinusoids
15
To decide how to digitize audio data we need to answer
the following questions:
[Link] is the sampling rate?
[Link] finely is the data to be quantized, and is
quantization uniform?
3. How is audio data formatted? (file format)
16
• The Nyquist theorem states how frequently we must sample
in time to be able to recover the original sound.
(a)Figure (a) shows a single sinusoid: it is a single, pure, frequency
(only electronic instruments can create such sounds).
(b) If sampling rate just equals the actual frequency, Figure (b)
shows that a false signal is detected: it is simply a constant, with
zero frequency.
(c) Now if sample at 1.5 times the actual frequency, Figure (c)
shows that we obtain an incorrect (alias) frequency that is
lower than the correct one — it is half the correct one (the
wavelength, from peak to peak, is double that of the actual
signal).
(d) Thus for correct sampling we must use a sampling rate equal
to at least twice the maximum frequency content in the signal.
This rate is called the Nyquist rate.
17
(a): A single frequency.
(b): Sampling
at exactly
the frequency produces
a constant.
(c): Sampling at 1.5 times
per cycle produces an
alias perceived frequency.
18
• Nyquist Theorem:
If a signal is band-limited, i.e., there is a
lower limit f1 and an upper limit f2 of
frequency components in the signal, then the
sampling rate should be at least 2(f2 − f1).
19
Signal to Noise Ratio (SNR)
• The ratio of the power of the correct signal and the
noise is called the signal to noise ratio (SNR) — a
measure of the quality of the signal.
• The SNR is usually measured in decibels (dB), where 1
dB is a tenth of a bel. The SNR value, in units of dB, is
defined in terms of base- 10 logarithms of squared
amplitudes, as follows:
20
Audio Filtering
• Prior to sampling and AD conversion, the audio signal is also
usually filtered to remove unwanted frequencies. The
frequencies kept depend on the application:
(a) For speech, typically from 50Hz to 10kHz is retained, and
other frequencies are blocked by the use of a band-pass
filter that screens out lower and higher frequencies.
(b) An audio music signal will typically contain from about 20Hz
up to 20kHz.
(c) At the DA converter end, high frequencies may reappear in
the output — because of sampling and then quantization.
(d) So at the decoder side, a lowpass filter is used after the DA
circuit.
22
Synthetic Sounds
• Digitized sound must be converted to analog for us to hear it.
• There are two fundamentally different approaches to handling
stored sampled audio:
1. FM (frequency modulation)
2. WaveTable
23
Frequency Modulation
• In general,
Frequency Modulation is the process in which the frequency
of the carrier signal is varied by modulating signal while the
amplitude remains constant.
In FM approach, a carrier sinusoid is changed by adding another
term involving a second, modulating frequency.
A more interesting sound is created by changing the argument of
the main cosine term, putting the second cosine inside the
argument itself—then we have a cosine of a cosine.
A time-varying amplitude “envelope” function multiplies the
whole signal, and another time-varying function multiplies the
inner cosine, to account for overtones. Adding a couple of extra
constants, the
resulting function is complex indeed. 24
Wave Table
• Amore accurate way of generating sounds from digital signals is
called wave table Wave Table synthesis.
In this technique, digital samples are stored sounds from real
instruments. Since wave tables are stored in memory on the
sound card, they can be manipulated by software so that sounds
can be combined, edited, and enhanced.
Sound reproduction, is a good deal better with wave tables than
with FM synthesis.
28
MIDI: Musical Instrument Digital Interface
• MIDI Overview
(a)MIDI is a protocol adopted by the electronic music industry
in the early 80s for controlling devices, such as synthesizers and
sound cards, that produce music and allowing them to
communicate with each other.
(b) MIDI is a scripting language — it codes “events” that stand
for the production of sounds. E.g., a MIDI event might include
values for the pitch of a single note, its duration, and its volume.
29
(c) The MIDI standard is supported by most synthesizers,
so sounds created on one synthesizer can be played and
manipulated on another synthesizer and sound reasonably
close.
(d) Computers must have a special MIDI interface, but this
is incorporated into most sound cards.
(3) A MIDI file consists of a sequence of MIDI instructions
(messages). So, would be quite small in comparison to a
standard audio file.
30
MIDI Concepts
MIDI channels are used to separate messages.
(a)There are 16 channels numbered from 0 to 15. The
channel forms the last 4 bits (the least significant bits) of
the message.
(b) Usually a channel is associated with a particular
instrument: e.g., channel 1 is the piano, channel 10 is the
drums, etc.
(c) Nevertheless, one can switch instruments midstream, if
desired, and associate another instrument with any
channel.
31
• System messages
(a)Several other types of messages, e.g. a general message
for all instruments indicating a change in tuning or timing.
• The way a synthetic musical instrument responds to a
MIDI message is usually by simply ignoring any play sound
message that is not for its channel.
– If several messages are for its channel (say play multiple
notes on the piano), then the instrument responds,
provided it is multi-voice, i.e., can play more than a single
note at once (as opposed to violins).
32
• General MIDI: A standard mapping specifying what
instruments will be associated with what channels.
(a)For most instruments, a typical message might be a
Note On message (meaning, e.g., a keypress and release),
consisting of what channel, what pitch, and what “velocity”
(i.e., volume).
(b) For percussion instruments, however, the pitch data
means which kind of drum.
(c) A Note On message consists of “status” byte — which
channel, what pitch — followed by two data bytes. It is
followed by a Note Off message, which also has a pitch
(which note to turn off) and a velocity (often set to zero).
33
• The data in a MIDI status byte is between 128 and 255;
each of the data bytes is between 0 and 127. Actual
MIDI bytes are 10-bit, including a 0 start and 0 stop bit.
34
End of Chapter 6
36
Federal Democratic Repbuplic
of Ethiopia Ministry of
Education
Wolkite University College of
Computing and Informatics
Information Technology
Department
This Summary is an Online Content from this Book:
Ze-Nian Li and Mark S Drew, Fundamentals of Multimedia, 1/e, Prentice-Hall, 2004
It is edited for
Multimedia Systems Course- InTc2114
by:
Amanuel Getachew
37