Study Guide: Short-Time Speech Analysis & Time-Domain Parameters

1. The Big Idea: Why Analyze Speech in Short Windows?

 Speech is dynamic (constantly changing).
 However, for very short periods (like 5-200 ms), the vocal tract shape and sound
are relatively stable (quasi-stationary).
 We can analyze these short, stable chunks (windows or frames) to extract useful
parameters that describe the speech.
2. The Compromise: Time vs. Frequency Resolution
 Time Resolution: Ability to see when changes happen (e.g., finding the start of a sound).
Better with shorter windows.
 Frequency Resolution: Ability to see fine details in the spectrum (e.g., separating close-
together harmonics). Better with longer windows.
 You cannot have perfect both. You must choose a window size that is the
best compromise for your task.
3. Windowing: The Process
 What it is: Multiplying the speech signal s(n) by a shorter function called a window w(n).
 Purpose: To isolate a short segment of speech for analysis.
 The window is shifted in time to analyze the entire signal step-by-step (see Figure 6.1).
4. Choosing a Window: Size and Shape
 Size (Duration):
o Short windows (~5-10 ms): Good for capturing fast events (like a 't' or 'p' sound).
Good time resolution, poor frequency resolution.
o Long windows (~20-50 ms): Good for analyzing stable sounds (like vowels). Good
frequency resolution, poor time resolution.
 Shape: Determines how much we weight the samples in the middle of the window vs.
those at the edges.
o Rectangular Window: The simplest. Gives equal weight to all samples. Has sharp
edges which can cause problems.
o Hamming/Hanning Window: Very common. Has tapered (smooth) edges. This
reduces the bad effects of chopping the signal at an arbitrary point (like the
middle of a pitch period). It provides a smoother, more reliable analysis.
5. Window Shape's Effect on the Spectrum (Very Important!)
 The spectrum of the windowed signal X(ejω) is a smoothed (blurred) version of the true
speech spectrum S(ejω).
 The window acts like a low-pass filter.
 Rectangular Window: Has a narrower main lobe but higher side lobes. This can
cause spectral leakage (smearing energy across frequencies).
 Hamming Window: Has a wider main lobe but much lower side lobes. It
provides smoother and cleaner spectral estimates, which is usually preferred.
6. Wideband vs. Narrowband Spectrograms
 This is a direct result of the time-frequency resolution trade-off.
 Wideband Spectrogram:
o Uses a short window (~3 ms).
o Good time resolution, poor frequency resolution.
o The wide bandwidth (~300 Hz) smoothes out the harmonics, making
the formants (dark bands) clearly visible. Best for seeing vocal tract movements.
 Narrowband Spectrogram:
o Uses a long window (~20 ms).
o Good frequency resolution, poor time resolution.
o The narrow bandwidth (~45 Hz) resolves individual harmonics. The vertical
stripes are the harmonics. Best for measuring pitch (F0).
7. Time-Domain Parameters (The Simplest Measures)
 We transform the complex speech signal into simpler, slowly-varying parameter signals.
 The general formula is:
Q(n) = ∑ [ Transformation(s(m)) * Window(n-m) ]
 This is a smoothed version of some transformation of the speech signal.
8. Short-Time Energy and Short-Time Magnitude
 Short-Time Energy: T is a squaring operation. Q(n) = ∑ [s(m)² * w(n-m)]
o Pro: Emphasizes high-amplitude parts (like loud vowels).
o Con: Can have a large dynamic range.
 Short-Time Magnitude (Amplitude): T is an absolute value operation. Q(n) = ∑ [ |s(m)|
* w(n-m) ]
o Pro: Simpler to compute, avoids emphasizing loud parts too much.
 Uses:
o Segmenting speech into words, syllables, or voiced/unvoiced parts.
o Finding word endpoints in isolated word recognition.
o Voice Activity Detection (VAD) to save bandwidth in transmission systems.
9. FIR vs. IIR Windows for Smoothing
 FIR (Finite Impulse Response) Windows:
o What: Windows with a fixed number of samples (like Hamming, Rectangular).
o Pro: Simple, linear phase (fixed delay), can calculate output only when needed.
o Used for most basic short-time analysis.
 IIR (Infinite Impulse Response) Windows:
o What: Uses a recursive formula (e.g., Q(n) = a*Q(n-1) + T[s(n)]).
o Pro: Can be computationally cheaper.
o Con: Must calculate at the high original sampling rate; harder to manage delay.
10. Decimation: Reducing the Data Rate
 The parameter signals Q(n) vary slowly (low bandwidth).
  Therefore, we don't need to keep every single value. We can decimate (downsample)
them.
 Example: If speech is sampled at 10,000 Hz but Q(n) has a bandwidth of 50 Hz, we can
sample Q(n) at just 100 Hz. This is a 100:1 reduction in data!
11. Key Applications of These Techniques
 Automatic Speech Recognition (ASR): Segmenting speech, detecting voiced/unvoiced
parts.
 Speech Coding: Efficiently representing speech by transmitting parameters instead of
the raw waveform.
 Speech Synthesis: Controlling energy and pitch to make speech sound natural.
 Speech Transmission: Using Voice Activity Detection to save bandwidth.

6.3.2
This section explains the short-time energy and short-time amplitude, which are simple ways
to measure how "loud" a small segment of speech is.
 What it is: Imagine taking a short clip of speech (like 50ms) and either squaring the
signal (for energy) or just taking its absolute value (for amplitude). You then slide this
window along the entire audio signal to see how the loudness changes over time.
 The Difference:
o Energy emphasizes louder parts more (because squaring a large number makes it
much larger).
o Amplitude is a simpler, more balanced measure.
 Why it's Useful: Loudness changes a lot in speech. Voiced sounds (like vowels) are loud,
while unvoiced sounds (like 's' or 'f') or silence are quiet. This makes it great for:
1. Finding Word Boundaries: Automatically detecting the start and end of a spoken
word in a recording.
2. Segmenting Speech: Breaking continuous speech into smaller chunks, like
syllables.
3. Saving Bandwidth: In phone systems, it can identify silence so that bandwidth
isn't wasted transmitting it.
From 6.3.3 to 6.5.6
Study Guide: Speech Analysis (Time & Frequency Domain)
1. Short-Time Average Zero-Crossing Rate (ZCR)
 What it is: A simple measure of how often the speech signal crosses the zero-

amplitude line (changes sign).

 How it's calculated: For a window of speech, count the number of zero-

crossings per second.

 What it tells us:
 o High ZCR: Indicates high-frequency energy, typical of unvoiced
sounds (like /s/, /f/). (~4900 crossings/s)
o Low ZCR: Indicates low-frequency energy, typical of voiced
sounds (vowels). (~1400 crossings/s)
 Limitations:

o Very sensitive to noise (e.g., 60 Hz hum).

o Can be ambiguous for voiced fricatives (like /z/, /v/) which have both

low and high-frequency energy.

 Uses: A cheap way to get rough spectral info and help with voicing decisions.

2. Short-Time Autocorrelation Function 6.3.4

 What it is: A measure of the similarity between a signal and a time-shifted

version of itself.
 Key Properties:

o Maximum value is always at delay k=0 (perfect alignment).

o For periodic (voiced) speech, it will have peaks at delays equal to the

pitch period (k = P, 2P, 3P...).

o For noise (unvoiced) speech, it will quickly fall off to zero.

 Why it's useful: It's excellent for pitch (F0) detection and forms the

mathematical basis for Linear Predictive Coding (LPC).

 Visual Cue: In figures, look for a large peak at k=0 and repeating smaller

peaks for voiced speech. No repeating peaks for unvoiced speech.

3. Average Magnitude Difference Function (AMDF)
 What it is: An alternative to autocorrelation for pitch detection. Instead of

multiplying samples, it sums the absolute differences between a signal and

its delayed version.
 Key Difference: Where autocorrelation has peaks at the pitch period, AMDF

has valleys (minima).

 Advantage: Much simpler to compute (uses subtraction, not multiplication).

4. Frequency-Domain (Spectral) Analysis 6.4

 The Main Idea: Speech is easier to analyze based on its frequency content

(spectrum) than its raw waveform. This matches the source-filter model of
speech production.
 Two Main Methods:
 1. Filter-Bank Analysis: Speech is passed through a set of bandpass filters
(e.g., following the Bark or mel scale). The output energy from each
filter gives a simple spectral representation.
2. Short-Time Fourier Transform (STFT): The standard method. A window
is moved along the speech signal, and a Fourier transform is calculated
for each windowed segment. This creates a time-varying spectrum.
5. The Spectrogram: A Visual Tool
 What it is: A 2D plot that shows frequency (vertical axis) vs. time (horizontal

axis). The darkness indicates signal energy at that frequency and time.
 Two Types:

o Wideband Spectrogram: Uses a short window (~3 ms).

 Shows: Good time resolution. You see vertical striations for

each pitch period and dark horizontal bands for formants.

 Best for: Seeing vocal tract movements and formant transitions.

o Narrowband Spectrogram: Uses a long window (~20 ms).

 Shows: Good frequency resolution. You see horizontal stripes for

individual harmonics.
 Best for: Measuring pitch (F0).

6. Pre-Emphasis
 What it is: A simple high-pass filter applied to speech before analysis.

Formula: y(n) = s(n) - A*s(n-1) (α is usually ~0.95).

 Why it's done: Voiced speech has less energy at high frequencies. Pre-

emphasis boosts the high frequencies, making the overall spectrum "flatter".
This helps in formant estimation and LPC analysis.
 De-emphasis: The inverse filter, applied during synthesis to restore the

original spectral balance.

7. Formant Estimation & Tracking
 What are formants? Resonant frequencies of the vocal tract (F1, F2, F3, etc.).

They are the dark bands on a wideband spectrogram.

 The Challenge: Automatically finding and tracking them as they move over

time is difficult.
 Why it's hard:

o Formants can merge (e.g., F1 and F2 in some vowels).

o Nasal sounds introduce extra resonances and "zeros".

 o Obstruent consonants (stops, fricatives) abruptly change the spectrum.
o High-pitched voices (e.g., children's) have harmonics too far apart to

clearly define a formant shape.

 Methods:

o Peak-Picking: Finding the peaks in a spectrum (from STFT or LPC).

o Root-Solving: Using LPC to find the poles of the vocal tract filter, which

correspond to formants.
o Chirp Z-Transform (CZT): A clever version of the Fourier transform that

can provide better frequency resolution to separate close formants.

8. Linear Predictive Coding (LPC) Analysis (The Star of the Show) 6.5
 The Core Idea: A current speech sample can be predicted by a linear

combination of its past p samples. The coefficients (a₁, a₂, ... aₚ) that best do
this describe the vocal tract shape.
 The Model: It assumes an all-pole model (like a tube with resonances but no

anti-resonances). This is a good approximation for many speech sounds.

 The Process: The goal is to find the LPC coefficients that minimize

the prediction error (the difference between the real speech and the
predicted speech). This error signal is called the residual.
 Why it's so popular:

o It provides a very compact and efficient representation of speech (just

a few coefficients).
o It directly gives a smooth estimate of the spectral envelope (hiding the

harmonic details).
o It's computationally efficient (using the Levinson-Durbin algorithm).

 Key Outputs:

o LPC Coefficients (aₖ): Describe the vocal tract filter.

o Gain (G): Related to the energy of the signal.

o Residual (e(n)): Contains the excitation information (pitch and voicing).

 Autocorrelation Method vs. Covariance Method:

o Autocorrelation: Windows the speech signal. Simpler, guarantees a

stable filter, uses the efficient Levinson-Durbin recursion.

o Covariance: Windows the error signal. More accurate but more

complex, doesn't guarantee stability.

9. Important LPC Concepts
  Model Order (p): The number of past samples used for prediction. A trade-
off:
o Too low (p=8): Can't model all formants.

o Just right (p=10-12 for 8 kHz speech): Models ~5 resonances (good for

formants).
o Too high (p=20): Starts modeling the harmonic structure, not just the

envelope.
 Reflection Coefficients (kₘ): An alternative to LPC coefficients. They have a

physical interpretation related to acoustic tubes. Their magnitude is always ≤

1, which guarantees filter stability.
 Spectral Modeling: The LPC spectrum |H(e^jω)| models
the peaks (formants) of the speech spectrum very well but does a poor job
with the valleys. This is because the error minimization process cares more
about peaks.
10. Advanced & Alternative Methods
 Energy Separation Algorithm (ESA): Analyzes tiny amplitude and frequency

modulations (AM-FM) within a single pitch period, supposedly caused by

nonlinear air flow.
 Sample-by-Sample (Adaptive) LPC: Instead of processing frames, LPC

coefficients are updated for every new sample. Used in real-time applications
like echo cancellation. Methods include:
o Gradient/LMS Algorithm: Simpler, less accurate.

o Kalman Filter: More complex, more accurate.

Study Guide: Advanced Speech Analysis Topics

1. Practical LPC Considerations (Sec. 6.5.9)
 Window Size (N) vs. Model Order (p):

o p is fixed by the speech bandwidth (e.g., p=10 for 4kHz bandwidth).

o N (window size) is a trade-off: larger N gives better spectral accuracy

but more computation.

 Pitch-Asynchronous vs. Pitch-Synchronous Analysis:

o Pitch-Asynchronous (Standard): Windows are placed at fixed intervals,

ignoring the pitch period. This is simpler but can lead to poor estimates
if a window cuts through a high-energy pitch pulse (glottal epoch).
 o Pitch-Synchronous: Windows are carefully aligned to contain a single
full pitch period. This is more accurate but requires a reliable and
complex pitch detector first.
 Problems with High F0 Voices: For children or singers, many pitch pulses

occur in a standard window. LPC might mistakenly lock onto a single

harmonic instead of the true formant center. Solutions involve pre-
processing the spectrum to smooth out the harmonics before LPC.
2. Modifications to Standard LPC (Sec. 6.5.10)
 Perceptual Linear Prediction (PLP): A very popular modification for speech

recognition.
o What it does: It warps the spectrum to mimic the human ear's critical

bands (Bark scale), compresses the amplitude (logarithmically), and

applies an equal-loudness curve.
o Why it's good: It creates speaker-independent features that are more

robust for recognizing words, ignoring details like precise pitch.

 Emphasizing Low Frequencies: Our ears are more sensitive to lower

frequencies (where F1 and F2 are). Modifications try to make LPC focus more
on this important region, for example by using a frequency-weighted error or
warping the frequency axis (like in PLP).
3. Pole-Zero Models (ARMA) (Sec. 6.5.12)
 The Idea: Standard LPC uses an all-pole (AR) model. A pole-zero
(ARMA) model includes both poles and zeros.
 Pros: Can theoretically model speech more accurately, especially for sounds

with spectral zeros (like nasals).

 Cons: The math becomes much more complex (non-linear equations) and

computationally expensive. The improvement in quality is often small, so it's

rarely used in practice compared to simple all-pole LPC.
4. Cepstral Analysis (Sec. 6.6)
 The Core Idea: A clever way to separate the excitation (pitch) from the vocal

tract filter (formants) by using the inverse Fourier tr

o n the cepstrum at the value of the pitch period.

o Vocal Tract: Shows up as the slowly varying part near the beginning

(n=0) of the cepstrum.

 Why "Cepstrum"? It's a funny name made bansform of the log spectrum.
  Excitation: Shows up as a sharp peak iy reversing the first half of "spectrum"
(spectrum -> cepstrum). Similarly, "quefrency" is for frequency.
 Real Cepstrum vs. Complex Cepstrum: The real cepstrum (using the log-

magnitude spectrum) is used most often. The complex cepstrum (which also
uses phase) is more powerful but much more difficult to compute.
 Applications: (Sec. 6.6.2)

o Pitch (F0) Detection: Find the peak in the cepstrum corresponding to

the pitch period.

o Formant Tracking: The Fourier transform of the low-time part of the

cepstrum gives a smooth "cepstrally-smoothed" spectrum, perfect for

seeing formants without harmonic clutter.
5. Mel-Frequency Cepstral Coefficients (MFCCs) (Sec. 6.6.3)
 The #1 Feature for Speech Recognition: This is the most important

takeaway.
 How they are made:

1. Take the DFT of a windowed speech frame.

2. Map the powers onto the mel scale (which approximates human
hearing) using triangular filter banks.
3. Take the log of the power in each mel filter.
4. Take the Discrete Cosine Transform (DCT) of these log powers.
5. The first 12-13 coefficients are the MFCCs.
 What they represent: The MFCCs compactly represent the smooth spectral

envelope (the vocal tract shape), stripped of excitation details like

pitch. MFCC[0] is the average energy. MFCC[1] roughly represents the tilt of
the spectrum (vowel vs. fricative). Higher coefficients represent finer spectral
details.
6. Other Spectral Methods (Sec. 6.7)
 Karhunen-Loève Transform (KLT): The optimal transform for compressing

data. It finds the best basis functions for each specific speech frame. It's
computationally massive and not used much in practice.
 Wavelet Transform: Uses filters with variable bandwidth—wide for low

frequencies (good frequency resolution) and narrow for high

frequencies (good time resolution). This matches human hearing better than
a fixed-bandwidth Fourier transform but is more complex.
  Wigner Distribution: A method that can show good time and frequency
resolution simultaneously. However, it creates confusing "interference
terms" (artifacts) for signals like speech with multiple components, making it
hard to use.
7. F0 (Pitch) Estimation (Sec. 6.8)
 The Challenge: Finding the fundamental frequency of voiced speech is tricky

due to its wide range, non-stationarity, and noise.

 Two Main Approaches:

o Time-Domain: Look for periodicity in the waveform itself (e.g., finding

the time between consecutive peaks). Faster, gives epoch locations,

but less accurate.
o Frequency-Domain: Look for harmonic structure in the spectrum (e.g.,

finding the common divisor of all harmonics). More accurate,

but slower and doesn't give epoch times.
 Common Problems:

o Doubling/Halving: Mistaking the first harmonic for F0 (doubling) or

missing every other pulse (halving).

o Voicing Decision: Deciding if a frame is voiced or unvoiced is a separate

but related problem.

 Pre-processing Tricks: Center-clipping or infinite clipping the speech
waveform can simplify it and make the periodicity easier to find for time-
domain methods.
 A Good Compromise: The AMDF function is a simple, effective method that

often ranks high in subjective tests for coders.

CHAPTER 8
Study Guide: Speech Enhancement (SE)
1. Objectives of Speech Enhancement (Sec. 8.2)
 Goal: To improve a noisy speech signal.

 Two Main Objectives:

1. Improve Quality (Naturalness): Make the speech sound clearer and

less fatiguing to listen to. This is the goal for medium-to-high SNR (>5
dB).
2. Improve Intelligibility: Make the words easier to understand. This is the
critical goal for low SNR situations.
 Key Challenges:

o Detecting when speech is present vs. when there's only noise.

o Enhancing weak, unvoiced sounds (like 'f', 's') which are easily lost in
noise.
o Handling non-stationary (changing) noise.
o Working in real-time with low computational cost.
 Uses: For human listeners, as a pre-processor for speech coders, or for

automatic speech recognizers (ASR). The method may differ based on the
end goal (e.g., sounding natural vs. being accurately recognized by a
machine).
2. Nature of Interfering Sounds (Sec. 8.3)
 Types of Noise:

o Continuous Broadband Noise: (e.g., static, background babble). Most

common target for SE.

o Impulsive Noise: (e.g., clicks, pops). Short, loud bursts.

o Periodic Noise: (e.g., 60 Hz hum from power lines, machinery). Energy

at specific frequencies.
o Convolutive Noise: (e.g., echo, reverberation). Caused by the
environment.
o Interfering Speakers: The "cocktail party problem." Difficult because it

occupies the same frequency range as the desired speech.

 A Key Insight: For non-additive noise (like multiplicative or convolutive), we

can often use a logarithmic transformation to convert it into an additive

noise problem, which is easier to solve.
3. Four Classes of SE Techniques (Sec. 8.4)
 1. Spectral Subtraction & Filtering: Estimate and subtract the noise spectrum
from the noisy speech spectrum.
2. Harmonic Filtering: Identify the pitch (F0) and harmonics of the desired
speech and suppress everything else.
3. Parametric Resynthesis: Use a speech model (like LPC) to analyze the noisy
speech and then re-synthesize a "clean" version from the estimated
parameters.
4. Subtraction with a Reference Signal: Use a second microphone to get a noise
reference and subtract it from the primary signal. This is the most effective
method if available.
4. Spectral Subtraction (SS) (Sec. 8.5) - The Most Common Method
 The Basic Idea:

1. Take the Fourier Transform of the noisy speech P(ω) and of the
estimated noise I(ω).
2. Subtract the noise magnitude from the speech magnitude: |P(ω)| - α|
I(ω)|. (α is an overestimation factor, often 1.5, to prevent under-
subtraction).
3. Set any negative results to zero ("half-wave rectification").
4. Combine this new magnitude spectrum with the original phase from
the noisy speech.
5. Perform an Inverse Fourier Transform to get the enhanced time-
domain signal.
 Pros: Simple, intuitive, and effective at reducing noise power (improves

quality).
 Cons:

o Introduces "musical noise": annoying tonal artifacts caused by the

random remnants of noise in frequencies that were set to zero.
o Often reduces intelligibility because it also suppresses weak speech
components (like fricatives) along with the noise.
5. Filtering and Adaptive Noise Cancellation (ANC) (Sec. 8.6)
 Wiener Filtering: A more formal version of spectral subtraction. It designs a

filter that minimizes the mean-square error between the clean signal and the
estimate. It suppresses frequencies where the noise is strong relative to the
speech.
  Adaptive Noise Cancellation (ANC): 8.6.2 The best method if you have two
microphones.
o Primary Mic: Picks up desired speech + noise.

o Reference Mic: Placed near the noise source, picks up mostly just noise.

o How it works: An adaptive filter (e.g., using the Least-Mean-Squares -

LMS - algorithm) continuously adjusts itself to make the reference

signal match the noise in the primary signal. Once matched, it subtracts
it out.
o The Challenge: The filter must adapt to changes in the noise and avoid

adapting when the desired speech is present, otherwise it will cancel

the speech too!
o Uses: Very effective for canceling echo in telephony and consistent

noise like engine hum in a cockpit.

6. Methods Involving Fundamental Frequency (F0) Tracking (Sec. 8.7)
 Comb Filtering: Used when the desired speech is voiced (periodic).

o The Idea: Create a filter whose frequency response looks like a comb—

it has peaks at the fundamental frequency (F0) and its harmonics, and
valleys in between.
o Effect: It amplifies the harmonic frequencies of the target speaker

and suppresses the noise in the valleys between them.

o How it's done: In the time domain, it's essentially an averaging of the

signal over multiple pitch periods. This reinforces the periodic part and
cancels out the non-periodic noise.
o Limitations:

 Requires a very accurate and reliable pitch tracker.

 Works poorly if the pitch is changing quickly.

 Only works on voiced speech; useless for unvoiced sounds or

noise.
7. Enhancement by Resynthesis (Sec. 8.8)
 The Idea: Use a speech production model (like the LPC vocoder model).

1. Analyze the noisy speech to estimate parameters: LPC coefficients

(vocal tract filter), gain (volume), and a voicing/F0 decision (excitation).
2. Use these parameters to drive a clean synthesizer.
 Pros: The output speech can be completely free of the original noise.
  Cons: The output often sounds "mechanical" or robotic (like a vocoder). The
quality depends entirely on how well the parameters were estimated from
the noisy input, which is very difficult.

CHAPTER 9
Study Guide: Speech Synthesis
1. Principles of Speech Synthesis (Sec. 9.2)
 What it is: The process of converting text into artificial speech.

 The Core Trade-off: Quality vs. Memory.

o Large Units (Words/Phrases): High quality but require huge memory.

Not suitable for unlimited text.

o Small Units (Phonemes): Low memory (only need ~40 sounds) but

sound robotic due to coarticulation (how sounds blend into each

other). Requires complex rules to smooth the transitions between
units.
2. Types of Stored Speech Units (Sec. 9.2.1)
The choice of unit is a balance between naturalness and storage needs.
 Phrases/Sentences: Best quality, but extremely limited. Used for very

specific messages (e.g., "Fasten your seatbelt").

 Words: Good for limited vocabulary systems.

 Diphones: The most common unit for high-quality synthesizers. A diphone

is the sound from the middle of one phoneme to the middle of the next.
This captures the transitions between sounds, which is where most
coarticulation happens. (e.g., the word "straight" is built from diphones
like /s-t/, /t-r/, /r-eɪ/, etc.).
 Demisyllables: Syllables split in half, through the vowel. Also good for

capturing transitions.
 Phonemes: The smallest units. Most efficient for memory but produce the

worst, most robotic-sounding speech because the transitions between them

are not natural.
3. Synthesis Method (Sec. 9.2.3)
How the speech is parameterized and generated.
 Waveform Concatenation: Highest quality. Stores and pieces together actual

snippets of recorded speech (like diphones). Requires a lot of memory.

  Parametric Synthesis (Vocoders): Lower quality, but efficient. Does not
store speech waves. Instead, it stores parameters that describe the speech
(e.g., LPC coefficients, formant frequencies). A synthesizer uses these
parameters to generate speech from a model. This is how unlimited text-to-
speech (TTS) systems work.
 Articulatory Synthesis: The most complex method. Tries to physically model

the vocal tract (tongue, lips, etc.) to produce sound. It is a research topic and
not used commercially because it is very difficult to get right.
4. Synthesizer Methods (Sec. 9.3)
A deeper look into how synthetic speech is generated.
4.1 Formant Synthesis (Sec. 9.3.2)
A type of parametric synthesis that models the key resonant frequencies
(formants) of the vocal tract.
 The Model: It uses a source ( buzz for vowels, hiss for fricatives) and a filter

(a series of resonators that shape the source into vowels and consonants).
 Two Main Structures:

o Cascade Formant Synthesizer: Resonators are connected in a series. It's

very good for modeling vowels naturally.

o Parallel Formant Synthesizer: Resonators are connected side-by-side.

It's better for modeling consonants (like stops and fricatives) because it
allows more independent control over the amplitude of each formant.
 How it's Controlled: The synthesizer is driven by time-varying
parameters: Formant Frequencies (F1, F2, F3...),
Bandwidths, and Amplitudes. Rules dictate how these parameters change to
produce different sounds.
4.2 Synthesis of Nasals (Sec. [Link])
 The Challenge: Nasal sounds (like /m/, /n/) are hard because they involve

a second acoustic tube (the nasal cavity) branching off from the main vocal
tract.
 The Effect: This extra tube introduces an extra resonance (a new formant)

and, crucially, a spectral zero (an anti-resonance where energy is sucked

out).
 The Solution: To synthesize nasals accurately, the model must include:

1. An extra resonator to model the new nasal formant.

 2. An antiresonator (a filter that creates a dip or zero in the spectrum) to
model the spectral zero.

What is a Fricative?
A fricative is a type of consonant sound (like /f/, /s/, or /sh/) that is produced by
forcing air through a narrow channel or constriction made by your articulators (like
your tongue, teeth, or lips). This creates a turbulent, hissing, or friction-like sound.

Key Characteristics:
1. The "Friction" Sound: The most defining feature. Fricatives are continuous
sounds that can be held out, like the "ssssss" in snake or the "shhhhh"
in ship. This distinguishes them from stops (like /p/, /t/, /k/), which are quick,
plosive sounds.
2. Voiced vs. Unvoiced: Fricatives often come in pairs that use the same mouth
shape but differ in vocal cord vibration.
o Unvoiced (Voiceless): No vibration in the vocal cords. Just the sound of

air friction.
 /f/ as in fan

 /θ/ as in thin

 /s/ as in sip

 /ʃ/ as in shoe

o Voiced: Your vocal cords vibrate, creating a buzz along with the friction.

 /v/ as in van

 /ð/ as in this
  /z/ as in zoo
 /ʒ/ as in measure

3. Place of Articulation: They are classified by where in the vocal tract the
constriction happens.
o Labiodental: Lower lip against upper teeth (/f/, /v/)

o Dental: Tongue against teeth (/θ/ , /ð/)

o Alveolar: Tongue against the alveolar ridge (bumpy spot behind teeth)

(/s/, /z/)
o Post-alveolar/Palatal: Tongue towards the hard palate (/ʃ/, /ʒ/)

Why are they important?

 In Speech: They are crucial for distinguishing words (e.g., fan vs. van, sip

vs. zip).
 In Speech Technology: They are often the most difficult sounds to analyze

and synthesize because their energy is spread out as noise, unlike the clear,
structured harmonics of vowels. They are also

What is a Formant?
A formant is a concentration of acoustic energy around a particular frequency in
the speech spectrum. In simple terms, it's a resonant peak in the sound wave that
makes up a vowel or voiced consonant.
Think of the vocal tract (your throat, mouth, and nose) as a musical instrument like
a flute. When you blow into a flute, the column of air inside it resonates at specific
frequencies, which determines the note's pitch and quality.
Similarly, when sound from your vocal cords passes through your vocal tract, the
shape of your tract (determined by your tongue, lips, and jaw) amplifies some
frequencies and dampens others. The resulting amplified frequency bands are the
formants.
Why are Formants Important?
 Speech Perception: Our brains use formant patterns to identify which vowel

we are hearing.
 Speech Technology: They are absolutely crucial for:

o Speech Synthesis: Formant synthesizers work by generating these

resonant peaks to create artificial vowels.

 o Speech Recognition: Software analyzes the formant patterns to identify
which words are being spoken.
o Speech Coding: Efficiently transmitting speech (e.g., in cell phones)

often involves coding formant information rather than the full sound
wave.
In short, a formant is the acoustic fingerprint of a vowel, created by the resonant
frequencies of the vocal tract. It's a core concept for understanding how speech
works.