Module 4: Numerical Calculus (Weeks 7-8)

This module addresses a fundamental challenge: what do you do when you need to
calculate a derivative or an integral, but you don't have a simple formula for the
function? This happens all the time in science and engineering, where functions might
be defined by complex simulations or, more commonly, by a set of discrete data points
from an experiment.

The core idea of numerical calculus is to approximate these operations using the data
points we do have.

Topic 7: Numerical Differentiation

The Goal: Approximate the derivative f'(x) of a function at a specific point x, given only
discrete data points like (x₀, f(x₀)), (x₁, f(x₁)), etc.

Why? We might not know the analytical form of f(x), or the derivative might be too
complex to compute directly. The derivative represents the instantaneous rate of
change or the slope of the tangent line at a point.

A. Finite Difference Approximations

The geometric intuition is to approximate the slope of the tangent line at x_i with the
slope of a secant line passing through nearby points. The distance between the points is
the step size, h.

1. Forward Difference

• Core Idea: Approximate the slope by looking "forward" to the next point.

• Formula: f'(x_i) ≈ [f(x_i + h) - f(x_i)] / h

• Geometric Interpretation: The slope of the secant line connecting the points (x_i,
f(x_i)) and (x_i + h, f(x_i + h)).

• Use Case: Useful when you need the derivative at the very end of a dataset and
can't look "backward."

2. Backward Difference

• Core Idea: Approximate the slope by looking "backward" to the previous point.

• Formula: f'(x_i) ≈ [f(x_i) - f(x_i - h)] / h

• Geometric Interpretation: The slope of the secant line connecting the points (x_i
- h, f(x_i - h)) and (x_i, f(x_i)).
 • Use Case: Useful when you need the derivative at the very beginning of a dataset
and can't look "forward."

3. Central Difference

• Core Idea: Approximate the slope by looking at points on both sides of x_i. This is
generally the most accurate of the three.

• Formula: f'(x_i) ≈ [f(x_i + h) - f(x_i - h)] / (2h)

• Geometric Interpretation: The slope of the secant line connecting the points (x_i
- h, f(x_i - h)) and (x_i + h, f(x_i + h)). This line is often a much better
approximation of the tangent line.

• Use Case: The preferred method for interior points where both forward and
backward data are available.

B. Error Analysis of Derivative Formulas

The approximation is not perfect. The error comes from truncation error, which arises
from replacing a function with a simpler approximation (derived from its Taylor series).

• Forward & Backward Difference: The truncation error is proportional to the step
size h. We write this as O(h), meaning the error decreases linearly as h gets
smaller. If you halve h, you roughly halve the error.

• Central Difference: The truncation error is proportional to h². We write this as

O(h²). This is a huge improvement! If you halve h, you reduce the error by a factor
of four (2²).

The Trade-off:

• As h gets smaller, truncation error decreases.

• However, as h gets very small, subtractive cancellation (a form of round-off error)

can increase. When you subtract two nearly equal numbers (f(x_i + h) - f(x_i - h)),
you lose significant digits, and the division by a tiny 2h can amplify this error.

• Conclusion: There is an optimal h that balances these two types of error. In

practice, the central difference is almost always preferred for its superior O(h²)
accuracy.

Topic 8: Numerical Integration (Quadrature)

The Goal: Approximate the definite integral ∫[a,b] f(x) dx, which represents the area
under the curve of f(x) from a to b.
Why? Many functions are impossible to integrate analytically (e.g., e^(-x²)), or we might
only have data points, not a function definition.

A. Newton-Cotes Formulas

The core idea is to replace the complex function f(x) with a simpler polynomial P(x) that
is easy to integrate. The different formulas come from using different polynomials.

1. Trapezoidal Rule

• Core Idea: Approximate the area under the curve by a series of trapezoids.

• Geometric Interpretation: Connect consecutive data points with straight lines.

The area under each line segment is a trapezoid.

• Composite Trapezoidal Rule (Most Practical): Break the interval [a, b] into n
subintervals of width h = (b-a)/n and sum the areas of all the trapezoids.

• Formula: ∫[a,b] f(x) dx ≈ (h/2) * [f(x₀) + 2f(x₁) + 2f(x₂) + ... + 2f(x_{n-1}) + f(x_n)]
(Note: The first and last terms have a coefficient of 1, all interior points have a
coefficient of 2).

• Error: The truncation error is O(h²). To halve the error, you need to halve the step
size h.

2. Simpson's Rule

• Core Idea: Achieve higher accuracy by approximating the area with parabolic
arcs instead of straight lines. A parabola can "hug" the curve of a smooth
function much better.

• Geometric Interpretation: Fit a parabola through every three consecutive points.

• Composite Simpson's Rule: Break the interval [a, b] into n subintervals, where n
must be an even number.

• Formula: ∫[a,b] f(x) dx ≈ (h/3) * [f(x₀) + 4f(x₁) + 2f(x₂) + 4f(x₃) + ... + 2f(x_{n-2}) +
4f(x_{n-1}) + f(x_n)] (Note: The pattern of coefficients is 1, 4, 2, 4, 2, ..., 4, 1).

• Error: The truncation error is O(h⁴). This is dramatically better than the
Trapezoidal Rule! If you halve h, you reduce the error by a factor of sixteen (2⁴).
Simpson's Rule is often the go-to method for its excellent balance of simplicity
and accuracy.

B. Introduction to Adaptive Quadrature

• The Problem with Composite Rules: They use a uniform step size h across the
entire interval. This is inefficient if the function f(x) is smooth and gentle in some
regions but very "wiggly" or rapidly changing in others. The wiggly regions need a
small h for accuracy, but the smooth regions don't.

• Core Idea of Adaptive Methods: Intelligently change the step size h depending on
the behavior of the function.

o Use a small h where the function is changing rapidly.

o Use a large h where the function is smooth.

• How it Works (Simplified):

1. Apply an integration rule (like Simpson's) to an interval.

2. Split the interval in half and apply the rule to the two sub-intervals.

3. Compare the result from step 1 with the sum of the results from step 2.

4. If the difference is small (below a tolerance), the approximation is good

enough.

5. If the difference is large, the function is "wiggly" in that interval, so

recursively repeat the process on the two sub-intervals.

• Benefit: Achieves a desired level of accuracy across the whole interval with far
fewer function evaluations, making it much more efficient than a simple
composite rule for complex functions.