Math Review
Aria Nosratinia Probability and Statistics 1-1
Sequences and Limits
A sequence is just an innite string of numbers, e.g.
1 2 3 4 5 6
0 0 0 0 0 0
1 1 1 1 1 1
1
1
2
1
4
1
8
1
16
1
32
We often show a sequence as x(n) or x[n] or x
n
.
We say a sequence has a limit x
if for each > 0 there exists a
natural number N such that
n > N |x
n
x
| <
Limit is a property of the tail of the sequence.
Which one of the sequences above have a limit?
Aria Nosratinia Probability and Statistics 1-2
Functions
A function is a mapping from a domain to a range
Example: y(t) = sin(t), where the domain is R and the range is
[1, 1].
For any function, to each value of the domain, one and only one
value is assigned in the range
Range
Domain
R R
f
f
x
f(x)
Aria Nosratinia Probability and Statistics 1-3
One-to-one Functions, Limit
If each value in the range has only one corresponding value in the
domain, we call the function one-to-one.
not 1-to-1 1-to-1 1-to-1
We say a function has a limit lim
xx
0
f(x) = f
0
if for any > 0
there exists a > 0 such that
|x x
0
| < |f(x) f
0
| <
Aria Nosratinia Probability and Statistics 1-4
Continuity
We say a function is continuous at point x
0
if the limit of the
function at this point exists and is equal to f(x
0
).
Note: limits do not always exist. For example, the function:
f(x) = sin(
1
x
)
has no limit at x = 0.
We can dene a limit from the right or the left, leading to
continuity from the right or the left.
Continuous Right-Continuous Left-Continuous
Aria Nosratinia Probability and Statistics 1-5
Derivative
dy/dt
t t
y
dt
dy
Derivative is the slope of the curve representing a function
Denition:
d
dx
f(x)
x
0
= lim
xx
0
f(x) f(x
0
)
x x
0
Aria Nosratinia Probability and Statistics 1-6
Properties of the Derivative
_
cf(x)
_
= cf
(x)
_
f(x) + g(x)
_
= f
(x) + g
(x)
_
f(x)g(x)
_
= f
(x)g(x) + f(x)g
(x)
_
f(x)
g(x)
_
=
f
(x)g(x) f(x)g
(x)
g
2
(x)
_
f(g(x))
_
= f
_
g(x)
_
g
(x) Chain rule
Aria Nosratinia Probability and Statistics 1-7
Common Derivatives
d
dx
x
n
= nx
n1
d
dx
e
x
= e
x
d
dx
log x =
1
x
d
dx
sinx = cos x
d
dx
cos x = sin x
d
dx
tanx = sec
2
(x)
d
dx
sin
1
(x) =
1
1 x
2
d
dx
cos
1
(x) =
1
1 x
2
d
dx
tan
1
(x) =
1
1 + x
2
d
dx
sinh(x) = cosh(x)
d
dx
cosh(x) = sinh(x)
d
dx
tanh(x) = sech
2
(x)
Aria Nosratinia Probability and Statistics 1-8
Integration
x
f(x)
a b
The integral is the area under the curve representing a function.
Denition:
_
b
a
f(x)dx = lim
0
ba
n=0
f(a + n)
The above formula is known as a denite integral. It is a number.
Aria Nosratinia Probability and Statistics 1-9
Indenite Integrals
y(x) dx
t
x
y
t
0
0
t
t
0
The value of a denite integral depends on its boundaries.
We can x one boundary, make the other a variable.
The result is the indenite integral.
Aria Nosratinia Probability and Statistics 1-10
Fundamental Theorem of Calculus
Shows the relation between derivative and integral. Has two parts.
Part I: If for all x [a, b]
F(x) =
_
x
a
f(t) dt
then F(x) is dierentiable, and F
(x) = f(x).
Part II: If f(x) = F
(x), then:
_
b
a
f(x) dx = F(b) F(a)
Aria Nosratinia Probability and Statistics 1-11
Useful Integration Formulas
Integration by substitution (change of variable)
_
b
a
f
_
g(t)
_
g
(t) dt =
_
g(b)
g(a)
f(x)dx
Integration by parts
_
b
a
f(x)g
(x) dx =
_
f(x)g(x)
_
b
a
_
b
a
g(x)f
(x)dx
or, more compactly:
_
f dg = f g
_
g df
The Leibnitz rule: for dierentiation with respect to limits of
integral as well as the integrand.
Aria Nosratinia Probability and Statistics 1-12
Partial Derivatives
Used with functions of multiple variables, e.g., f(x, y).
Partial derivative: keep all variables xed, except one.
f
x
(x
0
,y
0
)
= lim
xx
0
f(x, y
0
) f(x
0
, y
0
)
x x
0
f
y
(x
0
,y
0
)
= lim
yy
0
f(x
0
, y) f(x
0
, y
0
)
y y
0
A vector of partial derivatives is known as the gradient
The total derivative with respect to t is dened as:
d
dt
f(x, y) =
f
x
dx
dt
+
f
y
dy
dt
Aria Nosratinia Probability and Statistics 1-13
Double (Multiple) Integration
Consider a 2-D area (x, y) A. The integral
__
A
f(x, y) dxdy
is the volume under the curve f(x, y) over area A.
Challenge: when area A does not match the coordinate system.
2
0
2
2
0
2
1
0.5
0
0.5
1
Example 1
2
0
2
2
0
2
0
0.2
0.4
0.6
0.8
1
Example 2
Aria Nosratinia Probability and Statistics 1-14
Example 1: A = {max{|x|, |y|} } and f(x, y) = sin xcos y
sin xcos y dxdy =
sin x dx
cos y dy = 0
Example 2: A = {x
2
+ y
2
1} and f(x, y) = 1.
A
1 dxdy =
1
1
1x
2
1x
2
dy
dx
=
1
1
2
1 x
2
dx
=
1
1
2(1 x
2
) d(sin
1
(x))
=
/2
/2
2(1 sin
2
t)dt
=
/2
/2
(1 + cos 2t)dt =
1-x
2
x
with change of variable t = sin
1
x, or equivalently, sin t = x.
Aria Nosratinia Probability and Statistics 1-15
Diracs Delta Function
Consider a concentration of mass at one point
Then the density at the mass point is innite (undened) but the
integral of the density (mass) is dened.
Diracs delta function:
(t) = 0 t = 0
_
(t) dt = 1
(t-a)
t a
(t)
t
This is also known as the impulse function, widely used in circuit
analysis.
We shall use impulse functions to characterize discrete probabilities
Aria Nosratinia Probability and Statistics 1-16
System of Linear Equations
The basic motivation of linear algebra comes from the solution of
linear equations
3x
1
+ 5x
2
= 1
4x
1
2x
2
= 3
Or in general:
a
11
x
1
+ a
12
x
2
= b
1
a
21
x
1
+ a
22
x
2
= b
2
We have learned to write this as:
_
_
a
11
a
12
a
21
a
22
_
_
_
_
x
1
x
2
_
_
=
_
_
b
1
b
2
_
_
Aria Nosratinia Probability and Statistics 1-17
Systems of Equations (2)
We can solve system of equations by a step-by-step reduction:
3x
1
+ 5x
2
= 1
10x
1
5x
2
= 7.5 (2.5)
13x
1
+ 0 = 8.5 (+Eq.1)
10x
1
5x
2
= 7.5
These are called elementary row operations, and the process is known
as Gaussian Elimination.
The same process can be done on a matrix. The matrices related to
each other via row operations are, in a sense, equivalent.
Aria Nosratinia Probability and Statistics 1-18
Matrices
System of Equations show the importance of matrices, which are
also important in many other applications.
Matrices are just tables of numbers.
Matrix addition is dened as element-by-element addition
Matrix multiplication
A
mn
= B
m
C
n
is dened as:
a
i,j
=
k=1
b
i,k
c
k,j
In other words, a
i,j
is the dot product of the i-th row of B with
the j-th column of C
Matrix product DOESNT commute.
Aria Nosratinia Probability and Statistics 1-19
Some Matrices and Matrix Operations
Transpose: exchange rows with columns
[A
t
]
i,j
= [A]
j,i
Symmetric Matrix: A
t
= A
Hermitian: conjugate transpose
Inverse (of a square matrix)
A
1
A = AA
1
= I
A matrix is unitary if A
1
= A
t
. This means rows (columns) are
ortho-normal.
Matrix A is positive denite if for all vectors x we have
x
t
Ax > 0
Aria Nosratinia Probability and Statistics 1-20
Some Useful Identities
1 + x + x
2
+ x
n1
=
x
n
1
x 1
1 + x + x
2
+ x
3
+ =
1
1 x
|x| < 1
1 x + x
2
x
3
+ =
1
1 + x
|x| < 1
Aria Nosratinia Probability and Statistics 1-21