Machine Learning Formula Sheet
I. Evaluation Metrics
Concept Formula
Precision TP / (TP + FP)
Recall (True Positive Rate) TP / (TP + FN)
Accuracy (TP + TN) / (P + N)
Sample Error errors(f) = (1/n) * Σ[δ(f(x), c(x))]
Absolute Error Σ |f(x_i) - ■_i|
Sum of Squares Error (SSE) Σ (f(x_i) - ■_i)^2
Number of Misclassifications Σ δ(f(x_i), y_i), δ=1 if mismatch else 0
II. Regression (Linear Models)
Concept Formula
Linear Regression Hypothesis hθ(x) = Σ β_i * x_i
Cost Function J(θ) = (1/(2m)) Σ (hθ(x^(i)) - y^(i))^2
Batch Gradient Descent Update θ_j := θ_j + α Σ (y^(i) - hθ(x^(i))) * x_j^(i)
Simple Linear Regression Slope β1 = [nΣxy - (Σx)(Σy)] / [nΣx^2 - (Σx)^2]
Intercept β0 = (Σy - β1Σx) / n
Population Line Y = β0 + β1x1 + ε
III. Decision Trees
Concept Formula
Entropy −p+ log2(p+) − p− log2(p−)
Information Gain Gain(S,A) = Entropy(S) − Σ (|Sv|/|S|) Entropy(Sv)
GINI Index 1 − Σ [p(c)]^2
IV. Instance-Based Learning & Similarity
Concept Formula
Euclidean Distance D(c1,c2) = √Σ(attr_i(c1) − attr_i(c2))^2
Minkowski Distance d(xi,xj) = (Σ |xis − xjs|^p)^(1/p)
Pearson Correlation r = Σ(Xi − X■)(Yi − ■) / √Σ(Xi − X■)^2 Σ(Yi − ■)^2
Signal to Noise Ratio S2N = (µX − µY) / (σX − σY)
Cosine Similarity cos(xi, xj) = (xi · xj) / (||xi|| ||xj||)
V. Collaborative Filtering
Concept Formula
Pearson Correlation (CF) sim(u,v) = Σ(r_ui − r■_u)(r_vi − r■_v) / √Σ(r_ui − r■_u)^2 Σ(r_vi − r■_v)^2
User-based Prediction p_ui = r■_u + [Σ sim(u,v)(r_vi − r■_v)] / Σ |sim(u,v)|
Item-based Prediction p_ui = [Σ sim(i,j) r_uj] / Σ |sim(i,j)|
VI. Probability & Bayesian Learning
Concept Formula
Bayes Rule P(h|D) = P(D|h)P(h) / P(D)
Conditional Probability P(X|Y) = P(X∩Y)/P(Y)
Law of Total Probability P(X) = Σ P(X|Y=yj) P(Y=yj)
Naive Bayes Rule argmax_k [ P(Y=yk) Π P(Xi|Y=yk) ]
VII. Logistic Regression
Concept Formula
Sigmoid Function g(z) = 1 / (1 + e^(−z))
Hypothesis hβ(x) = 1 / (1 + e^(−β^T X))
Conditional Probability P(y|x) = h(x)^y (1 − h(x))^(1−y)
Log Likelihood l(β) = Σ [ yi log h(xi) + (1−yi) log(1−h(xi)) ]
SGA Update βj := βj + α (y^(i) − hβ(x^(i))) xj^(i)
VIII. SVM & Kernels
Concept Formula
Linear SVM Objective Minimize (1/2)||w||^2 subject to yi(w·xi + b) ≥ 1
Soft Margin Objective Minimize ||w||^2 + C Σ ξk subject to yi(w·xi+b) ≥ 1−ξk
Kernel Function K(xa,xb) = φ(xa)·φ(xb)
Gaussian RBF Kernel K(xi,xj) = exp(−||xi−xj||^2 / (2σ^2))
Polynomial Kernel K(xi,xj) = (1 + xi·xj)^p
IX. Neural Networks
Concept Formula
Weighted Sum y = Σ wi xi
Perceptron Update ∆wi = η (y − ■) xi
Logistic Unit Update ∆wi = η Σ [ (yd − ■d) ■d (1−■d) xid ]
Backprop Delta δk = ok(1−ok)(yk−ok)
Backprop Weight Update ∆wij = η δj xi
X. AdaBoost
Concept Formula
Weak Classifier Error εt = Σ Dt(i) δ(ht(xi) ≠ yi)
Weak Learner Weight αt = 0.5 * ln((1−εt)/εt)
Weight Update Dt+1(i) = [Dt(i) exp(−αt yi ht(xi))] / Zt
Final Classifier H(x) = sign( Σ αt ht(x) )
XI. Computational Learning Theory
Concept Formula
Finite H (Consistent) m ≥ (1/ε)(ln|H| + ln(1/δ))
Finite H (Inconsistent) m ≥ (1/(2ε^2))(ln|H| + ln(2/δ))
Boolean Functions 2^(2^n)
XII. Clustering
Concept Formula
K-Means SSE SSE = Σ Σ ||xi − µi||^2
Davies-Bouldin Index DB = (1/n) Σ max_j≠i [ (σi+σj) / d(ci,cj) ]