Probability & Statistics
1. Probability Theory
• 1.1 Foundational Concepts
o Sample space, events
o Axioms of probability
o Conditional probability
o Law of total probability
o Bayes’ theorem
• 1.2 Random Variables
o Discrete & continuous RVs
o CDF, PDF, PMF
o Joint, marginal, conditional distributions
o Expectation, variance, moments
• 1.3 Important Distributions
o Bernoulli, Binomial, Poisson
o Uniform, Exponential, Normal
o Gamma, Beta, Chi-square
o Multivariate distributions
• 1.4 Transformations of Random Variables
o Change of variables formula
o Jacobian transform
• 1.5 Inequalities
o Markov, Chebyshev
o Jensen’s inequality
o Chernoff bounds
• 1.6 Limit Theorems
o Law of Large Numbers (weak/strong)
o Central Limit Theorem
o Slutsky’s theorem
o Delta method
• 1.7 Measure-Theoretic Probability
o Probability spaces
o Sigma-algebras
o Probability measures
o Almost sure convergence
• 1.8 Conditional Expectation
o Tower property
o Martingale definition
o Doob martingales
• 1.9 Characteristic Functions
o Inversion formulas
o Uniqueness theorem
2. Random Processes (Stochastic Processes)
• 2.1 Discrete-Time Stochastic Processes
o Sequences of random variables
o Markov chains
▪ Transition matrices
▪ Stationary distributions
▪ Classification of states
o Branching processes
• 2.2 Continuous-Time Processes
o Markov processes
o Poisson processes
▪ Interarrival times
▪ Non-homogeneous Poisson processes
• 2.3 Gaussian Processes
o Covariance functions
o Brownian motion (Wiener process)
o Ornstein–Uhlenbeck process
• 2.4 Martingales
o Martingale convergence
o Optional stopping theorem
• 2.5 Stochastic Calculus
o Ito integral
o Ito’s lemma
o Stochastic differential equations (SDEs)
o Stratonovich calculus
• 2.6 Stationary & Ergodic Processes
o Autocorrelation
o Power spectral density
o Ergodicity conditions
• 2.7 Queueing Theory
o Birth–death processes
o M/M/1, M/M/c queues
• 2.8 Renewal Theory
o Renewal processes
o Reward processes
3. Bayesian Statistics
• 3.1 Foundations
o Prior, likelihood, posterior
o Conjugate priors
o Bayesian updating
• 3.2 Bayesian Inference
o Posterior distributions
o Posterior predictive distributions
o Credible intervals
• 3.3 Computational Bayesian Methods
o Monte Carlo integration
o Markov Chain Monte Carlo (MCMC)
▪ Gibbs sampling
▪ Metropolis–Hastings
o Hamiltonian Monte Carlo
• 3.4 Bayesian Decision Theory
o Loss functions
o Bayesian risk
o Optimal Bayesian decisions
• 3.5 Hierarchical Bayesian Models
o Latent variables
o Multilevel models
• 3.6 Bayesian Nonparametrics
o Dirichlet process
o Gaussian process priors
• 3.7 Bayesian Model Selection
o Bayes factors
o Posterior odds
o Marginal likelihood
• 3.8 Bayesian Deep Learning
o Variational inference
o Bayesian neural networks
o Monte Carlo dropout
4. Statistical Inference
• 4.1 Parametric Estimation
o Point estimation
o Maximum likelihood (MLE)
o Method of moments
o Cramer–Rao lower bound
• 4.2 Hypothesis Testing
o Null & alternative hypotheses
o Type I/II errors
o Likelihood ratio tests
o Neyman–Pearson lemma
• 4.3 Confidence Intervals
o Exact & asymptotic intervals
o Bootstrap intervals
• 4.4 Nonparametric Methods
o Kernel density estimation
o Rank tests
o Empirical distribution function
• 4.5 Multivariate Statistical Analysis
o Covariance matrices
o Principal component analysis (PCA)
o Factor analysis
o Clustering
• 4.6 Regression Analysis
o Linear regression
o Logistic regression
o Ridge, LASSO
o Generalized linear models (GLMs)
• 4.7 Time Series Analysis
o Autoregressive models (AR)
o Moving average models (MA)
o ARIMA models
o Stationarity & differencing
• 4.8 Resampling Methods
o Bootstrap
o Jackknife
5. Information Theory
• 5.1 Entropy
• 5.2 Mutual information
• 5.3 KL divergence
• 5.4 Channel capacity
6. Statistical Learning Theory
• 6.1 Empirical risk minimization
• 6.2 VC dimension
• 6.3 PAC learning
• 6.4 Regularization theory
7. Asymptotic Statistics
• 7.1 Consistency
• 7.2 Asymptotic normality
• 7.3 Delta method
8. Experimental Design
• 8.1 ANOVA
• 8.2 Randomized designs
• 8.3 Factorial experiments
9. Causal Inference
• 9.1 Counterfactual models
• 9.2 DAGs (directed acyclic graphs)
• 9.3 Propensity scores
10. Decision Theory
• 10.1 Minimax rules
• 10.2 Risk functions
• 10.3 Admissibility
11. Computational Statistics
• 11.1 Optimization for statistics
• 11.2 EM algorithm
• 11.3 Variational inference
12. Survival Analysis
• 12.1 Hazard functions
• 12.2 Kaplan–Meier estimator
• 12.3 Cox proportional hazards model