Acceptance/Rejection Sampling & MCMC Metropolis-Hastings Sampling with full Computational Simulation — 1: Introduction & Motivation In parametric Bayesian Inference, our objective is to recover the posterior distribution of the parameter (or parameters) of interest. …

Derivations of the Classic Trinity of Inferential Tests with Full Computational Simulation — 1: Introduction & Motivation Let’s say we’re working on an inferential statistics analysis. Assume we’re working from a frequentist perspective. …

Specification and Derivations of Connections between Total Variation, KL Divergence, and MLE — 1: Introduction & Motivation In this piece, I look to cover the mathematical underpinnings of Maximum Likelihood Estimation (MLE); a commonly used procedure for constructing sampling estimators for parameters of interest of a distribution. Though very commonly used, MLE is a procedure not always well understood or well motivated mathematically from a teaching perspective.

Mathematical Derivations of Kernelized Features Spaces for Linear Smoothers, with a full Computational Simulation — 1: Introduction For linear smoothers and linear-predictor based sampling estimators, Mercer Kernels are a highly convenient tool for fitting linear decision boundaries in high dimensional feature spaces. In fact, such feature spaces can even be infinitely dimensional (as we will show). From the perspective of Machine Learning, Mercer Kernels can be viewed…

Mathematical Derivations of Boosting Procedures with full Computational Simulation — 1: Introduction Boosting is a family of ensemble Machine Learning techniques for both discrete and continuous random variable targets. Boosting models take the form of Non-Parametric Additive models and are most typically specified with additive components being “weak learners”. From an empirical risk decomposition perspective, where it can be easily shown the…

Mathematical Derivations and a Computational Simulation — 1: Introduction Generalized Linear Models (GLMs) play a critical role in fields including Statistics, Data Science, Machine Learning, and other computational sciences. Part I of this Series provided a thorough mathematical overview with proofs of common GLMs, both in Canonical and Non-Canonical forms. Part II provided historical and mathematical context of common…

Mathematical Derivations and Implementation of Iterative Fitting Techniques with Computational Simulations. — 1: Background and Motivation Generalized Linear Models (GLMs) play a critical role in fields including Statistics, Data Science, Machine Learning, and other computational sciences. In Part I of this Series, we provided a thorough mathematical overview (with proofs) of common GLMs both in Canonical and Non-Canonical forms. …

Intuition for Unifying Theory of GLMs with Derivations in Canonical and Non-Canonical Forms — 1: Background and Motivation Generalized Linear Models (GLMs) play a critical role in fields including Statistics, Data Science, Machine Learning, and other computational sciences. This class of models are a generalization of ordinary linear regression for certain response variable types with error distribution models other than a normal distribution.

Efficiency & Statistical Power gains from Conditional Covariate Adjustment in A/B Testing & Randomized Trials — 1: Background and Motivation Causal Inference is a field that touches several domains and is of interest to a wide range of practitioners including Statisticians, Data Scientists, Machine Learning Scientists, and other Computational Researchers. To date I have written several pieces on methods/topics in the Causal Inference…