Wing

Classical Machine Learning

18th century

Gradient Descent

Augustin-Louis Cauchy, 1847

First-order optimization: descend a 2-D quadratic loss surface along negative gradient steps.

1900s

Principal Component Analysis

Karl Pearson, 1901

Rotate to the axis of greatest variance: orthogonal projection of a 2-D scatter onto its dominant principal direction.

1940s

Logistic Regression

Joseph Berkson, 1944

Sigmoid decision boundary separating two classes with small-batch gradient descent on log loss in the plane.

Simplex Method (2D)

George Dantzig, 1947

Linear program in two variables: move along vertices of the feasible polytope toward the optimum.

1960s

Kalman Filter

Rudolf E. Kálmán, 1960

Optimal linear-Gaussian state estimation by alternating prediction and measurement correction.

Karatsuba Multiplication

Anatoly Karatsuba, 1960

Divide two n-digit numbers using three multiplications of size n/2 plus additions.

FFT (Cooley-Tukey)

James Cooley & John Tukey, 1965

Fast Fourier transform factorizes DFT size recursively into O(n log n) butterflies.

k-Means Clustering

James MacQueen et al., 1967

Alternates assigning each point to its nearest centroid and recomputing centroids until the clustering stabilizes.

Strassen Matrix Multiply

Volker Strassen, 1969

Recursive matrix multiplication below O(n³) using seven half-size products per level.

1980s

Interior-Point Method (LP)

Narendra Karmarkar et al., 1984

Follows a barrier-central trajectory through the interior of the feasible polytope toward the optimum.

1990s

Naive Bayes Text Classifier

Various, 1997

Classifies a tiny bag-of-words document by assuming word counts are independent given spam vs. ham.

2000s

t-SNE (2D embedding)

Laurens van der Maaten & Geoffrey Hinton, 2008

Nonlinear dimensionality reduction that preserves local neighborhoods in a low-dimensional map.