Math 466: Mathematics of Machine Learning

Reuben-Cooke Building 127
Tuesday-Thursday: 8:30am-9:45am

Office Hours: Tuesday 10-11am, Thursdays 1-2pm. Gross Hall 3rd floor.

Recomended Prerequisites: Mathematics 230/340 and 218/216/221.

YOUTUBE PLAYLIST

Intro to ML (1 Lecture)
Topics: Regression & Classification, Supervised & Unsupervised Learning, Training, Bias-Variance Tradeoff, Model Complexity.

Statistical Learning Theory (2 Lectures)
Topics: Empirical Risk Minimization, Hypotheses Classes, PAC Learnability, No Free Lunch Theorem, VC Dimension.
COURSE NOTES
HOMEWORK WITH SOLUTIONS

Intro to NN (3 Lectures)
Topics: Components of Neural Networks, Backprop, Cross-Entropy, Regularization, Dropout, Weight Initialization, Vanishing/Exploding Gradients, Universal Approximation Theorem.
COURSE NOTES
HOMEWORK WITH SOLUTIONS

Optimization Theory (4 Lectures)
Topics: Convex Geometry, Convex Functions, 1st and 2nd Order Conditions for Convexity, Convex Optimization, Logistic Regression, SVM, Linear Search, Gradient Descent, Newton’s Method, Subgradients, SGD, Momentum.
COURSE NOTES
HOMEWORK WITH SOLUTIONS

Convolutional Neural Networks (2 Lectures)
Topics: Filters, Pooling, Convolution, Stride, Boundary, Translation Equivariance, Reduction of Complexity, CNN + GANs, CNN on Graphs.
COURSE NOTES

Representation Learning (2 Lectures)
Topics: Window Co-Occurence, CBOW, Skip-Gram, RNNs, Information Theory, Hierarchical Softmax, Sentiment Analysis, Various Autoencoder Architectures.
COURSE NOTES
HOMEWORK WITH SOLUTIONS

MIDTERM: THURSDAY, FEBRUARY 24TH

Dimensionality Reduction (4 lectures)
Topics: Curse and Blessing of Dimensionality, PCA, Kernel PCA, MDS, Random Projections, Johnson-Lindenstrauss, Isomap, Laplacian Eigenmaps.
COURSE NOTES
HOMEWORK WITH SOLUTIONS

Reinforcement Learning (4 Lectures)
Topics: k-Bandit Problem, Greed vs. Exploration, Environments, States, Actions, Rewads, Agents, Policies, Value Functions, Bellman Equations, Policy Evaluation, Policy Iteration, Optimality Equations, Convergence Theorems, Monte-Carlo Methods, Temporal-Difference Methods.
COURSE NOTES

Bayesian Models & Gaussian Processes (3 Lectures)
Topics: The multivariate Gaussian distribution, formulas for conditional and marginal Gaussians, the Gaussian distribution and Bayes’ Rule, the Bayesian Approach to Linear Regression, the predictive distribution, the equivalent kernel, Gaussian Processes, Regression with Gaussian Processes.
COURSE NOTES

FINAL PRESENTATIONS: APRIL 19,21