Math 466: Mathematics of Machine Learning

Reuben-Cooke Building 127
Tuesday-Thursday: 8:30am-9:45am

Office Hours: Tuesday 10-11am, Thursdays 1-2pm. Gross Hall 3rd floor.

Recomended Prerequisites: Mathematics 230/340 and 218/216/221.


Intro to ML (1 Lecture)
Topics: Regression & Classification, Supervised & Unsupervised Learning, Training, Bias-Variance Tradeoff, Model Complexity.

Statistical Learning Theory (2 Lectures)
Topics: Empirical Risk Minimization, Hypotheses Classes, PAC Learnability, No Free Lunch Theorem, VC Dimension.

Intro to NN (3 Lectures)
Topics: Components of Neural Networks, Backprop, Cross-Entropy, Regularization, Dropout, Weight Initialization, Vanishing/Exploding Gradients, Universal Approximation Theorem.

Optimization Theory (4 Lectures)
Topics: Convex Geometry, Convex Functions, 1st and 2nd Order Conditions for Convexity, Convex Optimization, Logistic Regression, SVM, Linear Search, Gradient Descent, Newton’s Method, Subgradients, SGD, Momentum.

Convolutional Neural Networks (2 Lectures)
Topics: Filters, Pooling, Convolution, Stride, Boundary, Translation Equivariance, Reduction of Complexity, CNN + GANs, CNN on Graphs.

Representation Learning (2 Lectures)
Topics: Window Co-Occurence, CBOW, Skip-Gram, RNNs, Information Theory, Hierarchical Softmax, Sentiment Analysis, Various Autoencoder Architectures.

Dimensionality Reduction (3 lectures)
Topics: Curse and Blessing of Dimensionality, PCA, Kernel PCA, MDS, Random Projections, Johnson-Lindenstrauss, Isomap, Laplacian Eigenmaps.

Bayesian Models & Gaussian Processes (4 Lectures)
Topics: The multivariate Gaussian distribution, formulas for conditional and marginal Gaussians, the Gaussian distribution and Bayes’ Rule, the Bayesian Approach to Linear Regression, the predictive distribution, the equivalent kernel, … (more to be added)

Reinforcement Learning (4 Lectures)
Topics: k-Bandit Problem, Greed vs. Exploration, Environments, States, Actions, Rewads, Agents, Policies, Value Functions, Bellman Equations, Policy Evaluation, Policy Iteration, Optimality Equations, Convergence Theorems, Monte-Carlo Methods, Temporal-Difference Methods.