Introduction
Introduction Lecture
Linear Regression
Linear Regression
The Least Squares Method
Linear Algebra Solution to Least Squares Problem
Example: Linear Regression
Summary: Linear Regression
Problem Set: Linear Regression Problem Set Linear Regression
Solution Set: Linear Regression Solution Set Linear Regression
Linear Discriminant Analysis
Classification
Linear Discriminant Analysis
The Posterior Probability Functions
Modelling the Posterior Probability Functions
Linear Discriminant Functions
Estimating the Linear Discriminant Functions
Classifying Data Points Using Linear Discriminant Functions
LDA Example 1
LDA Example 2
Summary: Linear Discriminant Analysis
Problem Set: Linear Discriminant Analysis Problem Set Linear Discriminant Analysis
Solution Set: Linear Discriminant Analysis Solution Set Linear Discriminant Analysis
Logistic Regression
Logistic Regression
Logistic Regression Model of the Posterior Probability Function
Estimating the Posterior Probability Function
The Multivariate Newton-Raphson Method
Maximizing the Log-Likelihood Function
Example: Logistic Regression
Summary: Logistic Regression
Problem Set: Logistic Regression Problem Set Logistic Regression
Solution Set: Logistic Regression Solution Set Logistic Regression
Artificial Neural Networks
Artificial Neural Networks
Neural Network Model of the Output Functions
Forward Propagation
Choosing Activation Functions
Estimating the Output Functions
Error Function for Regression
Error Function for Binary Classification
Error Function for Multi-class Classification
Minimizing the Error Function Using Gradient Descent
Backpropagation Equations
Summary of Backpropagation
Summary: Artificial Neural Networks
Problem Set: Artificial Neural Networks Problem Set Artificial Neural Networks
Solution Set: Artificial Neural Networks Solution Set Artificial Neural Networks
Maximal Margin Classifier
Maximal Margin Classifier
Definitions of Separating Hyperplane and Margin
Maximizing the Margin
Definition of Maximal Margin Classifier
Reformulating the Optimization Problem
Solving the Convex Optimization Problem
KKT Conditions
Primal and Dual Problems
Solving the Dual Problem
The Coefficients for the Maximal Margin Hyperplane
The Support Vectors
Classifying Test Points
Maximal Margin Classifier Example 1
Maximal Margin Classifier Example 2
Summary: Maximal Margin Classifier
Problem Set: Maximal Margin Classifier Problem Set Maximal Margin Classifier
Solution Set: Maximal Margin Classifier Solution Set Maximal Margin Classifier
Support Vector Classifier
Support Vector Classifier
Slack Variables: Points on Correct Side of Hyperplane
Slack Variables: Points on Wrong Side of Hyperplane
Formulating the Optimization Problem
Definition of Support Vector Classifier
A Convex Optimization Problem
Solving the Convex Optimization Problem (Soft Margin)
The Coefficients for the Soft Margin Hyperplane
The Support Vectors (Soft Margin)
Classifying Test Points (Soft Margin)
Support Vector Classifier Example 1
Support Vector Classifier Example 2
Summary: Support Vector Classifier
Problem Set: Support Vector Classifier Problem Set Support Vector Classifier
Solution Set: Support Vector Classifier Solution Set Support Vector Classifier
Support Vector Machine Classifier
Support Vector Machine Classifier
Enlarging the Feature Space
The Kernel Trick
Summary: Support Vector Machine Classifier
Conclusion
Concluding Letter