Loss Functions
Review: Loss functions evaluate how well the model performs (how much the predicted result deviates from actual result)
Regression Loss Functions
Predicts continuous value (eg. floor size, # of rooms)
Mean Square Error / Quadratic Loss / L2 Loss
Measures magnitude without considering direction
Squaring penalizes further deviated values much more than less deviated values
Mean Absolute Error / L1 Loss
More robust to outliers (no squaring)
Needs linear programming to calculate gradients
Mean Bias Error
Same as MAE but no absolute
This is kinda bad as it makes things less accurate
Used for seeing positive or negative bias
Classification Loss Functions
Predict output from finite set of categorical values (e.g. 0-9)
Hinge Loss / Multiclass SVM Loss
Score of correct category should be greater than sum of scores of all incorrect categories by some safety margin (usually one)
Usually for SVM
Cross Entropy Loss / Negative Log Likelihood
Most common
Loss increases as predicted probability diverges from actual label
Penalizes heavily predictions that are confident but WRONG
Cross Entropy Loss vs. KL Divergence
https://www.youtube.com/watch?v=pH9xkCK4ATc (great video!)
Last updated
