Summary of Week 2 -- Capsule 4 -- Model Evaluation

00:00:00 - 00:15:00

This video covers model evaluation, specifically generalization. The author explains how to calculate loss on new data and demonstrates that the best model has the smallest loss. He also uses this information to illustrate how model performance can reverse when new data is used instead of old data.

• 00:00:00 In this video, the author discusses model evaluation, specifically discussing the concept of generalization. The author shows how to calculate the loss of a model on new data, and demonstrates that the best model has the smallest loss. The author then uses this information to illustrate how model performance can be reversed when new data is used instead of old data.
• 00:05:00 In this week's video, the loss of model 2 is smaller than the loss of model 1 and number 3, which is good evidence that minimizing the blue loss may not be the best way to go. Capacity, or the ability of a model to fit a variety of functions, is introduced and explained. As the capacity of the model increases, the generalization gap (the difference between the test error and train error) starts increasing. The optimal capacity is found at a point where the gap between the test error and train error is the largest. Drawing a green curve on the x-axis shows that the model with the optimal capacity is the model with the lowest generalization on the right hand side. All models on the right hand side are in the overfitting zone. The model with the optimal capacity is the model with the lowest generalization on the right hand side.
• 00:10:00 This video discusses how the generalization error is affected by the capacity of the model, as well as the number of training examples available. The video also compares the test performance of two different models, one with too little capacity and one with optimal capacity.
• 00:15:00 The author discusses the benefits and drawbacks of larger capacity models. All else being equal, larger capacity models will require more data to be fit right, but they will also be better.