Summary

This week, we will focus on model selection and related theory. We will introduce cross-validation as an important and generic technique for model selection and hyperparameter tuning. We will also look at some theory related to generalization including the bias-variance tradeoff, model flexibility, and overfitting.

Learning Objectives

After completing this week, you are expected to be able to:

  • Understand how model flexibility relates to the bias-variance tradeoff and thus model performance.
  • Tune models by manipulating their flexibility through the use of a tuning parameter to find a model that generalizes well.
  • Avoid overfitting by selecting a model of appropriate flexibility through the use of a validation set or cross-validation.
  • Use GridSearchCV to tune models or pipelines with cross-validation.

Topics

  • Generalization
    • Model Flexibility
    • Overfitting
    • Bias-Variance Tradeoff
  • Cross-Validation
    • \(k\)-Fold Cross-Validation
    • GridSearchCV

Activities