import pandas as pd
Lab 02: Credit Ratings
Scenario: Suppose you work for a small local bank, perhaps a credit union, that has a credit card product offering. For years, you relied on credit agencies to provide a rating of your customers’ credit, however, this costs your bank money. One day, you realize that it might be possible to reverse engineer your customers’ (and thus potential customers) credit rating based on the credit ratings that you have already purchased, as well as the income and demographic information that you already have, such as age, education level, etc.
Goal
The goal of this lab is to create a regression model that predicts the credit rating for an individual based on income and demographic information.
Data
This lab will use data from the textbook An Introduction to Statistical Learning.1
Response
Rating
Features
Income
Age
Education
Gender
Student
Married
Ethnicity
Data in Python
To load the data in Python, use:
= pd.read_csv("https://cs307.org/lab-02/data/credit-train.csv") credit_train
To create the X
and y
variants of the training data, use:
# create X and y for train
= credit_train.drop("Rating", axis=1)
X_train = credit_train["Rating"] y_train
Sample Statistics
Before modeling, be sure to look at the data. Calculate the summary statistics requested on PrairieLearn and create a visualization for your report.
Models
For this lab you will select one model to submit to the autograder. You may use any modeling techniques you’d like. The only rules are:
- Models must start from the given training data, unmodified.
- Importantly, the types and shapes of
X_train
andy_train
should not be changed. - In the autograder, we will call
mod.predict(X_test)
on your model, where your model is loaded asmod
andX_test
has a compatible shape with and the same variable names and types asX_train
. - We assume that you will use a
Pipeline
andGridSearchCV
fromsklearn
as you will need to deal with heterogeneous data, and you should be using cross-validation.- So more specifically, you should create a
Pipeline
that is fit withGridSearchCV
. Done correctly, this will store a “model” that you can submit to the autograder.
- So more specifically, you should create a
- Importantly, the types and shapes of
- Your model must have a
fit
method. - Your model must have a
predict
method. - Your model should be created with
scikit-learn
version1.4.0
or newer. - Your model should be serialized with
joblib
version1.3.2
or newer.- Your serialized model must be less than 5MB.
To obtain the maximum points via the autograder, your model performance must meet or exceed:
Test RMSE: 112.5
Model Persistence
To save your model for submission to the autograder, use the dump
function from the joblib
library. Check PrairieLearn for the filename that the autograder expects.
Discussion
As always, be sure to state a conclusion, that is, whether or not you would use the model you trained and selected for the real world scenario described at the start of the lab! If you are asked to train multiple models, first make clear which model you selected and are considering for use in practice. Discuss any limitations or potential improvements.
Additional discussion topics:
- Ignoring model performance, does it seem appropriate to use these features for the stated goal of these models?
- Is it legal to do so? Is it ethical to do so?
- A very important lesson to learn now that you will have the power of machine learning: Just because you can, does not mean you should.
When answering discussion prompts: Do not simply answer the prompt! Answer the prompt, but write as if the prompt did not exist. Write your report as if the person reading it did not have access to this document!
Template Notebook
Submission
On Canvas, be sure to submit both your source .ipynb
file and a rendered .html
version of the report.
Footnotes
The data can be found in the
ISLP
package, but we do not recommend installing the package because it requires old versions of software that we need to be up-to-date.↩︎