5/14/25, 4:13 PM Notebook 4_ Machine Learning

Reference Guide for R (student resource) - Check out our reference guide for a full
listing of useful R commands for this project.

Data Science Project: Use data to determine the best

and worst colleges for conquering student debt.
Notebook 4: Machine Learning
Does college pay off? We'll use some of the latest data from the US Department of
Education's College Scorecard Database to answer that question.

In this notebook (the 4th of 4 total notebooks), you'll use R to add polynomial terms to
your multiple regression models (i.e. polynomial regression). Then, you'll use the
principles of machine learning to tune models for a prediction task on unseen data.

In [1]: ## Run this code but do not edit it. Hit Ctrl+Enter to run the code.
# This command downloads a useful package of R commands
library(coursekata)

── CourseKata packages ──────────────────────────────────── coursekata 0.1

✔ ✔ Metrics
8.0 ──

✔ ✔ lsr
dslabs 0.7.6 0.1.4

✔ ✔ mosaic
Lock5withR 1.2.2 0.5.2

✔ ✔ supernova
fivethirtyeightdata 0.1.0 [Link]
fivethirtyeight 0.6.2 2.5.7

The Dataset ( four_year_colleges.csv )

General description - In this notebook, we'll be using the
four_year_colleges.csv file, which only includes schools that offer four-year
bachelors degrees and/or higher graduate degrees. Community colleges and trade

[Link] 1/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

schools often have different goals (e.g. facilitating transfers, direct career education)
than institutions that offer four-year bachelors degrees. By comparing four-year
colleges only to other four-year colleges, we'll have clearer analyses and conclusions.

This data is a subset of the US Department of Education's College Scorecard

Database. The data is current as of the 2020-2021 school year.

Description of all variables: See here

Detailed data file description: See here

In [8]: ## Run this code but do not edit it. Hit Ctrl+Enter to run the code.
# This command downloads data from the file '[Link]' and stores it
dat <- [Link]('[Link]

1.0 - Motivating non-linear regression

So far, we've focused entirely on linear regression and multiple linear regression
models, which use linear functions to relate predictors (e.g.
net_tuition , grad_rate , pct_PELL ) to the outcome ( default_rate ).

In this notebook, we're going to investigate ways to model non-linear relationships. To

make this task a bit more manageable at the start, let's reduce the size of our dataset by
taking a random sample of 20 colleges from the dat dataframe. We will store our
sample in a new R dataframe called sample_dat .

In [9]: ## Run this code but do not edit it

# create a dataset to train the model with 20 randomly selected observati
[Link](2)
sample_dat <- sample(dat, size = 20)

Note: When getting a random sample, we'll get different results each time we run our
code because it's ... well ... random. This can be quite annoying. So, in the code above,
we used the command [Link](2) . This ensures that each time the code is
executed, we get the same results for our random sample - the results stored in seed
2 . We could have also set the seed to 1 or 3 or 845 or 12345 . The seed numbers
serve merely as a unique ID that corresponds to a certain result from a random draw. By
setting a certain seed, we'll always get a certain random draw.

1.1 Let's take a look at our sample data set. Print out the head and dim of
sample_dat .

In [12]: head(sample_dat)

[Link] 2/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

OPEID name city state region median_debt default_rate highest

<int> <chr> <chr> <chr> <chr> <dbl> <dbl>

Saint
975 379400 Martin's Lacey WA Far West 17.125 2.6 G
University

Northeastern
Rockies &
710 316100 State Tahlequah OK 12.490 3.7 G
Southwest
University

Muhlenberg
774 330400 Allentown PA Northeast 23.250 1.9 B
College

Spring Arbor Spring

416 231800 MI Midwest 24.645 3.6 G
University Arbor

Adrian
392 223400 Adrian MI Midwest 14.750 7.6 G
College

University of
273 189200 Iowa City IA Midwest 17.750 2.2 G
Iowa

In [11]: dim(sample_dat)

20 · 27

Check yourself: The dimensions of sample_dat should be 20 rows and 27

columns.

In prior notebooks, we focused on institutional and economic predictors of student loan

default rates. In this notebook, we'll begin by analyzing an academic variable: SAT_avg .
This variable shows the average SAT score of students who matriculate to a college.

The following code creates a scatterplot of the relationship between SAT_avg

(predictor) and default_rate (outcome) from the dataset sample_dat :

In [13]: ## Run this code but do not edit it

# create scatterplot: default_rate ~ SAT_avg
gf_point(default_rate ~ SAT_avg, data = sample_dat)

[Link] 3/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

1.2 - Describe the direction of the relationship between SAT_avg and

default_rate . Is it positive or negative? Why do you think this is?

Double-click this cell to type your answer here: A scatterplot analysis reveals a
negative relationship between colleges' average SAT scores and student loan default
rates, suggesting that students at institutions with higher SAT scores tend to have better
financial outcomes. This correlation may be driven by factors like stronger academic
preparedness, better institutional resources, and socioeconomic advantages. However,
potential exceptions and non-linearity in the relationship highlight the need for further
analysis and consideration of contextual factors like tuition costs and graduation rates.

1.3 - Create the same scatterplot as above, but with the simple linear model between
default_rate (outcome) and SAT_avg (predictor) overlayed on top.

Hint: Recall the gf_lm command from notebook 2.

In [14]: gf_point(default_rate ~ SAT_avg, data = sample_dat) %>%

gf_lm(color = "red")

[Link] 4/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

1.4 - Would you say that this model provides a "good" fit for this dataset? Explain.

Double-click this cell to type your answer here: The linear model provides a
moderate but imperfect fit, with SAT scores explaining approximately 45.7% of the
variance in default rates. The model's limitations, including curvature and outliers,
suggest that adding a polynomial term could improve fit and better capture the
relationship between SAT scores and default rates.

1.5 - Use the lm command to fit the linear regression model, where we use
SAT_avg (predictor) to predict default_rate (outcome) in the dataset
sample_dat . Store the model in a variable named sat_model_1 and use the
summary command to print out information about the model fit.

In [15]: sat_model_1 <- lm(default_rate ~ SAT_avg, data = sample_dat)

summary(sat_model_1)

[Link] 5/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Call:
lm(formula = default_rate ~ SAT_avg, data = sample_dat)

Residuals:
Min 1Q Median 3Q Max
-3.2133 -1.2490 -0.0765 0.6196 4.9881

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 19.480574 3.989422 4.883 0.00012 ***
SAT_avg -0.013315 0.003421 -3.893 0.00107 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.156 on 18 degrees of freedom

Multiple R-squared: 0.4571, Adjusted R-squared: 0.4269
F-statistic: 15.15 on 1 and 18 DF, p-value: 0.001067

Check yourself: The R

2
value shown in the model summary should be 0.4571

1.6 - Does the model's R2 value indicate that this model provides a strong fit for this
dataset? Explain.

Double-click this cell to type your answer here: The linear model's R² value of
0.4571 indicates a moderate fit for the dataset. While it explains about 45.7% of the
variance in default rates based on SAT scores, this wouldn't be considered a "strong" fit
by most standards in social science research where R² values above 0.7-0.8 are typically
considered strong.

1.7 - If this model were curved, rather than linear, do you believe the R
2
could be
higher? Explain.

Double-click this cell to type your answer here: Yes, a curved model could potentially
have a higher R² value. The scatterplot likely shows a non-linear relationship between
SAT scores and default rates, where the relationship changes direction or strength at
different SAT score ranges. A polynomial model could better capture this curvature,
potentially explaining more variance in the data.

2.0 - Polynomial regression

Recall that simple linear regression follows this formula:

y
^ = β0 + β1 x

Where:

[Link] 6/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

β0 is the intercept

β1 is the slope (coefficient of x)

^
y is the predicted default_rate

x is the value of SAT_avg

If we want to capture the curvature in a scatter plot by creating a non-linear model, we

can use a technique called polynomial regression. For example, we could use a
degree 2 polynomial (quadratic), which looks like this:

2
^ = β0 + β1 x + β2 x
y

Where:

β0 is the intercept

β1 is the coefficient of x (linear term)

β2 is the coefficient of x2 (squared term)

y
^ is the predicted default_rate

x is the SAT_avg

Below, we visualize the fit of this degree-2 polynomial (quadratic) model between
SAT_avg and default_rate :

In [16]: ## Run this code but do not edit it

# create scatterplot: default_rate ~ SAT_avg, with degree 2 polynomial mo
gf_point(default_rate ~ SAT_avg, data = sample_dat) %>% gf_lm(formula = y

[Link] 7/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

2.1 - Make a prediction: Will this polynomial regression model have a higher or lower
R
2
value than the linear regression model? Justify your reasoning.

Double-click this cell to type your answer here: The polynomial regression model will
likely have a higher R² value than the linear model because it can better fit the apparent
curvature in the data. The quadratic term allows the model to capture any U-shaped or
inverted U-shaped relationships that a straight line cannot.

Let's test your prediction. To do so, we'll first need to fit the polynomial model. We can fit
a degree 2 polynomial to the data using the poly() function inside of the lm()
function. Run the cell below to see how it's done.

In [17]: ## Run this code but do not edit it

# degree 2 polynomial model for default_rate ~ SAT_avg
sat_model_2 <- lm(default_rate ~ poly(SAT_avg, 2), data = sample_dat)
sat_model_2

Call:
lm(formula = default_rate ~ poly(SAT_avg, 2), data = sample_dat)

Coefficients:
(Intercept) poly(SAT_avg, 2)1 poly(SAT_avg, 2)2
4.065 -8.391 4.355

The equation for this model would be

2
y
^ = 4.065 − 8.391x + 4.355x

[Link] 8/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Where:

β0 = 4.065 is the intercept

β1 = −8.391 is the coefficient of x, the linear term

β2 = 4.355 is the coefficient of x2 , the squared term

^
y is the predicted default_rate

x is the SAT_avg

2.2 - Use the summary command on sat_model_2 to see summary information

about the quadratic model.

In [18]: summary(sat_model_2)
Call:
lm(formula = default_rate ~ poly(SAT_avg, 2), data = sample_dat)

Residuals:
Min 1Q Median 3Q Max
-3.6183 -0.9604 0.1192 0.9562 3.9014

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.0650 0.4361 9.321 4.3e-08 ***
poly(SAT_avg, 2)1 -8.3909 1.9504 -4.302 0.000483 ***
poly(SAT_avg, 2)2 4.3553 1.9504 2.233 0.039280 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.95 on 17 degrees of freedom

Multiple R-squared: 0.5802, Adjusted R-squared: 0.5308
F-statistic: 11.75 on 2 and 17 DF, p-value: 0.0006251

Check yourself: The R

2
value shown in the model summary should be 0.5802

2.3 - How does this model's R2 value compare to that of the linear model? Was your
prediction right? Explain.

Double-click this cell to type your answer here: Higher-degree polynomials can
improve fit but risk overfitting. The degree-2 model strikes a good balance—it captures
the key curvature in the data while remaining interpretable. Always validate models on
test data to ensure they generalize

This analysis raises a natural question: Why stop at degree 2? By raising the degree, we
can add more curves to our model, potentially better fitting the data! Let's visualize what

[Link] 9/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

happens when we increase the degree in our polynomial regression models.

Degree 3 Polynomial Model

2 3
y
^ = β0 + β1 x + β2 x + β3 x

In [19]: ## Run this code but do not edit it

# create scatterplot: default_rate ~ SAT_avg, with degree 3 polynomial mo
gf_point(default_rate ~ SAT_avg, data = sample_dat) %>% gf_lm(formula = y

Degree 5 Polynomial Model

2 3 4 5
y
^ = β0 + β1 x + β2 x + β3 x + β4 x + +β5 x

In [20]: ## Run this code but do not edit it

# create scatterplot: default_rate ~ SAT_avg, with degree 5 polynomial mo
gf_point(default_rate ~ SAT_avg, data = sample_dat) %>% gf_lm(formula = y

[Link] 10/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Degree 12 Polynomial Model

2 3 4 5 6 12
^ = β0 + β1 x + β2 x
y + β3 x + β4 x + +β5 x + +β6 x +. . . +β12 x

Note: The following code is pre-run, to save computer space.

In [22]: ## Note: This code was pre-run, to save computer space

# create scatterplot: default_rate ~ SAT_avg, with degree 12 polynomial m
# gf_point(default_rate ~ SAT_avg, data = sample_dat) %>% gf_smooth(metho

2.4 - Examine each plot for the polynmial models with degrees 3, 5, 12. Which model

[Link] 11/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

do you think would have the largest R2 value? Why?

Double-click this cell to type your answer here: The degree-12 model will have the
highest R^2 on training data because it can fit all nuances. However, this perfection likely
indicates overfitting—simpler models (degree-3 or 5) often generalize better to new data

To determine which polynomial model fits the data the best, we will fit models for each
degree (3, 5, 12).

In [23]: ## Run this code but do not edit it

# degree 3, 5, and 12 polynomial models for default_rate ~ SAT_avg
sat_model_3 <- lm(default_rate ~ poly(SAT_avg, 3), data = sample_dat)
sat_model_5 <- lm(default_rate ~ poly(SAT_avg, 5), data = sample_dat)
sat_model_12 <- lm(default_rate ~ poly(SAT_avg, 12), data = sample_dat)

Now we can compare each model's R2 value. Normally, we use the summary
command and read the R
2
value. However, since we've fit so many models, we don't
want to print out the entire summary for each one.

Instead, we'll use commands like this: summary(sat_model_1)$[Link] . The $

operator is used to extract just the [Link] element from the full summary . We
execute this command for each model, then print the results for ease of comparison.

In [24]: ## Run this code but do not edit it

# r-squared value for each model
r2_sat_model_1 <- summary(sat_model_1)$[Link]
r2_sat_model_2 <- summary(sat_model_2)$[Link]
r2_sat_model_3 <- summary(sat_model_3)$[Link]
r2_sat_model_5 <- summary(sat_model_5)$[Link]
r2_sat_model_12 <- summary(sat_model_12)$[Link]

# print each model's r-squared value

print(paste("The R squared value for the degree 1 model is", r2_sat_model
print(paste("The R squared value for the degree 2 model is", r2_sat_model
print(paste("The R squared value for the degree 3 model is", r2_sat_model
print(paste("The R squared value for the degree 5 model is", r2_sat_model
print(paste("The R squared value for the degree 12 model is", r2_sat_mode

[1] "The R squared value for the degree 1 model is 0.457055427196517"

[1] "The R squared value for the degree 2 model is 0.580193597490376"
[1] "The R squared value for the degree 3 model is 0.60314009577391"
[1] "The R squared value for the degree 5 model is 0.647445002110733"
[1] "The R squared value for the degree 12 model is 0.775449820710613"

Check yourself: The R

2
for the degree 5 model should be about 0.647

2.5 - The degree 12 model has the highest R2 value. Does that mean it's the "best"
model? Why or why not?

[Link] 12/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Hint: Think about which model would do the best for predicting the rest of the data
from the original full dataset.

Double-click this cell to type your answer here: No, the degree 12 model with the
highest R² is not necessarily the "best" model. While it fits the training data extremely well
(possibly perfectly), such high-degree polynomials often overfit the data - capturing
random noise rather than the true underlying relationship. This would likely perform
poorly on new, unseen data despite its high training R².

3.0 - Prediction, model tuning, & machine learning

In prior notebooks, we've used our models to make inferences about default rates.
However, sometimes in data science, we care more about predictions than we do about
inferences. In particular, many data science tasks ask for making accurate predictions on
new data - data that hadn't yet been collected when we first fit the model. This process of
building models to predict new data, especially when it's automated, is called machine
learning.

The key to machine learning is building models that make accurate predictions on test
data - unseen data that weren't used when fitting the model. Let's see how this works.
First, let's create a test dataset of 10 randomly sampled colleges. Importantly, these are
colleges that our models didn't see while fitting:

In [26]: ## Run this code but do not edit it

# create a data set to test the model with 10 new, randomnly selected obs
# not used to train the model
[Link](23)
test_dat <- sample(dat, size = 10)

3.1 - Use the head command on the test_dat data set.

In [27]: head(test_dat)

[Link] 13/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

OPEID name city state region median_debt default_rate highe

<int> <chr> <chr> <chr> <chr> <dbl> <dbl>

Texas
Rockies &
925 364600 Woman's Denton TX 14.000 5.1
Southwest
University

Benedictine
284 1025600 Atchison KS Midwest 19.000 4.0
College

Avila Kansas
456 244900 MO Midwest 19.209 5.1
University City

Catawba
615 291400 Salisbury NC South 14.486 6.5
College

Texas A &
M
College Rockies &
913 363200 University- TX 15.404 2.2
Station Southwest
College
Station

Wisconsin
1015 2136600 Lutheran Milwaukee WI Midwest 19.244 3.0
College

The following code visualizes the new test data alongside the training data (the data we
used to originally fit our models).

In [28]: ## Run this code but do not edit it

# label train and test sets
sample_dat$phase <- "train"
test_dat$phase <- "test"

# concatenate two datasets

full_dat <- rbind(sample_dat, test_dat)

# create scatterplot: default_rate ~ SAT_avg, with degree 5 polynomial mo

gf_point(default_rate ~ SAT_avg, data = full_dat, color = ~phase, shape =

[Link] 14/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

3.2 - Of all the polynomial models we fit before, which do you think would do best in
predicting the default rates in the test dataset?

Note: Use your gut and intution here. No calculations required.

Double-click this cell to type your answer here: The degree 2 or 3 polynomial model
would likely perform best on the test data. These models capture the essential curvature
without being too complex. The higher degree models (5, 12) are probably overfitting the
training data's noise.

Let's see how good one of our models is at predicting default rates. The R code in the
next cell uses the predict function to make predictions on the test dataset. In this
case, output shows the predicted default rates for the 10 test set colleges, as predicted
by our degree 5 model.

In [29]: ## Run this code but do not edit it

# get predictions for degree 5 model
pred_deg5 <- predict(sat_model_5, newdata = [Link](SAT_avg = test_dat
pred_deg5

1: 4.71195777211723 2: 2.80117510935355 3: 4.76610631136996 4: 5.83135112143679

5: 1.1525539865398 6: 3.41638983019025 7: 9.85190742866518 8: 18.1060265766056
9: 1.29744229099948 10: 4.28267983941371

[Link] 15/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

So, how can we interpret these values? Well, the last college in our test set is
University of New Orleans , which has an SAT_avg value of 1088 and a
default_rate of 6.6 . It's shown here on the graph, alongside our degree 5 model.

Note: The following code is pre-run.

In [30]: ### Run this code but do not edit it

## create scatterplot: default_rate ~ SAT_avg, with degree 5 polynomial m
#gf_point(default_rate ~ SAT_avg, data = sample_dat, color = ~phase, shap

Our degree 5 model's predicted default rate for this first data point was 4.28 . That
means that our degree 5 model under-estimates the actual value for default rate by...

6.6 − 4.28 = 2.32

The model's prediction and error is visualized in the plot below:

So, its predicted default rate is "off" by about 2 percentage points! This is pretty amazing,
considering the model had only 20 training data values and the University of New

[Link] 16/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Orleans wasn't included among them. This is the power of machine learning! Predicting
previously unseen data!

This is just one prediction. We're really interested in how this model performed across all
its predictions. For that, let's measure its R2 (prediciton strength) on the test set!

We can use the cor function to correlate the predictions with the actual default rates (r
) and then square that value to get R2 , which gets us the prediction strength!

In [31]: ## Run this code but do not edit it

# Get correlation between predicted and actual default rates in test set
cor(test_dat$default_rate, pred_deg5) ^ 2

0.616546630372038

We can now repeat this same process for all polynomial degrees.

In [32]: ## Run this code but do not edit it

# Storing test set predictions for all models
pred_deg1 <- predict(sat_model_1, newdata = [Link](SAT_avg = test_dat
pred_deg2 <- predict(sat_model_2, newdata = [Link](SAT_avg = test_dat
pred_deg3 <- predict(sat_model_3, newdata = [Link](SAT_avg = test_dat
pred_deg5 <- predict(sat_model_5, newdata = [Link](SAT_avg = test_dat
pred_deg12 <- predict(sat_model_12, newdata = [Link](SAT_avg = test_d

# print each model's r-squared value

print(paste("The test R squared value for the degree 1 model is", cor(tes
print(paste("The test R squared value for the degree 2 model is", cor(tes
print(paste("The test R squared value for the degree 3 model is", cor(tes
print(paste("The test R squared value for the degree 5 model is", cor(tes
print(paste("The test R squared value for the degree 12 model is", cor(te

[1] "The test R squared value for the degree 1 model is 0.55824446697698"
[1] "The test R squared value for the degree 2 model is 0.70025122602337"
[1] "The test R squared value for the degree 3 model is 0.733729012084851"
[1] "The test R squared value for the degree 5 model is 0.616546630372038"
[1] "The test R squared value for the degree 12 model is 0.17612156142752
4"

Check yourself: The R

2
for the degree 5 model should be about 0.6165

3.3 - Compare the estimates for each model. Which models did well? Which
2
R

models did poorly? Why do you think this is?

[Link] 17/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

Double-click this cell to type your answer here: The middle-degree models (likely
degrees 2-3) performed best on test data, while the simplest (degree 1) and most
complex (degree 12) models performed worse. This demonstrates the bias-variance
tradeoff: linear models underfit (high bias), while high-degree polynomials overfit (high
variance). The moderate complexity models strike the right balance.

3.4 - In machine learning, the central goal is to build our models so as to avoid
"underfitting" and "overfitting" our models to the training data. What do you think
these terms mean? Which of our models were underfit? Which do you think were
overfit? Explain.

Double-click this cell to type your answer here: Underfitting occurs when a model is
too simple to capture the data's true structure (like our linear model). Overfitting occurs
when a model is too complex, fitting training data noise rather than general patterns (like
our degree 12 model). The degree 1 model underfits, while degrees 5 and 12 overfit.
Degree 2 or 3 appears optimally fit.

Recall that we built our polynomial models here with just one predictor: x ( SAT_avg ).
Yet, those models could end up being quite complex...

2 3
y
^ = β0 + β1 x + β2 x + β3 x

Now, imagine that we wanted to bring in multiple predictors (x1 = SAT_avg , x2 =

net_tuition , x3 = grad_rate ) for muliple regression. Plus, imagine that we
decided to add in some polynomial terms for each of these predictors. We could end up
with a model that looks ever more complicated, with literally hundreds of terms...

2 3 2 3 2
^ = β 0 + β 1 x1 + β 2 x
y + β3 x + β 4 x2 + β 5 x + β5 x + β6 x3 + β7 x +. . .
1 1 2 2 3

3.5 - Is it always good to add more predictors and add more polynomial terms to your
model? Explain why or why not.

Double-click this cell to type your answer here: No, adding more predictors and
polynomial terms isn't always beneficial. While it may improve training set performance, it
can lead to overfitting, reduced interpretability, poor generalization to new data, potential
multicollinearity issues The best models balance complexity with generalizability, using
only meaningful predictors and necessary polynomial terms.

4.0 - In-class prediction competition

Now you have all the tools you need to build very powerful prediction models! This
means that it's time for a friendly competition :)

[Link] 18/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

The code below takes the full dataset and splits it into larger train and test datasets. 80%
of the colleges will go into the train dataset. 20% will go into the test dataset. Because we
all are setting the same seed ( 2025 ), everyone will get the exact same train and test
sets:

In [37]: ## Run but do not edit this code

# set training data to be 80% of all colleges

train_size <- floor(0.8 * nrow(dat))

## sample row indeces

[Link](2025)
train_ind <- sample(seq_len(nrow(dat)), size = train_size)

train <- dat[train_ind, ]

test <- dat[-train_ind, ]

In [ ]: dim(train)

In [ ]: dim(test)

Now it's time to compete!

Goal: Create the most accurate prediction model of colleges' default rates.

Evaluation: Whichever student has the highest R2 on the test set wins.

Guidelines Save your best model as an object called my_model . You are only allowed
to fit models on the train set (not on the test set). You may use as many predictors and
as many polynomial terms as you'd like. Just be warned: Don't fall into the trap of
overfitting! Choose only the most important variables and keep your models simple, so
that you can generalize well to the test set. Periodically test your model on the test set
and then make adjustments as necessary.

Go!

In [38]:

Error in library(caret): there is no package called ‘caret’

Traceback:

1. library(caret)

In [39]:

Error in eval(expr, envir, enclos): object 'my_model' not found

Traceback:

1. predict(my_model, newdata = test)

[Link] 19/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

5.0 - NATIONWIDE prediction competition

Competition: We're hosting a nationwide competition to see which student can build the
best model for predicting student loan default rates at different colleges. Here's an article
about last year's winners.

Evaluation: Across the country, all students are using the same train and test sets as
you did in the prior exercise to fit and evaluate their models. Your goal: Build a model
that gives the best predictions on this test set. The student models that produce the
highest R2 value on the test set will be announced as champions! Note: this year's train
and test sets are different from prior years, so replicating a prior winning model won't be
a winning strategy.

Submission Process (due by June 6, 2025 at 11:59pm CT):

1. Print and have a parent/guardian sign the media release form. This form gives
permission to feature you and publish your results, in the event that you're a finalist!
Take a picture or scan the signed form and submit it during as a part of Step #2
(below).
2. Submit this google form (note: you'll have to log into a google account), which allows
you to upload your media release form, model, and notebook. This counts as your
final submission.

Rules and notes to avoid disqualification:

Do not change the seed ( 2025 ) in the code block that splits the data into the train
and test sets. Using the common seed of 2025 will ensure everyone across the
country has the exact same train/test split.
Make sure your model is fit using the train data. In other words, it should look
like: my_model <- lm(default_rate ~ ..., data = train) .
Your model must be fit using the lm command. Models outside the scope of the
concepts covered in this project (e.g random forest, neural networks, etc) are not
allowed.
Your model must predict default_rate directly. Applying transformations (e.g.,
log, square root, scaling) to the outcome variable before modeling is not allowed. All
submissions must use default_rate as provided in the dataset.
Transformations of predictors are allowed. You may apply transformations (e.g., log,
standardization, polynomial terms, interaction terms) to the predictor variables to
improve your model.
Make sure you find the R
2
value on the test$default_rate data, using the
provided code.
Only individual submissions are allowed. Group submissions will not be accepted,
and each participant must submit their own work. Note: Students can still share

[Link] 20/21
5/14/25, 4:13 PM Notebook 4_ Machine Learning

ideas and strategies with one another, but all final submissions must be an individual
student's own work.
There are ways to "cheat" on this competition by looking directly at the test set data
values and designing your model to predict those values exactly (or approximately).
However, based on the design of your model (which we'll see when you share your
notebook), it's pretty easy for us to tell if you've done this. So, don't do it! Your
submission will be discarded.
Use online resources responsibly. We acknowledge that data science solutions,
including similar problems and code, are publicly available online. You are
encouraged to learn from these resources, but your submission must be your own
work. Directly copying code or solutions without understanding and adapting them
will result in disqualification.

Summer Opportunity: Do you want to learn more about Data

Science & AI?
Join our Data Science & AI Summer Bootcamp, where you'll take your learning from
this project to the next level. No prior coding or statistics experience required!
Designed by Harvard grads, the bootcamp allows students from all experience levels
to dive deeper into data science concepts, from the basics (e.g. linear regression) to
the advanced (e.g. AI neural networks). Students learn in a supportive and
collaborative environment, and they walk away with their own real-world project that
can be shared on college and internship applications.

📢 Scholarships are available! We’re committed to making this opportunity accessible

to all students.

📝 Applications are considered on a rolling basis. Final application deadline: May 30,
2025

🔗 Learn more and apply here: [Link]

[Link] 21/21