Question 0

Require tidyverse and load the game_summary.rds data to an object called games. (Note, you must load the data from your local computer, not from the link)

# INSERT CODE HERE

Question 1 [0.75 points]

How many points, on average, did the New York Knicks score at home and away games in the 2017 season? Calculate this answer and also plot the multivariate relationship. Explain why your chosen visualization is justified. Draw two vertical lines for the average points at home and away.

# Create extra object to plot vertical lines for average points at home and away
vertLines <- games %>%
filter() %>% # Filter to the 2017 season (yearSeason) AND to the New York Knicks (nameTeam)
  group_by() %>% # Group by the location of the game
  summarise() # Calculate the average points (pts)

## Error in games %>% filter() %>% group_by() %>% summarise(): could not find function "%>%"

games %>%
  filter() %>% # Filter to the 2017 season (yearSeason) AND to the New York Knicks (nameTeam)
  ggplot() + # Create a multivariate plot comparing points scored between home and away games
  geom_...() + # Choose the appropriate geom_... for this plot (i.e., geom_histogram(), geom_density(), geom_bar(), etc.)
  labs(title = '', # Add clear descriptions for the title, subtitle, axes, and legend
       subtitle = '',
       x = '',
       y = '',
       color = '') + 
  geom_vline() # add vertical lines for the average points scored at home and away.

## Error in games %>% filter() %>% ggplot(): could not find function "%>%"

Write answer here

Question 2 [0.75 points]

Now recreate the same plot for the 2018, 2019, and combined seasons. Imagine that you work for the Knicks organization and Gersson Rosas (the GM), asks you if the team scores more points at home or away? Based on your analysis, what would you tell him?

# By season
vertLines <- games %>%
filter() %>% # Filter to the New York Knicks (nameTeam)
  group_by() %>% # Group by the location and the season
  summarise() # Calculate the average points (pts)

## Error in games %>% filter() %>% group_by() %>% summarise(): could not find function "%>%"

games %>%
  filter() %>% # Filter to the New York Knicks (nameTeam)
  ggplot() + # Create a multivariate plot comparing points scored between home and away games
  geom_...() + # Choose the appropriate geom_... for this plot (i.e., geom_histogram(), geom_density(), geom_bar(), etc.)
  labs(title = '', # Add clear descriptions for the title, subtitle, axes, and legend
       subtitle = '',
       x = '',
       y = '',
       color = '') + 
  facet_wrap() + # Create separate panels for each season (facet_wrap())
  geom_vline() # add vertical lines for the average points scored at home and away.

## Error in games %>% filter() %>% ggplot(): could not find function "%>%"

# Over all seasons combined
vertLines <- games %>%
filter() %>% # Filter to the New York Knicks (nameTeam)
  group_by() %>% # Group by the location
  summarise() # Calculate the average points (pts)

## Error in games %>% filter() %>% group_by() %>% summarise(): could not find function "%>%"

games %>%
  filter() %>% # Filter to the New York Knicks (nameTeam)
  ggplot() + # Create a multivariate plot comparing points scored between home and away games
  geom_...() + # Choose the appropriate geom_... for this plot (i.e., geom_histogram(), geom_density(), geom_bar(), etc.)
  labs(title = '', # Add clear descriptions for the title, subtitle, axes, and legend
       subtitle = '',
       x = '',
       y = '',
       color = '') + 
  geom_vline() # add vertical lines for the average points scored at home and away.

## Error in games %>% filter() %>% ggplot(): could not find function "%>%"

Write answer here

Question 3 [0.75 points]

Gersson Rosas thanks you for your answer, but is a well-trained statistician in his own right, and wants to know how confident you are in your claim. Bootstrap sample the data 1,000 times to provide him with a more sophisticated answer. How confident are you in your conclusion that the Knicks score more points at home games than away games? Make sure to set.seed(123) to ensure you get the same answer every time you knit your code!

set.seed(123) # Set the seed!
forBS <- games %>% # To make things easier, create a new data object that is filtered to just the Knicks so we don't have to do this every time in the loop
    filter() # Filter to the Knicks (nameTeam)

## Error in games %>% filter(): could not find function "%>%"

bsRes <- NULL # Instantiate an empty object to store data from the loop
for(i in 1:1000) { # Loop 1,000 times
  bsRes <- forBS %>%
    sample_n() %>% # Sample the data with replacement using all possible rows
    group_by() %>% # Group by the location of the game
    summarise() %>% # Calculate the average points (pts)
    ungroup() %>% # Best practices!
    spread() %>% # Spread the data to get one column for average points at home and another for average points away
    mutate(, # Calculate the difference between home and away points
           ) %>% # Save the bootstrap index
    bind_rows(bsRes) # Append the result to the empty object from above
} 

## Error in forBS %>% sample_n() %>% group_by() %>% summarise() %>% ungroup() %>% : could not find function "%>%"

# Calculate the confidence
bsRes %>%
  summarise(, # Calculate the proportion of bootstrap simulations where the home points are greater than the away points
            ) # Calculate the overall average difference

## Error in bsRes %>% summarise(, ): could not find function "%>%"

Write answer here

Question 4 [0.75 points]

Re-do this analysis for three other statistics of interest to Gersson: total rebounds (treb), turnovers (tov), and field goal percent (pctFG). Do you notice anything strange in these results? What might explain it?

bsRes <- NULL # Instantiate an empty object to store data from the loop
for(i in 1:1000) { # Loop 1,000 times
  bsRes <- forBS %>%
    sample_n() %>% # Sample the data with replacement using all possible rows
    group_by() %>% # Group by the location of the game
    summarise(, # Calculate the average total rebounds (treb)
              , # Calculate the average turnovers (tov)
              ) %>% # Calculate the average field goal shooting percentage (pctFG)
    ungroup() %>% # Best practices!
    pivot_wider(, # Pivot wider to get each measure in its own colunm for home and away games
                ) %>% # Use the values from the variables you created above
    mutate(, # Calculate the difference between home and away total rebounds
           , # Calculate the difference between home and away turnovers
           , # Calculate the difference between home and away field goal percentages
           ) %>% # Save the bootstrap index
    bind_rows(bsRes) # Append the result to the empty object from above
} 

## Error in forBS %>% sample_n() %>% group_by() %>% summarise(, , ) %>% ungroup() %>% : could not find function "%>%"

# Calculate the confidence
bsRes %>%
  summarise(, # Calculate the confidence for rebounds being greater than zero
            , # Calculate the confidence for turnovers being greater than zero
            )

## Error in bsRes %>% summarise(, , ): could not find function "%>%"

Write answer here

Extra Credit [0.5 points]

Now Gersson is asking for a similar analysis of other teams. Calculate the difference between home and away points for every team in the league and prepare a summary table that includes both the average difference for each team, as well as your confidence about whether the difference is not zero. Based on these data, would you argue that there is an overall home court advantage in terms of points across the NBA writ large? Visualize these summary results by plotting the difference on the x-axis, the teams (reordered) on the y-axis, and the points colored by whether you are more than 90% confident in your answer. How should we interpret confidence levels less than 50%?

# INSERT CODE HERE

Write answer here

Question 0

Require tidyverse and load the youtube_individual.rds data to an object called yt.

# INSERT CODE HERE

Question 1 [0.5 point]

We are interested in how the YouTube recommendation algorithm works. These data are collected from real users, logged into their real YouTube accounts, allowing us to see who gets recommended which videos. We will investigate three research questions in this problem set:

1. What is the relationship between average ideology of recommendations shown to each user, and the average ideology of all the videos the user has watched?

2. What is the relationship between the average ideology of recommendations shown to each user, and the average ideology of the current video the user was watching when they were shown the recommendation?

3. Which of these relationships is stronger? Why?

Start by answering all three of these research questions, and explaining your thinking. Be very precise about your assumptions!

Write answer here

Question 2 [0.75 points]

Based on your previous answer, which variables are the \(X\) (predictors) and which are the \(Y\) (outcome) variables?

Write answer here

Now create univariate visualizations of all three variables, making sure to label your plots clearly.

# Univariate visualization of Y
# INSERT CODE HERE

# Univariate visualization of X1
# INSERT CODE HERE

# Univariate visualization of X2
# INSERT CODE HERE

Question 3 [0.75 points]

Let’s focus on the first research question. Create a multivariate visualization of the relationship between these two variables, making sure to put the \(X\) variable on the x-axis, and the \(Y\) variable on the y-axis. Add a straight line of best fit. Does the data support your theory?

# Multviariate visualization of Y and X1
# INSERT CODE HERE

Write answer here

Now run a linear regression using the lm() function and save the result to an object called model_watch.

model_watch <- lm(formula = ..., # Write the regression equation here (remember to use the tilde ~!)
             data = ...) # Indicate where the data is stored here.

## Error: '...' used in an incorrect context

Using either the summary() function (from base R) or the tidy() function (from the broom package), print the regression result.

require(broom)

## Loading required package: broom

tidy(model_watch)

## Error: object 'model_watch' not found

In a few sentences, summarize the results of the regression output. This requires you to translate the statistical measures into plain English, making sure to refer to the units for both the \(X\) and \(Y\) variables. In addition, you must determine whether the regression result supports your hypothesis, and discuss your confidence in your answer, referring to the p-value.

Write answer here

Question 4 [0.75 points]

Now let’s do the same thing for the second research question. First, create the multivariate visualization and determine whether it is consistent with your theory.

# Multviariate visualization of Y and X2
# INSERT CODE HERE

Write answer here

Second, run a new regression and save the result to model_current. Then print the result using either summary() or tidy(), as before.

model_current <- lm(formula = ..., # Write the regression equation here (remember to use the tilde ~!)
             data = ...) # Indicate where the data is stored here.

## Error: '...' used in an incorrect context

tidy(model_current)

## Error: object 'model_current' not found

Finally, describe the result in plain English, and interpret it in light of your hypothesis. How confident are you?

Write answer here

Based ONLY on the preceding analysis, are you able to answer research question 3?

Write answer here

Question 5 [1.25 points + 0.5 EC points]

Now let’s evaluate the models. Start by calculating the “mistakes” (i.e., the “errors” or the “residuals”) generated by both models and saving these as new columns (errors_watch and errors_current) in the yt dataset.

# Calculating errors 
yt <- yt %>%
  mutate(preds_watch = ..., # Get the predicted values from the first model (Yhat)
         preds_current = ...) %>% # Get the predicted values from the second model (Yhat)
  mutate(errors_watch = ..., # Calculate errors for the first model (Y - Yhat)
         errors_current = ...) # Calculate errors for the second model (Y - Yhat)

## Error in yt %>% mutate(preds_watch = ..., preds_current = ...) %>% mutate(errors_watch = ..., : could not find function "%>%"

Now create two univariate visualization of these errors. Based on this result, which model looks better? Why? EC [+0.5 point]: Plot both errors on the same graph using pivot_longer().

# Univariate visualization of watch history model errors 
yt %>%
  ggplot(aes(x = ...)) + # Put the errors from the first model on the x-axis
  geom_...() +  # Choose the best geom_...() to visualize based on the variable's type
  labs(...) # Provide clear labels to help a stranger understand!

## Error in yt %>% ggplot(aes(x = ...)): could not find function "%>%"

# Univariate visualization of current video model errors 
yt %>%
  ggplot(aes(x = ...)) + # Put the errors from the first model on the x-axis
  geom_...() +  # Choose the best geom_...() to visualize based on the variable's type
  labs(...) # Provide clear labels to help a stranger understand!

## Error in yt %>% ggplot(aes(x = ...)): could not find function "%>%"

# EC [0.5 points]: Plot both errors on a single plot. Hint: use pivot_longer().

Write answer here

Finally, create a multivariate visualization of both sets of errors, comparing them against the \(X\) variable. Based on this result, which model looks better? Why? EC [+0.5 point]: Create two plots side-by-side using facet_wrap(). This is SUPER HARD, so don’t worry if you can’t get it.

# Multivariate visualization of watch history errors
yt %>%
  ggplot(aes(x = ...,      # Put the predictor on the x-axis
             y = ...)) + # Put the errors on the y-axis
  geom_...() + # Choose the best geom_...()
  geom_...() + # Add a curvey line of best fit
  geom_hline(...) + # Add a horizontal dashed line at zero
  labs(...) # Provide clear labels to help a stranger understand!

## Error in yt %>% ggplot(aes(x = ..., y = ...)): could not find function "%>%"

# Multivariate visualization of current video errors
yt %>%
  ggplot(aes(x = ...,      # Put the predictor on the x-axis
             y = ...)) + # Put the errors on the y-axis
  geom_...() + # Choose the best geom_...()
  geom_...() + # Add a curvey line of best fit
  geom_hline(...) + # Add a horizontal dashed line at zero
  labs(...) # Provide clear labels to help a stranger understand!

## Error in yt %>% ggplot(aes(x = ..., y = ...)): could not find function "%>%"

# EC [0.5 point]: Try to create two plots side-by-side. (SUPER HARD)

Write answer here

Extra Credit [0.5 points]

Calculate the Root Mean Squared Error (RMSE) using 100-fold cross validation with a 50-50 split for both models. How bad are the first model’s mistakes on average? How bad are the second model’s mistakes? Which model seems better? Remember to talk about the result in terms of the range of values of the outcome variable! Plot the errors by the model using geom_boxplot(). HINT: you’ll need to use pivot_longer() to get the data shaped correctly.

# INSERT CODE HERE

Write answer here