Regression

class: center, middle, inverse, title-slide

.title[
# Regression
]
.subtitle[
## Part 1a
]
.author[
### Prof. Weldzius
]
.institute[
### Villanova University
]

---

# Agenda

1. Modeling Conditional Variation

2. Adding Regression to the **Process**

3. Next time: Demonstrating Regressions

---

# Regression & Conditional Analysis

- Recall our discussion of **conditional** analysis

- Conditional &rarr; **depends on**
  
--

- Analyze with **conditional means**
  
---

# Reminder of the **Process**

1. Determine variable **type**

- I.e., categorical (ordered, unordered, binary) or continuous
  
  - In `R` terms: `chr`, `fct` for categorical, `dbl` for continuous
  
--

2. Type informs **univariate analysis**

- I.e., histograms for continuous, barplots for categorical
  
--

3. Combination of types informs **conditional analysis**

- Categorical X Categorical: proportions by categories (`geom_bar`)
  
  - Binary X Continuous: histograms by categories (`geom_histogram` / `geom_density`)
  
  - Categorical X Continuous: distributions by categories (`geom_boxplot` / `geom_violin`)
  
  - Continuous X Continuous: scatter plots (`geom_point`)
  
---

# What is regression?

- Conditional means for continuous data

---

# What is regression?

- People with hourly wages < $20 spend ~$50 on entertainment per week

---

# What is regression?

- People with hourly wages > $40 spend ~$95 on entertainment per week

---

# What is regression?

- .blue[Theory]: the more you earn, the more you spend

---

# What is regression?

- But **conditional means** make a lot of mistakes. Can we do better?

---

# What is regression?

- But **conditional means** make a lot of mistakes. Can we do better?

---

# Regression

- Calculating a **line** that minimizes mistakes *for every observation*

- NB: could be a curvey line! For now, just assume straight
  
--

- Recall from geometry how to graph a straight line

- `$Y = a + bX$`

- `$a$`: the "intercept" (where the line intercepts the y-axis)
  - `$b$`: the "slope" (how much `$Y$` changes for each increase in `$X$`)

- (Data scientists use `$\alpha$` and `$\beta$` instead of `$a$` and `$b$` b/c nerds)

- Regression analysis simply chooses the best line

- "Best"?

- The line that minimizes the mistakes (the **line of best fit**)

---

# Linear Regression

.leftcol[

]

.rightcol[

]

---

# Linear Regression

.leftcol[

]

.rightcol[

]
---

# Linear Regression

.leftcol[

]

.rightcol[

- **Error/Residual**: mistake made by a line

- In math: `$u_i = y_i - \hat{y}_i$`
  
  - In English: difference between true outcome value ( `$y_i$` ) and prediction ( `$\hat{y}_i$` )

]

---

# Linear Regression

.leftcol[

]

.rightcol[

- Use **errors** to find **line of best fit**

- **RMSE** (**R**oot **M**ean **S**quared **E**rror)

- Square the errors
  
  - Take their average
  
  - Take the square root
  
- **RMSE** = 1.23

]

---

# Linear Regression

.leftcol[

]

.rightcol[

- Use **errors** to find **line of best fit**

- **RMSE** (**R**oot **M**ean **S**quared **E**rror)

- Square the errors
  
  - Take their average
  
  - Take the square root
  
- **RMSE** = 1.48

]

---

# Linear Regression

.leftcol[

]

.rightcol[

- Use **errors** to find **line of best fit**

- **RMSE** (**R**oot **M**ean **S**quared **E**rror)

- Square the errors
  
  - Take their average
  
  - Take the square root
  
- **RMSE** = 2.19

]

---

# Linear Regression

.leftcol[

]

.rightcol[

- Use **errors** to find **line of best fit**

- **RMSE** (**R**oot **M**ean **S**quared **E**rror)

- Square the errors
  
  - Take their average
  
  - Take the square root
  
- **RMSE** = 1.03

]

---

# Visual Intuition

---

# Visual Intuition

---

# Regression

- The line is .blue[substantively meaningful]

- Red line on scatter plot of spending and wages: `$Y = 12 + 2*X$`

- `$\alpha$` tells us the value of `$Y$` when `$X$` is zero

- People who don't make any money spend $12 per week on entertainment
  
--

- `$\beta$` tells us how much `$Y$` increases for each additional `$X$`

- People spend an additional $2 per week for each additional $1 in hourly wages
  
---

# Regression

- These are called **"linear models"**

- **Not** because the line is straight (it might not be)
  
  - but because the components are additive ( `$\alpha + \beta X$` )
  
--

- Can extend to multiple predictors ( `$X$` 's)

- `$Y = \alpha + \beta_1 X_1 + \beta_2 X_2 + \dots + \varepsilon$`
  
--

- `$X_1$` might be wages and `$X_2$` might be age (for example)
  
--

- The final term `$\varepsilon$` measures how bad our mistakes are