Data Wrangling in R

class: center, middle, inverse, title-slide

.title[
# Data Wrangling in <code>R</code>
]
.subtitle[
## Look at the Data!
]
.author[
### Prof. Weldzius
]
.institute[
### Villanova University
]

---

# Agenda

1. What is "data wrangling"?

2. Why `R`?

3. `tibbles` (table dataframes)

4. Political + data **science**

- Michigan exit polls
  
---

# What is "data wrangling"?

- Preparing data for analysis

- DANGER: Most important but least appreciated step!
  
---

# Danger!

- Data wrangling gone bad

---

# Danger!

- Data wrangling gone bad

---

# Danger!

- Data wrangling gone bad

---

# Danger!

- How do we avoid these mistakes?

- Three principles

1. Replicability
  
--

2. Understandability
  
--

3. Robustness
  
--

- The big picture

- Load **raw data** into an `RStudio` script
  
--

- Wrangle the data within this script
  
--

- Save a **new version** of the wrangled data with a **.red[different file name]**
  
--

- The point: Someone else should be able to recreate your work from scratch!
  
---

# Why `R`?

- Checks all three boxes

- But even `R` can be corrupted...

- ...with `Save workspace image`

- **.red[NEVER `Save workspace image`]**

- If you do, the next time you open `R`, it will load the modified data
  
---

# NEVER SAVE WORKSPACE IMAGE

---

# NEVER SAVE WORKSPACE IMAGE

---

# Why `R`?

- **.green[ALWAYS]** start with an `empty environment`

- How do you know if the environment is empty?

---

# Why `R`?

- **.green[ALWAYS]** start with an `empty environment`

- How do you know if the environment is empty?

---

# Why `R`?

- `R` scripts or `RMarkDown` scripts should be:

1. Single purpose (but definitions can vary)

2. Well-commented (make it easy for anyone to understand)
  
--

3. Iteratively built (constant bug checking)
  
---

# Let's get started

- Set up three subfolders
  
--
  
<center><img src="https://github.com/rweldzius/PSC4175/raw/main/static/images/folders.png" width="60%"></center>

- Download [`MI2020_ExitPoll.rds`](https://github.com/rweldzius/PSC4175/raw/main/static/data/MI2020_ExitPoll.rds) to your `data` folder

---

# Pausing for .blue[science]

- What are these data?

- "Exit poll" data from Michigan in the fall of 2020

- Polls fielded either on or just before an election
  
--

- Used to understand *why* voters chose a candidate...
  
--

- ...not to *predict* who will win (voting data is used for that)
  
---

# Michigan

- A "swing state" in 2020, following unexpected support for Trump in 2016

---

# Exit Polls

- These polls ask voters to check a box in response to a question

---

# Exit Polls

- These polls ask voters to check a box in response to a question

---

# Exit Polls

- **Predictive**: Use data to *predict* an outcome of interest.

1. How many voters report voting for each candidate?
  2. What predicts support for Trump? For Biden?
  
--

- **Descriptive**: Use data to *describe* an event.

1. How did Trump support vary by: gender? Age? Education?
  2. When did voters make up their minds?
  3. Why did voters support Trump or Biden?
  4. How do Trump and Biden voters vary in their opinions on: Covid? Race relations?
  
---

# Getting Started

- Open `RStudio` and `require(tidyverse)`

``` r
require(tidyverse)
require(labelled)
```

- Load the data

``` r
MI_raw  <- read_rds('https://github.com/rweldzius/PSC4175/raw/main/static/data/MI2020_ExitPoll.rds')
```

- And take a look

``` r
MI_raw
```

```
## # A tibble: 1,231 × 63
##       ID WEIGHT LALVOTERID    GROUP       ZIP DISTRICT    Z1
##    <dbl>  <dbl> <chr>         <dbl+lbl> <dbl>    <dbl> <dbl>
##  1     9  0.405 LALMI6290066  3 [3]     49327        2    NA
##  2    66  1.81  LALMI2492492  1 [1]     48234       14    NA
##  3   225  0.860 LALMI5489814… 4 [4]     48301        9 48322
##  4   243  0.199 LALMI5053772… 1 [1]     48130        7 48130
##  5   286  0.177 LALMI6831689  1 [1]     49946        1    NA
##  6   293  0.492 LALMI4019782  1 [1]     48615        4    NA
##  7   365  1.37  LALMI4151378  1 [1]     48906        4 48813
##  8   367  1.15  LALMI5912584  1 [1]     49442        2    NA
##  9   388  1.50  LALMI6635050  1 [1]     48451        5    NA
## 10   417  1.30  LALMI3567125  1 [1]     48197       12    NA
## # ℹ 1,221 more rows
## # ℹ 56 more variables: S1 <dbl+lbl>, S2A <dbl+lbl>,
## #   S2B <dbl+lbl>, S3 <dbl+lbl>, S4 <dbl+lbl>,
## #   VERSION <dbl+lbl>, PRSMI20 <dbl+lbl>,
## #   SENMI20 <dbl+lbl>, TIME16 <dbl+lbl>, ISSUE20 <dbl+lbl>,
## #   QLT20 <dbl+lbl>, TEMPBIDEN <dbl+lbl>,
## #   TEMPTRUMP <dbl+lbl>, CONTROLSEN <dbl+lbl>, …
```

---

# Tabular Data

- Data comes in many different formats

- **Structured data**: standardized, well-defined structure, easily accessed

- I.e., tables, databases
  
--
  
- **Unstructured data**: messy, organic, disorganized, hard to use

- I.e., web pages, images, videos

- In this class, we will always be working with **structured** data...specifically "tabular data frames"

- This still requires work to prepare!

---

# Tabular Data Frame

- AKA a "tibble"

- These are "square" (although usually rectangular)

- Rows: **units of observation** (i.e., the entities we are studying)

- People (each row is a survey respondent, athlete, etc.)
  
  - Places (each row is a state, county, country, etc.)
  
  - Things (each row is a tweet, firm, product, etc.)
  
--

- Columns: **variables of interest** (i.e., attributes we are studying)

- Beliefs / behaviors / etc. (i.e., where rows are people)
  
  - Rainfall / crimes / etc. (i.e., where rows are places)
  
  - Likes / profits / etc. (i.e., where rows are things)
  
---

# Back to our Michigan exit poll data

- What is the **unit of observation** in these data?

- Voters...which voters?
  
--

- Voters in Michigan...all of them?
  
--

- No just a random sample of those leaving the ballot box in 2020

---

# The Process: Steps 1-3

- Step 1: **Look** at the data

- Step 2: **Wrangle** the data

- Step 3: **Analyze** the data

---

# Step 1: Look at the data

- Goals:

1. Understand variable "types"

- Most important: continuous versus categorical

2. Identify **missingness**: either `NA` or "unit non-response"

- Unit non-response: observations who didn't provide information
  
--

3. Identify **skew** or other phenomena that require wrangling

- Highly skewed data should be logged
  
  - Some variables should be transformed to **rates** or **proportions**

---

# Step 1: Look at the data

- Methods:

1. Just look: see the first few rows and the first few columns
  
--

2. `glimpse()`: see the first few rows for every column
  
--

3. `summary()`: see the lowest, highest, mean, median, and quartiles, along with missingness (**better for continuous data**)
  
--

4. `count()`: see the number of observations in each category (**categorical data only**)
  
---

# Step 1: Look at the data

``` r
glimpse(MI_raw)
```

```
## Rows: 1,231
## Columns: 63
## $ ID             <dbl> 9, 66, 225, 243, 286, 293, 365, 367…
## $ WEIGHT         <dbl> 0.4045421, 1.8052619, 0.8601966, 0.…
## $ LALVOTERID     <chr> "LALMI6290066", "LALMI2492492", "LA…
## $ GROUP          <dbl+lbl> 3, 1, 4, 1, 1, 1, 1, 1, 1, 1, 1…
## $ ZIP            <dbl> 49327, 48234, 48301, 48130, 49946, …
## $ DISTRICT       <dbl> 2, 14, 9, 7, 1, 4, 4, 2, 5, 12, 7, …
## $ Z1             <dbl> NA, NA, 48322, 48130, NA, NA, 48813…
## $ S1             <dbl+lbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1…
## $ S2A            <dbl+lbl> 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2…
## $ S2B            <dbl+lbl> NA,  1,  1,  1,  2,  1,  1,  1,…
## $ S3             <dbl+lbl>  1, NA, NA, NA, NA, NA, NA, NA,…
## $ S4             <dbl+lbl>  1, NA, NA, NA, NA, NA, NA, NA,…
## $ VERSION        <dbl+lbl> 1, 2, 2, 1, 1, 2, 2, 2, 1, 1, 1…
## $ PRSMI20        <dbl+lbl> 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2…
## $ SENMI20        <dbl+lbl> 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2…
## $ TIME16         <dbl+lbl>  5, NA, NA,  5,  5, NA, NA, NA,…
## $ ISSUE20        <dbl+lbl>  5, NA, NA,  9,  1, NA, NA, NA,…
## $ QLT20          <dbl+lbl>  4, NA, NA,  4,  3, NA, NA, NA,…
## $ TEMPBIDEN      <dbl+lbl>  1, NA, NA,  1,  1, NA, NA, NA,…
## $ TEMPTRUMP      <dbl+lbl>  2, NA, NA,  2,  2, NA, NA, NA,…
## $ CONTROLSEN     <dbl+lbl>  9, NA, NA,  1,  1, NA, NA, NA,…
## $ FINSIT         <dbl+lbl>  3, NA, NA,  3,  3, NA, NA, NA,…
## $ ECONVCORONA20  <dbl+lbl>  1, NA, NA,  1,  1, NA, NA, NA,…
## $ FAVBIDEN2      <dbl+lbl>  1, NA, NA,  1,  1, NA, NA, NA,…
## $ FAVTRUMP       <dbl+lbl>  2, NA, NA,  2,  2, NA, NA, NA,…
## $ FORCAND        <dbl+lbl> NA,  1,  1, NA, NA,  1,  2,  1,…
## $ NEWVOTER       <dbl+lbl> NA,  2,  2, NA, NA,  2,  2,  2,…
## $ NEC            <dbl+lbl> NA,  4,  4, NA, NA,  4,  4,  4,…
## $ HANDLEECON20   <dbl+lbl> NA,  1,  1, NA, NA,  1,  1,  1,…
## $ HANDLECORONA20 <dbl+lbl> NA,  1,  1, NA, NA,  1,  1,  1,…
## $ RACISM20       <dbl+lbl> NA,  2,  2, NA, NA,  2,  2,  2,…
## $ VOTE2016       <dbl+lbl> NA,  1,  1, NA, NA,  1,  1,  1,…
## $ COUNTACC       <dbl+lbl> NA,  2,  2, NA, NA,  1,  2,  1,…
## $ TRUMP4         <dbl+lbl> NA,  4,  4, NA, NA,  4,  4,  4,…
## $ CONTAINCOVID   <dbl+lbl> NA,  4,  4, NA, NA,  4,  3,  4,…
## $ COVIDHARDSHIP  <dbl+lbl> NA,  3,  1, NA, NA,  2,  2,  2,…
## $ AGE10          <dbl+lbl>  2, 10,  7,  9,  8,  7,  9,  8,…
## $ SEX            <dbl+lbl> 2, 2, 2, 1, 2, 2, 1, 1, 2, 1, 1…
## $ EDUC18         <dbl+lbl> 4, 1, 5, 4, 5, 3, 3, 3, 4, 4, 5…
## $ QRACEAI        <dbl+lbl> 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 9…
## $ LATINOS        <dbl+lbl> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2…
## $ PARTYID        <dbl+lbl> 3, 1, 1, 3, 3, 3, 1, 1, 2, 1, 3…
## $ PHIL3          <dbl+lbl> 2, 2, 1, 9, 1, 2, 9, 2, 3, 2, 3…
## $ INCOME20       <dbl+lbl>  3, NA, NA,  9,  4, NA, NA, NA,…
## $ BRNAGAIN       <dbl+lbl> NA,  1,  2, NA, NA,  2,  1,  2,…
## $ CHILD12        <dbl+lbl> NA,  2,  2, NA, NA,  1,  2,  1,…
## $ LGBT           <dbl+lbl> NA,  2,  2, NA, NA,  2,  2,  2,…
## $ UNIONHH12      <dbl+lbl>  2, NA, NA,  2,  2, NA, NA, NA,…
## $ QN5            <dbl+lbl> 3, 2, 3, 2, 2, 3, 2, 2, 3, 2, 2…
## $ QN6A           <dbl+lbl> NA, NA,  2,  1, NA, NA,  2, NA,…
## $ QN6B           <dbl+lbl>  1,  1, NA, NA,  1,  1, NA,  1,…
## $ QN6C           <dbl+lbl> 4, 0, 0, 0, 2, 1, 0, 2, 1, 1, 2…
## $ County         <chr> "NEWAYGO", "WAYNE", "OAKLAND", "WAS…
## $ SMPFIPS        <dbl> 26123, 26163, 26125, 26161, 26013, …
## $ TTID           <dbl> 94989, 94079, 94040, 94468, 94564, …
## $ GEOCODE        <dbl> 5, 1, 2, 2, 5, 3, 2, 5, 2, 2, 3, 4,…
## $ a1age5         <dbl> 1, 5, 4, 5, 4, 4, 5, 4, 3, 4, 5, 5,…
## $ A1RACE         <dbl+lbl> 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 3…
## $ A1RACE3        <dbl+lbl> 1, 2, 1, 1, 1, 1, 1, 1, 1, 2, 3…
## $ A1SEX          <dbl+lbl> 2, 2, 2, 1, 2, 2, 1, 1, 2, 1, 1…
## $ A1EDUC         <dbl+lbl> 2, 1, 2, 2, 2, 1, 1, 1, 2, 2, 2…
## $ EDUCWHITE      <dbl+lbl> 1, 4, 1, 1, 1, 2, 2, 2, 1, 3, 3…
## $ VOTEMETHOD     <dbl> 1, 2, 2, 2, 3, 2, 2, 2, 1, 2, 1, 2,…
```

---

# Step 1-1: Variable Types

- **Continuous Variables**

- `dbl`: numeric data stored with great precision. Can be manipulated with mathematical functions.
  
  - `int`: numeric data stored as integers. `R` typically treats `dbl` and `int` as interchangeable.
  
--

- **Categorical Variables**
  
  - `chr`: string data, consisting of letters (and/or numbers). **Cannot** be manipulated with mathematical functions.
  
  - `fct`: string data that is stored with a number. Typically used to define group membership.
  
--

- **Mixtures**
  
  - `lbl`: string data that is stored with a number. Similar to `fct` but less commonly occurring in base `R`.
  
---

# Step 1-2: Missingness

- Two ways to indicate an observation is missing data

1. `NA` code
  
  2. Some bespoke code for "unit non-response" (often `9`, or `99`, or some large value ending in `9` that is dissimilar from the rest of the data)

---

# Step 1-2: Missingness

- To identify `NA`-style missingness, use `summary()`

``` r
summary(MI_raw %>% select(LALVOTERID,SEX,AGE10,PARTYID,LGBT,QLT20))
```

```
##   LALVOTERID             SEX           AGE10       
##  Length:1231        Min.   :1.00   Min.   : 1.000  
##  Class :character   1st Qu.:1.00   1st Qu.: 6.000  
##  Mode  :character   Median :2.00   Median : 8.000  
##                     Mean   :1.53   Mean   : 8.476  
##                     3rd Qu.:2.00   3rd Qu.: 9.000  
##                     Max.   :2.00   Max.   :99.000  
##                                                    
##     PARTYID           LGBT           QLT20      
##  Min.   :1.000   Min.   :1.000   Min.   :1.000  
##  1st Qu.:1.000   1st Qu.:2.000   1st Qu.:2.000  
##  Median :2.000   Median :2.000   Median :3.000  
##  Mean   :2.236   Mean   :2.224   Mean   :2.956  
##  3rd Qu.:3.000   3rd Qu.:2.000   3rd Qu.:4.000  
##  Max.   :9.000   Max.   :9.000   Max.   :9.000  
##                  NA's   :615     NA's   :616
```

---

# Step 1-2: Missingness

- To identify unit non-response, can use `count()`

``` r
MI_raw %>%
  count(PARTYID)
```

```
## # A tibble: 5 × 2
##   PARTYID                                 n
##   <dbl+lbl>                           <int>
## 1 1 [Democrat]                          425
## 2 2 [Republican]                        280
## 3 3 [Independent]                       416
## 4 4 [Something else]                     94
## 5 9 [[DON'T READ] Don’t know/refused]    16
```

- The number `9` indicates unit non-response for `PARTYID`

---

# Step 1-2: Missingness

- To identify unit non-response, can use `count()`

``` r
MI_raw %>%
  count(AGE10)
```

```
## # A tibble: 11 × 2
##    AGE10                         n
##    <dbl+lbl>                 <int>
##  1  1 [18 and 24,]              33
##  2  2 [25 and 29,]              28
##  3  3 [30 and 34,]              42
##  4  4 [35 and 39,]              46
##  5  5 [40 and 44,]              78
##  6  6 [45 and 49,]              83
##  7  7 [50 and 59,]             274
##  8  8 [60 and 64,]             143
##  9  9 [65 and 74,]             290
## 10 10 [75 or over?]            199
## 11 99 [[DON'T READ] Refused]    15
```

- The number `99` indicates unit non-response for `AGE10`

---

# Step 1-2: Missingness

- To identify unit non-response, can use `count()`

``` r
MI_raw %>%
  count(LGBT)
```

```
## # A tibble: 4 × 2
##   LGBT                                     n
##   <dbl+lbl>                            <int>
## 1  1 [Yes]                                23
## 2  2 [No]                                570
## 3  9 [[DON'T READ] Don’t know/Refused]    23
## 4 NA                                     615
```

- The number `9` indicates unit non-response for `LGBT` **AND** this variable also has `NA`

---

# Step 1-2: Missingness

- If the data isn't labelled, can still **look** to identify unit non-response

- Re-introducing...`ggplot()`

- Only interested in **single variable**
  
--

- For now, we need two parts:

1. `aes(x = [x-axis variable])`
  
  2. `geom_histogram()` (for continuous) or `geom_bar()` (for categorical)
  
---

# `ggplot()` Intro

``` r
MI_raw %>%
  ggplot(aes(x = AGE10)) + 
  geom_bar()
```

---

# `ggplot()` Intro

``` r
MI_raw %>%
  ggplot(aes(x = LGBT)) + 
  geom_bar()
```

---

# `ggplot()` Intro

``` r
MI_raw %>%
  ggplot(aes(x = PARTYID)) + 
  geom_bar()
```

---

# Step 1-3: Skew & Transformations

- To identify **skew**, we will also need `ggplot()`

- We will return to this later

---

# Step 2: Wrangle

- Why wrangle data?

``` r
MI_raw %>%
  summarise(avgAge = mean(AGE10,na.rm=T))
```

```
## # A tibble: 1 × 1
##   avgAge
##    <dbl>
## 1   8.48
```

---

# Step 2: Wrangle

- We over-estimate the average age category because `R` doesn't realize these are codes!

``` r
MI_raw %>%
  count(AGE10)
```

---

# Step 2: Wrangle

- We need to convert **codes** for missing data to `NA`

- Use `mutate()` and `ifelse()` to replace `99` with `NA`

- `mutate()`:

- Either creates a new column
  
  - Or changes an existing column
  
--

- `ifelse()`:

- Does exactly as it says
  
  - `ifelse([LOGIC],[VALUE IF TRUE],[VALUE IF FALSE])`
  
---

# Step 2: Wrangle

- If: `AGE10` is `99` (i.e., the unit non-response code)

- Then: give me an `NA` value

- Otherwise: give me whatever `AGE10` is

``` r
MI_raw %>%
  mutate(AGE10_new = ifelse(AGE10 == 99,NA,AGE10)) %>%
  select(AGE10,AGE10_new) %>%
  slice(c(35,1:20)) # Don't need to know slice() yet, ignore
```

```
## # A tibble: 21 × 2
##    AGE10                     AGE10_new
##    <dbl+lbl>                     <dbl>
##  1 99 [[DON'T READ] Refused]        NA
##  2  2 [25 and 29,]                   2
##  3 10 [75 or over?]                 10
##  4  7 [50 and 59,]                   7
##  5  9 [65 and 74,]                   9
##  6  8 [60 and 64,]                   8
##  7  7 [50 and 59,]                   7
##  8  9 [65 and 74,]                   9
##  9  8 [60 and 64,]                   8
## 10  6 [45 and 49,]                   6
## # ℹ 11 more rows
```

---

# Step 2: Wrangle

- The Assignment Operator (`<-`): `R`'s version of "Save As..."

- If we don't either:

1. Overwrite the `MI_raw` object...
  
  2. Or create a new object...
  
--

- `R` will not remember this new `AGE10_new` variable we created

``` r
MI_raw %>% select(AGE10_new)
```

```
## Error in `select()`:
## ! Can't select columns that don't exist.
## ✖ Column `AGE10_new` doesn't exist.
```

---

# Step 2: Wrangle

- Thus we need to use the **assignment operator** (`<-`)

``` r
MI_final <- MI_raw %>%
  mutate(AGE10_new = ifelse(AGE10 == 99,NA,AGE10))

MI_final %>%
  select(AGE10,AGE10_new) %>%
  slice(c(35,1:20)) # Don't need to know slice() yet, ignore
```

---

# Step 2: Wrangle

- We can now get a more accurate measure of the average age category

``` r
MI_final %>%
  summarise(avgAge = mean(AGE10_new,na.rm=T))
```

```
## # A tibble: 1 × 1
##   avgAge
##    <dbl>
## 1   7.36
```

---

# Variable Classes

- **How should we interpret this number?**

- Does this mean that the average age of respondents is 7?
  
--

- **NO**: we want to keep the definitions!
  
---

# Converting to `chr`

- There is a helpful package called `haven` which will extract these labels as `factors`

- Install it with `install.packages("haven")` but DON'T `require()` it

- We can call on useful functions without `require()` by using two `::`

``` r
haven::as_factor(MI_final$AGE10) %>% head()
```

```
## [1] 25 and 29,  75 or over? 50 and 59,  65 and 74, 
## [5] 60 and 64,  50 and 59, 
## 11 Levels: 18 and 24, 25 and 29, 30 and 34, ... [DON'T READ] Refused
```

---

# Step 2: Wrangle

- Goal: Create a new column that converts the value for `AGE10` to a label

- To do this, use `mutate()` + the `haven::as_factor()` function + the `as.character()` function

- Step 1: `as_factor()` function (from `haven`) converts to `fct` class
  
  - Step 2: `as.character()` function (from base `R`) converts to `chr` class

``` r
MI_final <- MI_final %>%
  mutate(AGE10_labs = as.character(haven::as_factor(AGE10)))
```

- **NB:** we **overwrite** `MI_final` to **add** the new column `AGE10_labs`

---

# Step 2: Wrangle

``` r
MI_final %>%
  ggplot(aes(x = AGE10_labs)) + 
  geom_bar(stat = 'count')
```

---

# Aside on `geom_bar()`

- Default: `stat = "count"` means show total observations in each category

- Override: `stat = "identity"` forces to show a specific value set in `aes()`

``` r
MI_final %>%
  count(AGE10_labs) %>%
  ggplot(aes(x = AGE10_labs,y = n)) + 
  geom_bar(stat = 'identity')
```

---

# Step 2: Wrangle

- Say we want the category labels for `PRSMI20`, `QLT20`, and `LGBT`

``` r
MI_final <- MI_final %>%
  mutate(preschoice = as.character(haven::as_factor(PRSMI20)),
         Qlty = as.character(haven::as_factor(QLT20)),
         LGBT_lab = as.character(haven::as_factor(LGBT)))

MI_final %>%
  select(preschoice,Qlty,LGBT_lab)
```

```
## # A tibble: 1,231 × 3
##    preschoice                   Qlty                LGBT_lab
##    <chr>                        <chr>               <chr>   
##  1 Joe Biden, the Democrat      Has good judgment   <NA>    
##  2 Joe Biden, the Democrat      <NA>                No      
##  3 Joe Biden, the Democrat      <NA>                No      
##  4 Joe Biden, the Democrat      Has good judgment   <NA>    
##  5 Joe Biden, the Democrat      Cares about people… <NA>    
##  6 Joe Biden, the Democrat      <NA>                No      
##  7 Joe Biden, the Democrat      <NA>                No      
##  8 Joe Biden, the Democrat      <NA>                No      
##  9 Donald Trump, the Republican Cares about people… <NA>    
## 10 Joe Biden, the Democrat      Cares about people… <NA>    
## # ℹ 1,221 more rows
```

---

# Preparing a new `tibble`

- .blue[Science] guides us toward which variables to focus on

``` r
names(MI_final)
```

```
##  [1] "ID"             "WEIGHT"         "LALVOTERID"    
##  [4] "GROUP"          "ZIP"            "DISTRICT"      
##  [7] "Z1"             "S1"             "S2A"           
## [10] "S2B"            "S3"             "S4"            
## [13] "VERSION"        "PRSMI20"        "SENMI20"       
## [16] "TIME16"         "ISSUE20"        "QLT20"         
## [19] "TEMPBIDEN"      "TEMPTRUMP"      "CONTROLSEN"    
## [22] "FINSIT"         "ECONVCORONA20"  "FAVBIDEN2"     
## [25] "FAVTRUMP"       "FORCAND"        "NEWVOTER"      
## [28] "NEC"            "HANDLEECON20"   "HANDLECORONA20"
## [31] "RACISM20"       "VOTE2016"       "COUNTACC"      
## [34] "TRUMP4"         "CONTAINCOVID"   "COVIDHARDSHIP" 
## [37] "AGE10"          "SEX"            "EDUC18"        
## [40] "QRACEAI"        "LATINOS"        "PARTYID"       
## [43] "PHIL3"          "INCOME20"       "BRNAGAIN"      
## [46] "CHILD12"        "LGBT"           "UNIONHH12"     
## [49] "QN5"            "QN6A"           "QN6B"          
## [52] "QN6C"           "County"         "SMPFIPS"       
## [55] "TTID"           "GEOCODE"        "a1age5"        
## [58] "A1RACE"         "A1RACE3"        "A1SEX"         
## [61] "A1EDUC"         "EDUCWHITE"      "VOTEMETHOD"    
## [64] "AGE10_new"      "AGE10_labs"     "preschoice"    
## [67] "Qlty"           "LGBT_lab"
```

---

# Let's Wrangle!

- We are interested in presidential vote choice (`PRSMI20` or `preschoice`) and the quality of the candidate the respondent likes most (`QLT20` or `Qlty`)

- Look at these first

``` r
MI_final %>% select(preschoice,Qlty)
```

```
## # A tibble: 1,231 × 2
##    preschoice                   Qlty                      
##    <chr>                        <chr>                     
##  1 Joe Biden, the Democrat      Has good judgment         
##  2 Joe Biden, the Democrat      <NA>                      
##  3 Joe Biden, the Democrat      <NA>                      
##  4 Joe Biden, the Democrat      Has good judgment         
##  5 Joe Biden, the Democrat      Cares about people like me
##  6 Joe Biden, the Democrat      <NA>                      
##  7 Joe Biden, the Democrat      <NA>                      
##  8 Joe Biden, the Democrat      <NA>                      
##  9 Donald Trump, the Republican Cares about people like me
## 10 Joe Biden, the Democrat      Cares about people like me
## # ℹ 1,221 more rows
```

---

# Let's Wrangle!

- We can get some preliminary descriptive info with `count()`

``` r
MI_final %>%
  count(preschoice)
```

```
## # A tibble: 6 × 2
##   preschoice                          n
##   <chr>                           <int>
## 1 Another candidate                  25
## 2 Donald Trump, the Republican      459
## 3 Joe Biden, the Democrat           723
## 4 Refused                            14
## 5 Undecided/Don’t know                4
## 6 Will/Did not vote for president     6
```

---

# Let's Wrangle!

- We can get some preliminary descriptive info with `count()` (.blue[science!])

``` r
MI_final %>%
  count(preschoice,SEX)
```

```
## # A tibble: 12 × 3
##    preschoice                      SEX            n
##    <chr>                           <dbl+lbl>  <int>
##  1 Another candidate               1 [Male]      17
##  2 Another candidate               2 [Female]     8
##  3 Donald Trump, the Republican    1 [Male]     247
##  4 Donald Trump, the Republican    2 [Female]   212
##  5 Joe Biden, the Democrat         1 [Male]     304
##  6 Joe Biden, the Democrat         2 [Female]   419
##  7 Refused                         1 [Male]       7
##  8 Refused                         2 [Female]     7
##  9 Undecided/Don’t know            1 [Male]       3
## 10 Undecided/Don’t know            2 [Female]     1
## 11 Will/Did not vote for president 1 [Male]       1
## 12 Will/Did not vote for president 2 [Female]     5
```

---

# Choosing the right function

- But `count()` is less useful for continuous variables

``` r
MI_final %>%
  count(WEIGHT)
```

```
## # A tibble: 411 × 2
##    WEIGHT     n
##     <dbl> <int>
##  1  0.100     1
##  2  0.113     1
##  3  0.119     1
##  4  0.133     2
##  5  0.141     1
##  6  0.142     1
##  7  0.144     1
##  8  0.146     1
##  9  0.147     1
## 10  0.149     5
## # ℹ 401 more rows
```

---

# Choosing the right function

- Use `summary()` instead

``` r
MI_final %>%
  select(WEIGHT) %>%
  summary(WEIGHT)
```

```
##      WEIGHT      
##  Min.   :0.1003  
##  1st Qu.:0.3775  
##  Median :0.8020  
##  Mean   :1.0000  
##  3rd Qu.:1.4498  
##  Max.   :5.0853
```

---

# Let's Wrangle!

- Recall some of the other helpful functions in `tidyverse`

``` r
MI_final %>%
  count(preschoice) %>%
  arrange(desc(n))
```

```
## # A tibble: 6 × 2
##   preschoice                          n
##   <chr>                           <int>
## 1 Joe Biden, the Democrat           723
## 2 Donald Trump, the Republican      459
## 3 Another candidate                  25
## 4 Refused                            14
## 5 Will/Did not vote for president     6
## 6 Undecided/Don’t know                4
```

---

# Let's Wrangle!

- Let's look for missing data

``` r
MI_final %>%
  count(Qlty)
```

```
## # A tibble: 6 × 2
##   Qlty                                n
##   <chr>                           <int>
## 1 Can unite the country             125
## 2 Cares about people like me        121
## 3 Has good judgment                 205
## 4 Is a strong leader                138
## 5 [DON'T READ] Don’t know/refused    26
## 6 <NA>                              616
```

---

# Let's Wrangle!

- Drop rows where the respondent didn't answer the "quality" question

``` r
MI_final %>%
  drop_na(Qlty) %>%
  count(Qlty)
```

```
## # A tibble: 5 × 2
##   Qlty                                n
##   <chr>                           <int>
## 1 Can unite the country             125
## 2 Cares about people like me        121
## 3 Has good judgment                 205
## 4 Is a strong leader                138
## 5 [DON'T READ] Don’t know/refused    26
```

---

# Let's Wrangle!

- But we still have the `[DON'T READ] Don't know/refused` respondents

- Convert "unit non-response" codes to `NA`

- Need to look at them first!

``` r
MI_final %>%
  count(Qlty)
```

---

# Let's Wrangle!

- Convert "unit non-response" codes to `NA`

``` r
MI_final <- MI_final %>%
  mutate(Qlty = ifelse(grepl("DON'T READ",Qlty),NA,Qlty))
MI_final %>%
  count(Qlty)
```

```
## # A tibble: 5 × 2
##   Qlty                           n
##   <chr>                      <int>
## 1 Can unite the country        125
## 2 Cares about people like me   121
## 3 Has good judgment            205
## 4 Is a strong leader           138
## 5 <NA>                         642
```

---

# Let's Wrangle!

- Drop rows where the respondent didn't answer the "quality" question

``` r
MI_final %>%
  drop_na(Qlty) %>%
  count(Qlty)
```

```
## # A tibble: 4 × 2
##   Qlty                           n
##   <chr>                      <int>
## 1 Can unite the country        125
## 2 Cares about people like me   121
## 3 Has good judgment            205
## 4 Is a strong leader           138
```

---

# Let's Wrangle!

- This is equivalent to `filter(!is.na(Qlty))`

``` r
MI_final %>%
  filter(!is.na(Qlty)) %>%
  count(Qlty)
```

---

# Aside on filtering `NA`

- Make sure to specify which column has the missing data!

- Otherwise get "complete cases" (respondents with no missing data)

``` r
MI_final %>%
  drop_na() %>%
  count(QLT20)
```

```
## # A tibble: 0 × 2
## # ℹ 2 variables: QLT20 <dbl+lbl>, n <int>
```

- Same as

``` r
MI_final %>%
  filter(complete.cases(.)) %>%
  count(QLT20)
```

```
## # A tibble: 0 × 2
## # ℹ 2 variables: QLT20 <dbl+lbl>, n <int>
```

---

# Let's Wrangle!

- Finally, let's subset the data to focus only on the other variables we are interested in

``` r
MI_final <- MI_final %>%
  select(SEX,AGE10,PARTYID,
         WEIGHT,QRACEAI,EDUC18,LGBT,
         BRNAGAIN,LATINOS,RACISM20,
         QLT20,Qlty,PRSMI20,preschoice)
```

- This is our **wrangled** data

``` r
MI_final
```

```
## # A tibble: 1,231 × 14
##    SEX        AGE10           PARTYID     WEIGHT QRACEAI  
##    <dbl+lbl>  <dbl+lbl>       <dbl+lbl>    <dbl> <dbl+lbl>
##  1 2 [Female]  2 [25 and 29,] 3 [Indepen…  0.405 1 [White]
##  2 2 [Female] 10 [75 or over… 1 [Democra…  1.81  2 [Black]
##  3 2 [Female]  7 [50 and 59,] 1 [Democra…  0.860 1 [White]
##  4 1 [Male]    9 [65 and 74,] 3 [Indepen…  0.199 1 [White]
##  5 2 [Female]  8 [60 and 64,] 3 [Indepen…  0.177 1 [White]
##  6 2 [Female]  7 [50 and 59,] 3 [Indepen…  0.492 1 [White]
##  7 1 [Male]    9 [65 and 74,] 1 [Democra…  1.37  1 [White]
##  8 1 [Male]    8 [60 and 64,] 1 [Democra…  1.15  1 [White]
##  9 2 [Female]  6 [45 and 49,] 2 [Republi…  1.50  1 [White]
## 10 1 [Male]    8 [60 and 64,] 1 [Democra…  1.30  2 [Black]
## # ℹ 1,221 more rows
## # ℹ 9 more variables: EDUC18 <dbl+lbl>, LGBT <dbl+lbl>,
## #   BRNAGAIN <dbl+lbl>, LATINOS <dbl+lbl>,
## #   RACISM20 <dbl+lbl>, QLT20 <dbl+lbl>, Qlty <chr>,
## #   PRSMI20 <dbl+lbl>, preschoice <chr>
```

---

# Finishing up

- And now save both the raw data and our prepared data together

``` r
save(MI_raw,MI_final,file = '../data/MI_prepped.RData')
```

- An `.RData` file can contain multiple objects

- When we load it, we get access to all the objects we've created

``` r
rm(MI_raw,MI_final)

load('../data/MI_prepped.RData')
```

- **NB: different file extensions require different functions!**

- `load` for `.RData` files
  
  - `read_rds` for `.rds` files
  
  - `read_csv` for `.csv` files

---

# Conclusion

- What you need to know from today

1. Why `RStudio` + `R` are great for **wrangling**
  
--
  
  2. `mutate()`
    
--

3. `ifelse()`
  
--

4. Assignment Operator ("Save As..."): `<-`