STA 101 - F23 - Logistic regression

So far in regression

Outcome: Numerical, Predictor: One numerical or one categorical with only two levels $\to$ Simple linear regression
Outcome: Numerical, Predictors: Any number of numerical or categorical variables with any number of levels $\to$ Multiple linear regression
Outcome: Categorical with only two levels, Predictors: Any number of numerical or categorical variables with any number of levels $\to$ Logistic regression
Outcome: Categorical with any number of levels, Predictors: Any number of numerical or categorical variables with any number of levels $\to$ Generalized linear models – Not covered in STA 101

Data + packages

library(tidyverse)
library(tidymodels)

hp_spam <- read_csv("data/hp-spam.csv")

4601 emails collected at Hewlett-Packard labs and contains 58 variables
Outcome: type
- type = 1 is spam
- type = 0 is non-spam
Predictors of interest:
- capitalTotal: Number of capital letters in email
- Percentages are calculated as (100 * number of times the WORD appears in the e-mail) / total number of words in email
  - george: Percentage of “george”s in email (these were George’s emails)
  - you: Percentage of “you”s in email

Glimpse at data

What type of data is type? What type should it be in order to use logistic regression?

hp_spam |>
  select(type, george, capitalTotal, you)

# A tibble: 4,601 × 4
    type george capitalTotal   you
   <dbl>  <dbl>        <dbl> <dbl>
 1     1      0          278  1.93
 2     1      0         1028  3.47
 3     1      0         2259  1.36
 4     1      0          191  3.18
 5     1      0          191  3.18
 6     1      0           54  0   
 7     1      0          112  3.85
 8     1      0           49  0   
 9     1      0         1257  1.23
10     1      0          749  1.67
# ℹ 4,591 more rows

EDA: How much spam?

hp_spam |>
  count(type) |>
  mutate(p = n / sum(n))

# A tibble: 2 × 3
   type     n     p
  <dbl> <int> <dbl>
1     0  2788 0.606
2     1  1813 0.394

EDA: AM I SCREAMING? `capitalTotal`

ggplot(hp_spam, aes(x = capitalTotal)) +
  geom_histogram()

EDA: `george`, is that `you`?

ggplot(hp_spam, aes(x = george)) +
  geom_histogram()
ggplot(hp_spam, aes(x = you)) +
  geom_histogram()

Logistic regression

Logistic regression takes in a number of predictors and outputs the probability of a “success” (an outcome of 1) in a binary outcome variable.
The probability is related to the predictors via a sigmoid link function, $p (y_{i} = 1) = \frac{1}{1 + exp (- \sum β_{i} x_{i})},$ whose output is in $(0, 1)$ (a probability).
In this modeling scheme, one typically finds $\hat{β}$ by maximizing the likelihood function, another objective function, different than our previous “least squares” objective.

Logistic regression, visualized

Using data to estimate $β_{i}$

To proceed with building our email classifier, we will, as usual, use our data (outcome $y_{i}$ and predictor $x_{i}$ pairs), to estimate $β$ (find $\hat{β}$ ) and obtain the model: $p (y_{i} = 1) = \frac{1}{1 + exp (- \sum {\hat{β}}_{i} x_{i})},$

Application exercise

Go to Posit Cloud and continue the project titled ae-09-Spam.

ICYMI

Today’s daily check-in access code: slopes (released in class)

Logistic regression

Warm up

Check-in

Project

Logistic regression

So far in regression

Data + packages

Glimpse at data

EDA: How much spam?

EDA: AM I SCREAMING? `capitalTotal`

EDA: `george`, is that `you`?

Logistic regression

Logistic regression, visualized

Using data to estimate $β_{i}$

Application exercise