Conditional Probability

Definition

Conditional probability is the probability of an event after we know that another event has occurred.

The conditional probability of $A$ given $B$ is:

where:

• $A$ is the event we want to predict.
• $B$ is the event we have observed.
• $P(B) > 0$.

Interpretation

The expression:

means:

among the cases where $B$ happened, how often does $A$ happen?

It restricts attention to the subset of cases where $B$ is true.

Example

Suppose:

• $A$ = customer returns within 30 days.
• $B$ = customer bought more than 10 items.

Then:

means:

the probability that a customer returns within 30 days, given that they bought more than 10 items.

Counting Formula

In a dataset, conditional probability can be estimated by counting:

where:

• $#(A \cap B)$ is the number of observations where both $A$ and $B$ happened.
• $#(B)$ is the number of observations where $B$ happened.

Retail Basket Example

Let:

• $A$ = basket contains a mug.
• $B$ = basket contains a teapot.

Then:

is the probability that a basket contains a mug given that it contains a teapot.

This is closely related to association rules and basket-completion prediction.

Relation to Independence

If $A$ and $B$ are independent, then knowing $B$ does not change the probability of $A$:

If:

then $B$ increases the probability of $A$.

Relation to Conditional Expectation

Conditional probability applies to events.

Conditional expectation applies to numerical random variables.

For a binary variable $Y \in {0,1}$:

This is why logistic regression can be viewed as a conditional mean model for binary outcomes.

Exercises

1. Explain $P(A \mid B)$ in words.
2. If 40 baskets contain a teapot and 25 of those also contain a mug, estimate $P(\text{mug} \mid \text{teapot})$.
3. Explain the difference between $P(A \mid B)$ and $P(B \mid A)$.

See

Expected Value

Conditional Expectation

Conditional Mean Estimation