Conditional Probability

20 min · 2 drills

What is it and why does it matter?

Imagine: you roll a die and your friend tells you "it's even". Now the probability of getting a 6 isn't $\frac{1}{6}$ anymore, because you know it's definitely 2, 4, or 6. Three options left, and 6 is one of them. So the probability = $\frac{1}{3}$.

That is exactly what conditional probability measures: the probability of an event when we already know something about the situation.

On EPM, you'll recognize these problems by phrases like:

• "Given that..."
• "If it is known that..."
• "Among those who..."

---

The Formula and What It Means

$$P(B\mid \mid A)==\frac{P(A\cap B)}{P(A)}$$

Read as: "probability of B given A".

Why does the formula look like this?

Once we learn that $A$ happened, the rest of the world doesn't matter anymore. Circle $A$ becomes our new "universe". And we ask: what fraction of this new universe is also $B$?

• Numerator $P(A\cap B)$: what we're looking for (both $A$ and $B$ together)
• Denominator $P(A)$: our new "universe"

Essentially, we're dividing "what we want" by "everything that's left".

---

Multiplication Rule for Dependent Events

From the conditional probability formula, an important rule follows:

$$\boxed{P(A\cap B)=P(A)\cdot P(B\mid A)}$$

This is the multiplication rule for dependent events. It allows us to find the probability that both events occur when the second depends on the first.

Example: From an urn with 5 white and 3 black balls, we draw 2 balls without replacement. What's the probability both are white?

• $P(\text{first white})=\frac{5}{8}$
• $P(\text{second white}|\text{first white})=\frac{4}{7}$ (4 white remain out of 7)
• $P(\text{both white})=\frac{5}{8}\cdot \frac{4}{7}=\frac{20}{56}=\frac{5}{14}$

---

Addition Rule for Unions (Additionssatz)

The multiplication rule answers the question about $A\cap B$, that is, about both events happening together. The second question is $A\cup B$: at least one of the two happens. The task text writes this as „oder“ (or), and that „oder“ explicitly includes the case where both are true.

$$P(A\cup \cup B)=$$

$$P(A)++P(B)--P(A\cap \cap B)$$

The subtraction at the end is what the formula is really about. Adding $P(A)$ and $P(B)$ alone counts every case belonging to both events twice: once in $P(A)$ and again in $P(B)$. One of those two counts is one too many, so $P(A\cap B)$ is taken off again exactly once.

Mutually Exclusive Events

If $A$ and $B$ cannot happen at the same time, then $A\cap B=\varnothing$ and therefore $P(A\cap B)=0$. The subtraction drops out and the formula shortens:

$$P(A\cup \cup B)==P(A)++P(B)\text{if}A\cap \cap B==\varnothing$$

The Union in the Table

In a four-field table, $A\cup B$ covers three of the four inner fields. The only field left outside is the one where neither $A$ nor $B$ happens:

That gives a second route which needs no subtraction and works well as a check:

$$P(A\cup \cup B)==1--P(\overline{A}\cap \cap \overline{B})$$

The table may also have more than two columns, three sectors or three device types for instance. This changes nothing about the addition rule; only the number of fields and the number of terms in the marginal sums grow.

Example: Stadium Sectors

A stadium has 30,000 seats in sectors $A$, $B$ and $C$. 23,100 spectators hold Austrian citizenship, the rest are guests from abroad. Filled in, the table looks like this:

$\ddot{O}$ stands for "the person holds Austrian citizenship", $G$ for "the person does not hold it", and $A$, $B$ and $C$ stand for the sector the person is sitting in.

$P(A\cup B)$: Nobody sits in two sectors at once, so $P(A\cap B)=0$ and the subtraction falls away.

$$P(A\cup \cup B)==\frac{12000}{30000}++\frac{9000}{30000}==0.40++0.30=$$

$$70\%$$

$P(C\cup G)$: Here the events do overlap, because 5,400 guests from abroad sit in sector $C$. Those people sit inside $P(C)$ and inside $P(G)$ alike.

$$P(C\cup \cup G)==\frac{9000}{30000}++\frac{6900}{30000}--\frac{5400}{30000}==0.30++0.23--0.18=$$

$$35\%$$

Without the subtraction the answer would read $53\%$. Checking it through the fields: sector $C$ holds $3600+5400=9000$ people, and outside $C$ another $600+900=1500$ guests are added, giving $10500$ out of $30000$, which is $35\%$ again.

The same table gets asked about in two ways. Some of the questions are spelled out in German („eine Person ist Gast aus dem Ausland und sitzt nicht im Sektor C“, a person is a guest from abroad and does not sit in sector C), the others are in pure symbolic notation such as $P(\overline{B})$, $P(A\cap B)$ or $P(A\cup B)$. Both mean the same thing: the sentence above is $P(G\cap \overline{C})$. Here „und“ stands for $\cap$, „oder“ for $\cup$, „nicht“ for the overbar and „unter denen, die ...“ for the vertical bar.

---

Examples

Example 1: Dice

Rolled a die. Given that it's even, what's the probability it's a 6?

Breaking it down:

• $A$ = even = 6: our condition, our new "universe"
• $B$ = six = 6: what we're looking for
• $A\cap B$ = 6: the intersection (a six that's also even, which is just six)

Calculating:

$$P(B\mid \mid A)==\frac{P(A\cap B)}{P(A)}==\frac{1/6}{3/6}==\frac{1}{3}$$

Or simpler: out of three even numbers (2, 4, 6), one works (6). Answer: $\frac{1}{3}$.

Example 2: Cards

Drew a card from a deck (52 cards). Given that it's red, what's the probability it's an ace?

Breaking it down:

• Red cards: 26 (hearts + diamonds)
• Aces: 4 (one in each suit)
• Red aces: 2 (ace of hearts and ace of diamonds)

Calculating: Out of 26 red cards, 2 are aces. So: $\frac{2}{26}=\frac{1}{13}$

---

Law of Total Probability

Sometimes we need to calculate the overall probability of an event, knowing probabilities in different groups.

Example: Three Machines

A workshop has three machines:

• First produces 50% of all parts, defect rate: 2%
• Second produces 30% of all parts, defect rate: 3%
• Third produces 20% of all parts, defect rate: 4%

What's the probability that a random part is defective?

$$P(\text{defect})==0.5\times \times 0.02++0.3\times \times 0.03++0.2\times \times 0.04==0.027$$

The logic: A defective part could come from any machine. We calculate each one's contribution:

• From machine 1: $0.50\times 0.02=0.010$ (half of parts, 2% defective)
• From machine 2: $0.30\times 0.03=0.009$ (a third of parts, 3% defective)
• From machine 3: $0.20\times 0.04=0.008$ (a fifth of parts, 4% defective)

Add them up: $0.010+0.009+0.008=0.027=2.7\%$

This is the law of total probability:

$$P(\text{defect})==P(H_1)\cdot \cdot P(\text{defect}\mid \mid H_1)++P(H_2)\cdot \cdot P(\text{defect}\mid \mid H_2)++P(H_3)\cdot \cdot P(\text{defect}\mid \mid H_3)$$

where $H_1,H_2,H_3$ mean "part came from machine 1, 2, 3".

General Formula

$$P(A)==\sum _{i}P(H_i)\cdot \cdot P(A\mid \mid H_i)$$

where $H_1,H_2,\dots$ is a complete set of mutually exclusive hypotheses (their probabilities sum to 1).

---

Bayes' Formula: "Flipping" the Condition

Now an interesting question: a part turned out defective. Which machine did it most likely come from?

We know $P(\text{defect}|H_1)$ = 2%: the probability of a defect given the part is from machine 1.

But we need $P(H_1|\text{defect})$: the probability of machine 1 given the part is defective.

This is the "reverse" problem, solved by Bayes' formula:

$$P(H_1\mid \mid \text{defect})==\frac{P(H_1)\cdot P(\text{defect}\mid H_1)}{P(\text{defect})}$$

Plugging in:

$$P(H_1\mid \mid \text{defect})==\frac{0.50\times 0.02}{0.027}==\frac{0.01}{0.027}\approx \approx 0.37$$

So about 37% of defective parts come from machine 1.

Why does this work? Bayes' formula answers: "What share of all defects comes from machine 1?". The numerator is machine 1's contribution to total defects. The denominator is total defects. Divide the two and the share falls out.

General Bayes' Formula

$$P(H_k\mid \mid A)==\frac{P(H_k)\cdot P(A\mid H_k)}{P(A)}$$

Remember the order:

• Numerator: P(hypothesis) × P(event | hypothesis)
• Denominator: P(event), calculated via the law of total probability

Example: Medical Test

A rare disease affects 1% of the population. A test exists:

• If a person is sick, the test shows "+" in 95% of cases (sensitivity)
• If a person is healthy, the test falsely shows "+" in 2% of cases (false positives)

The test showed "+". What's the probability you're actually sick?

Intuitive (wrong) answer: "Since the test is positive and its accuracy is 95%, the probability of disease is about 95%."

This is a trap! Let's apply Bayes' theorem.

Given:

• $P(\text{sick})=0.01$ (1% of population)
• $P(\text{healthy})=0.99$
• $P(+|\text{sick})=0.95$
• $P(+|\text{healthy})=0.02$

Step 1. Find P(+) using the Law of Total Probability:

$$P(+)==P(\text{sick})\cdot \cdot P(+\mid \mid \text{sick})++P(\text{healthy})\cdot \cdot P(+\mid \mid \text{healthy})$$ $$P(+)==0.01\cdot \cdot 0.95++0.99\cdot \cdot 0.02==0.0095++0.0198==0.0293$$

Step 2. Apply Bayes' theorem:

$$P(\text{sick}\mid \mid +)==\frac{P(\text{sick})\cdot P(+\mid \text{sick})}{P(+)}==\frac{0.01\cdot 0.95}{0.0293}\approx \approx 0.32$$

Answer: ~32%, not 95%!

Intuition: The disease is rare (only 1% are sick). Among all positive tests, most are false positives from healthy people (who make up 99%). Bayes' theorem accounts for the "base rate" of the event.

---

Independence of Events

Events $A$ and $B$ are independent if knowing about one doesn't change the probability of the other:

$$P(B\mid \mid A)==P(B)$$

This is equivalent to:

$$P(A\cap \cap B)==P(A)\cdot \cdot P(B)$$

Example

$P(A)=0.4$, $P(B)=0.5$, $P(A\cap B)=0.2$

Check: $P(A)\cdot P(B)=0.4\times 0.5=0.2$

Matches $P(A\cap B)$! So the events are independent.

Common Confusion

Mutually exclusive ≠ Independent!

• Mutually exclusive: can't happen together ($A\cap B=\varnothing$). Example: rolled 3 and rolled even (mutually exclusive).
• Independent: don't affect each other. Example: result of first roll and second roll (independent).

Mutually exclusive events are actually maximally dependent: if one happened, the other definitely didn't!

---

Common Mistakes

Mistake 1: Confusing the condition and event

$P(\text{vaccinated}|\text{sick})$ is NOT the same as $P(\text{sick}|\text{vaccinated})$.

First: "among the sick, what fraction is vaccinated". Second: "among the vaccinated, what fraction is sick".

Mistake 2: Wrong denominator

In $P(B|A)$ the denominator is the condition (what comes after the vertical bar).

$P(\text{ace}|\text{red})$: divide by number of reds, NOT aces.

Mistake 3: Confusing P(A|B) and P(B|A)

✗ Wrong: $P(\text{disease}|\text{positive test})=P(\text{positive test}|\text{disease})$

✓ Correct: These are completely different values!

• $P(+|\text{sick})=95\%$: test sensitivity
• $P(\text{sick}|+)\approx 32\%$: actual probability of disease

Bayes' formula specifically relates these quantities.

Mistake 4: Computing „oder“ without the subtraction

Before working out a union, check whether a single case can belong to both events. The example above puts the two possibilities side by side: two sectors exclude each other, „Sektor $C$“ and „Gast aus dem Ausland“ do not.

---

Practice with Tables

Many conditional probability problems are conveniently solved using contingency tables (Vierfeldertafel). This method allows you to work with concrete numbers instead of abstract probabilities.

For more on working with tables, see the topic "Vierfeldertafel".

---

Check Your Understanding

Your turn1 of 2

Two events satisfy $P(A)=0.3$, $P(B)=0.6$ and $P(A\cup B)=0.72$. Which statement about $A$ and $B$ is true?

A$A$ and $B$ are dependent and mutually exclusive.B$A$ and $B$ are independent and mutually exclusive.C$A$ and $B$ are dependent and overlap.D$A$ and $B$ are independent and overlap.

Check

Your turn2 of 2

A box holds 4 red and 6 blue pens. Two pens are drawn one after the other without replacement. Which expression gives the probability of drawing one red and one blue pen?

A$\frac{4}{10}\cdot \frac{6}{10}$B$2\cdot \frac{4}{10}\cdot \frac{6}{10}$C$\frac{4}{10}\cdot \frac{6}{9}$D$2\cdot \frac{4}{10}\cdot \frac{6}{9}$

Check

---

Formulas

Conditional probability:

$$P(B\mid \mid A)==\frac{P(A\cap B)}{P(A)}$$

Multiplication rule:

$$P(A\cap \cap B)==P(A)\cdot \cdot P(B\mid \mid A)$$

Addition rule:

$$P(A\cup \cup B)==P(A)++P(B)--P(A\cap \cap B)$$

Total probability:

$$P(A)==\sum _{i}P(H_i)\cdot \cdot P(A\mid \mid H_i)$$

Bayes' formula:

$$P(H_k\mid \mid A)==\frac{P(H_k)\cdot P(A\mid H_k)}{P(A)}$$

Independence:

$$P(A\cap \cap B)==P(A)\cdot \cdot P(B)$$