Axioms and Rules

Axioms and Rules of Probability

For each event, A, in S, the probability of event A, P(A), is a measure of the likelihood that the event will occur. Axioms are statements that are accepted without proof. The provide the foundation for the probability rules that follow.

NOTATION: P(A) denotes the probability of event A.

The Axioms of Probability

Axiom 1 For any event A, $P(A)\geq 0$.
Axiom 2 P(S) = 1.
Axiom 3 If $A_1, A_2, \ldots $ are an infinite collection of mutually exclusive (disjoint) events, then $\small{P(A_1\cup A_2\cup \ldots) = \sum_{i=1}^{\infty}P(A_i)}$.

It follows from these axioms that $P(\emptyset)=0$. This enables us to find the probability of the union of a finite collection of mutually exclusive events.

If $A_1, A_2, \ldots A_n$ are a finite collection of mutually exclusive events, then $P(A_1\cup A_2\cup \ldots \cup A_n) = \sum_{i=1}^{n}P(A_i)$.

Basic Rules of Probability

Let $o_1, o_2, \ldots o_n$ be the outcomes in S and let $p_1, p_2, \ldots p_n$ indicate the corresponding probabilities. That is, $P(o_i)=p_i$. Then

$0 \le p_i \le 1$
$\sum_{i=1}^np_i = p_1 + p_2 + \ldots + p_n = 1$.

Consider: What would it mean for a probability to be less than 0? More than 1?

Example: An experiment has 4 outcomes, $o_1, o_2, o_3,$ and $o_4$. If $p_1=0.2$, $p_2=0.2$, and $p_3=0.4$ what is $P_4$?

The sum of all the probabilities must be 1. So
$$\small{\begin{array}{rcl} p_1+p_2+p_3+p_4 & = & 1\\ 0.2+0.2+0.4+p_4 & = & 1\\ p_4 & = & 1-(0.2+0.2+0.4)\\ p_4 & = & 0.2\end{array}}$$

$p_3=2p_4$
$p_2=2p_3=4p_4$
$p_1=2p_2=4p_3=8p_4$

$$\begin{array}{rcl} p_1+p_2+p_3+p_4 & = & 1\\ 8p_4+4p_4+2p_4+p_4 & = & 1\\ 15p_4 & = & 1\\ p_4 & = & \frac{1}{15}\end{array}$$ Thus $p_4=\frac{1}{15}$
$p_3=\frac{2}{15}$
$p_2=\frac{4}{15}$
$p_1=\frac{8}{15}$.

Event A occurs if any of the outcomes in A occurs. Thus the probability of event A, P(A), is the sum of the probabilities of the outcomes in A.

Example: Let A be the event that the outcome of a die roll is even. What is P(A)?
$P(A) = P(2)+P(4)+P(6) = \frac{1}{6}+\frac{1}{6}+\frac{1}{6} = \frac{1}{2}$

The Complement Rule

According to the World Health Organization (WHO), globally, about 25% of people experience mental or neurological disorders some time in their lives.

If a person is randomly selected, either they will experience such an illness during their lifetime (event A) or they will not (event A'). These two events compose the entire sample space. That is S = A ∪ A'
Since A and A' are mutually exclusive, P(A ∪ A') = P(A) + P(A'). $$\small{\begin{array}{rcl} P(A) + P(A') & = & P(A \cup A')\\ & = & P(S)\\ & = & 1\end{array}}$$ Thus P(A) = 1- P(A')

Since about 25% of people experience mental or neurological disorders at some time in their lives, P(A) = 0.25 and the probability that a randomly selected person will not suffer from a mental disorder at some time in their lives is P(A') = 1-0.25 = 0.75. In other words, 75% of people in the world will not suffer suffer from one of these disorders during their lifetime.

A visual representation shows the complement rule in probability: ( P(A) = P(S) - P(A') ). On the left, a rectangle labeled S contains a circle labeled A, with the circle shaded green to represent ( P(A) ). In the middle, the entire rectangle S is shaded green to represent ( P(S) ), the total probability space. On the right, the rectangle S remains shaded, but the circle A is left unshaded, illustrating ( P(A') ), the complement of A. This demonstrates that the probability of A equals the total probability minus the probability of not A.

The Complement Rule
P(A) = 1- P(A')

When discussing probabilities, these expressions are equivalent:

The probability that a randomly chosen member from the population has a certain quality is $p$.
The proportion of members in the population that have a certain quality is $p$.
$p$% of the members of the population that have a certain quality.

The example above could be equivalently stated as the proportion of people who experience mental or neurological disorders at some time in their lives is 0.25.

With a question phrased in terms of a percentage, the complement rule looks a little different. To find the percentage associated with the complement, we subtract the given percentage from 100% rather than from 1. Thus the percentage of American women who do not say they have been sexually harassed is 100%-59% = 41%.

It is likely that some survey respondents would say they were unsure. Thus the sample space cannot be divided into those who say they have and those who say they have not been sexually harassed.

The complement rule can make complex probabilities easier to find.

Example: What is the probability of event A: at least one head is observed in three coin tosses?

A = {HHH, THH, HTH, HHT, TTH, THT, HTT}.

To find P(A) directly, find the probabilit of each of the outcomes within A. Since these are mutually exclusive, the probabilities can be summed to find P(A). $$\small{\begin{array}{rcl}P(A) & = & P(HHH) + P(THH) + P(HTH) + P(HHT) + P(TTH) + P(THT) + P(HTT) \\ & = & 1/8+1/8+1/8+1/8+1/8+1/8+1/8 \\ & = & 7/8. \end{array}}$$

A' = {TTT}, the event that no heads are observed in three coin tosses and P(A') = 1/8. $$\small{\begin{array}{rcl}P(A) & = & 1-1/8 \\ & = & 7/8. \end{array}}$$

The Addition Rule

Anxiety and depression are two of the most common mental disorders. The National Alliance on Mental Illness (NAMI) reported that, in the US, about 19% of adults suffer from anxiety and 8% experience depression. Thus the probability that a randomly selected US adult suffers from anxiety is about 0.19 and the probability that they have depression is about 0.08. What is the probability that a randomly selected US adult suffers from both anxiety and depression?

It might seem reasonable to add the probababilities together, 0.19 + 0.08 = 0.27, however, this result is too high. In fact, it is estimated that 21% of all US adults has any mental illness in a given 12 month period. Some people suffer from both anxiety and depression so if we simply add the probabilities of the two events, these people get double counted. To arrive at the correct figure, subtract the probability that a person suffers from both depression and anxiety.

To find the probability of a union, sum the probabilities associated with the two events individually and subtract the probability of the intersection. This is the Addition Rule.

The Addition Rule
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

$A probability diagram illustrates the addition rule: ( P(A \cup B) = P(A) + P(B) - P(A \cap B) ). On the left, a rectangle contains two overlapping circles labeled A and B, with the entire union shaded blue to represent ( P(A \cup B) ). In the next panel, only circle A is shaded, representing ( P(A) ). An arrow adds circle B, fully shaded, to represent adding ( P(B) ). In the third panel, the overlap between A and B is highlighted in darker blue to show that this shared area has been counted twice. A second arrow removes the overlap, ( P(A \cap B) ). The final panel shows the correct union of A and B shaded in blue, representing the total probability of ( P(A \cup B) ) without double-counting the intersection.$

Example: Let A denote the event that a US adult suffers from anxiety, P(A) = 0.19 and let D denote the event that a person suffers from depression, P(D) = 0.08. Estimates of the percentage of US adults who suffer from aniety and depression vary. Suppose P(A ∩ D) = 0.07. We can use the addition rule to find the probability that a randomly selected US adult suffers from anxiety or depression (or both!).

$$\small{\begin{array}{rcl} P(A ∪ D) & = & P(A) + P(D) - P(A ∩ D) \\ & = & 0.19 + 0.08 - 0.07 \\ & = & 0.20. \end{array}}$$ About 20% of US adults suffer from anxiety or depression in a given 12-month.

Let event A denote that a random woman in a California college computer science class said their father worked in a computing field, and let event B denote that they said their mother worked in a computing field.

Then P(A) = 0.21, and P(B) = 0.13, and P(A ∩ B) = 0.03.
P(A∪B) = 0.21+0.13-0.03 = 0.31.

31% of women in the survey had at least one parent who worked in a computing field.

Let event A denote the event that a man's father worked in a computing field, and let event B denote that their mother worked in a computing field.

P(A ∪ B) = 0.24, P(A) = 0.16, and P(A ∩ B) = 0.02.

$$\small{\begin{array}{rcl} P(A \cup B) & = & P(A) + P(B) - P(A \cap B) \\ P(B) & = & P(A \cup B) + P(A \cap B) - P(A) \\ & = & 0.24 + 0.02 - 0.16\\ & = & 0.1\end{array}}$$
10% of men said that their mother worked in a computing field.

If A and B are mutually exclusive P(A ∩ B) = 0, thus $$\small{\begin{array}{rcl} P(A ∪ B)& = & P(A) + P(B) - P(A \cap B)\\ & = & P(A) + P(B) - 0\\ & = & P(A) + P(B)\end{array}}$$