Independence

In solving the urn problem, we stumbled across the concept of independence. This will be important to us and we need to spend a few minutes on it. Consider again the urn problem:

Urn Problem: Suppose we have an urn with 30 blue balls and 50 red balls in it and that these balls are identical except for color. Suppose further the balls are well mixed and that we draw 3 balls, without replacement. Determine the probability that the balls are all of the same color.

Recall the final tree diagram given in Figure 2.3. The probabilities on the branches are called conditional probabilities . For example:

1.

Let B₂ denote the event that the second ball is blue and

2.

let A₁ denote the event that the first ball is blue.

Then the probability on the first step upward branch is the probability that B₂ occurs given that A₁ has occurred; i.e, 29/79. This is called the conditional probability of B₂ given A₁ and we will denote it by P(B₂ | A₁). The bar is pronounced "given".

In general for two events A and B, if P(B | A) = P(B), i.e, knowledge of A did not change the predicition of B, then we say A and B are independent events .

Note for the urn problem, if, as above, B₂ denotes the event that the second ball is blue and A₁ denotes the event that the first ball is blue then P(B₂ | A₁) = 29/79. What is the P(B₂)? All B₂ says is that the second ball drawn is blue. So to determine the P(B₂) go the the final tree diagram and look at all the end nodes for which the second ball is blue. Then add up all the probabilities associated with these end nodes. Try it. Now add up the probabilities. You get 30/80, the same as the P(A₁). Surprised? This may be counterintuitive, but what is so special about the first ball over the second ball? Nothing. Okay, what is the probability that the third ball is blue? It's 30/80. If you don't believe me go to the tree diagram and add up the probabilities on the final nodes associated with a blue third ball. If we continued with this urn game and drew all 80 balls without replacement, what is the probability that the eightieth ball is blue? YOU GUESSED IT, 30/80.

Before we forget it, for the urn problem we showed that P(B₂ | A₁) = 29/79 and P(B₂) = 30/80; hence, A₁ and B₂ are not independent events. We say that they are dependent events.

Lets return to the urn problem once again. Suppose we do the sampling with replacement . That is we remove a ball, record its color, put it back in the urn, mix the balls well, and then remove the next ball. We do this until we get 3 balls. Now what's P(B₂ | A₁), i.e, the probability that the second ball is blue given the first ball is blue. In this case, sampling with replacement, it's 30/80, because the first ball is replaced and, hence, the contents of the urn are the same on the second draw as they were on the first draw. Thus the events A₁ and B₂ are independent events in the case of sampling with replacement.

Actually we can get a neat formula out of conditional probability that will be useful from time to time and it will also give us a way to use independence.

Let A and B be arbitrary events. We want to determine P(B | A). Suppose that the tree diagram is too complicated. But we can use relative frequency. So we repeat the experiment many times, say, N. Now of these times we only want the times that A occurs $\char93 (A)$, (because we want the probability of B given that A has occurred).

Now of the times that A has occurred, count those times that B has occurred. What have you counted in this last count? Wait! Whatever this count is, lets denote it by "Last Count". I claim that

$$P(B \vert A)\textrm{, is approximately } \frac{\textrm{\it ''Last Count''}}{\char93 (A)}.$$

But what have we counted in getting "Last Count".? Say it! That's right! We have counted the number of times both A and B occurred simultaneously, i.e, #(A and B). Hence

$$P(B \vert A)\textrm{, is approximately } \frac{\char93 (A \textrm{ and } B)}{\char93 (A)}$$

But I won't change anything by dividing both the numerator and denominator of this fraction by N, the number of times that we repeated the experiment. Hence

$$P(B \vert A)\textrm{, is approximately } \frac{\char93 (A \textrm{ and }B)/N}{\char93 (A)/N}.$$

As N gets large this last fraction gets close to $\frac{P(A \textrm{ and }B)}{P(A)}$. Thus we define the conditional probability of B given A as

$$P(B \vert A) = \frac{P(A \textrm{ and }B)}{P(A)}.$$

Lets rewrite it as a formula for P(A and B), which is

P(A and B) = P(B | A)P(A)

This is called the multiplicative law . Finally, if A and B are independent events we get

P(A and B) = P(B)P(A)

If we can recognize independence then we can use this formula to compute P(A and B). As an example, consider the following.

Jet Example : A jet airplane has 3 engines which function independently of one another. The probability that an engine fails in flight is .0001. Furthermore, the plane can fly if at least one engine is functioning. Determine the probability that the airplane has a successful flight.

The event we want to consider is A = at least one engine operates throughout the flight. Consider the complement of A, A^c which is the event all three engines fail.

1.

Let B₁ be the event that engine one fails.

2.

Let B₂ be the event that engine two fails.

3.

Let B₃ be the event that engine three fails.

Hence, A^c is the event B₁ and B₂ and B₃ occurs. Thus

P(A) = 1 - P(A^c) = 1 - P(B₁ and B₂ and B₃)

It seems that the engines function independently of one another; hence, B₁, B₂, and B₃ are independent events. So

$$P(B_1 \textrm{ and } B_2 \textrm{ and } B_3) =.0001 \times .0001 \times .0001 = .00000000001.$$

Hence P(A) = .999999999999.

Exercise 3.6.1  

1.

Suppose we flip a fair coin 4 times. What's the probability of 4 heads?

2.

A basketball player free throw percentage is .70. If he shoots 6 free throws, find the probability that he makes all 6. What are we assuming here that may not be true?

3.

Suppose in the Jet airplane example, that one engine is broken before takeoff, but the plane takes off anyway. Determine the probability that the plane arrives safely.

4.

Suppose A and B cannot occur at the same time. Are they independent?

5.

In a call in poll, are the calls independent of one another?

6.

Suppose the subjects in a poll are selected by random phone calling. Are the calls independent of one another?

7.

A newspaper reporter goes out to the mall and asks people a question of local interest. Are these respondees independent of one another?