Probability Measures

Recall that a probability space is a triplet $(\Omega, \cF, \Pr)$. We already talked about the sample space $\Omega$ and the $\sigma$-Field $\cF$ in great detail. Here we talk about the probability measure $\Pr$.

Probability Axioms

Definition: Probability measure

A set function $\Pr$ on a $\sigma$-Field $\cF$ is a probability measure if it follows these probability axioms:

1. $0\le \Pr(A)\le 1$ for all $A\in\cF$,
2. $\Pr(\emptyset)=0$ and $\Pr(\Omega)=1$, and
3. For a sequence of events $\{A_i\}_{i\ge 1}$ that are disjoint sets in $\cF$,

$$\Pr(\cup_i A_i) = \sum_i \Pr(A_i).$$

Discrete probability space

As an example, let $\Omega$ be a set that is at most countable and $\cF$ be a $\sigma$-Field on it. Let $p$ be a function on $\Omega$ such that $p(\omega)\ge 0$ for all $\omega\in\Omega$ and $\sum_{\omega\in\Omega}p(\omega)=1$.

Then we can define a probability measure $\Pr(A)=\sum_{\omega\in A}p(\omega)$ for all $A\in \cF$.

In this case, $(\Omega, \cF, \Pr)$ is called a discrete probability space since $\Omega$ is at most countable. Note here that the function $p$ itself is not the measure, it is not a set function. The measure function is $\Pr$, which gives us a way to measure the content in a measurable set. What do we mean by a measurable set? It simply means an event; recall that any set in the $\sigma$-Field is an event.

In simpler words, a probability measure assigns a number to all events, and in that sense, lets us measure those events.

Additional properties

These can be verified by using the probability axioms above.

More properties of probability measures

Continuity

When we say $A_n\uparrow A$, it means that $\{A_n\}$ is a non-decreasing and converging sequence of events; i.e., for each $i\ge 1$, $A_i\subseteq A_{i+1}$ with $\lim_{n\to\infty}A_n=A$. Similarly, $A_n\downarrow A$ denotes a non-increasing sequence of events converging to $A$.

For a sequence of events $\{A_n\}$,

$$A_n \uparrow A \text{ or } A_n\downarrow A \implies \lim_{n\to\infty}\Pr(A_n)=\Pr(A).$$

Boole's identity

Also called the union bound. For a set of events $A_1,A_2,\ldots\in\cF$,

$$\Pr(\cup_i A_i)\le \sum_i\Pr(A_i).$$

Exercise 4

Problem

Prove the continuity property of probability measures described above.

Hint

Use countable additivity (axiom 3) by constructing a sequence of disjoint sets from $\{A_n\}$.