Distribution

Let $X$ be a random variable. Then $X$ induces a probability measure $\mu$ on $(\mathbb{R}, \mathcal{B})$ (where $\mathcal{B}$ is the Borel $\sigma$-field) by setting:

$$\mu(B) = \mathbb{P}(X \in B) \quad \text{for all } B \in \mathcal{B}$$

Exercise 5

Check that $\mu$ is a probability measure.

$\mu$ is called the distribution of $X$.

Distribution function

Definition: Distribution Function

The function $F$ given by

$$F(x) = \mathbb{P}(X \le x)$$

is called the distribution function of $X$.

Note

The distribution is fully characterized by $F$.

$$F(x) = \mu((-\infty, x])$$

Since the collection $\{(-\infty, x], x \in \mathbb{R}\}$ is a $\pi$-system and it generates $\mathcal{B}$ (refer to how $\mathcal{B}$ was generated), it follows that $F(x)$ uniquely determines $\mu$.

We learned here that we don’t need a function defined on all sets in $\mathcal{B}$, as $\mathcal{B}$ is quite large. We only need to define it on sets of the form $(-\infty, b]$ for $b \in \mathbb{R}$.

Properties of distribution function

1. $F$ is non-decreasing.
2. $\lim_{x \to \infty} F(x) = 1, \quad \lim_{x \to -\infty} F(x) = 0$
3. $F$ is right continuous: $\lim_{y \downarrow x} F(y) = F(x) = \mathbb{P}(X \le x)$
4. $\lim_{y \uparrow x} F(y) = \mathbb{P}(X < x) = F(x-)$ (notation)
5. $\mathbb{P}(X=x) = F(x) - F(x-)$

Proofs

1: Monotonicity

Monotonicity of probability:

$$\begin{aligned} x_1 \le x_2 &\implies (-\infty, x_1] \subseteq (-\infty, x_2] \\ &\implies \mathbb{P}(X \in (-\infty, x_1]) \le \mathbb{P}(X \in (-\infty, x_2]) \\ &\implies F(x_1) \le F(x_2) \end{aligned}$$

2: Limits

Continuity from above or below of probability:

$$\begin{aligned} \lim_{x \to \infty} (-\infty, x] = \mathbb{R} &\implies \lim_{x \to \infty} \mathbb{P}(X \le x) = \mathbb{P}(X \in \mathbb{R}) = 1 \\ \lim_{x \to -\infty} (-\infty, x] = \emptyset &\implies \mathbb{P}(X \in \emptyset) = 0 \end{aligned}$$

3: Right Continuity

Continuity from above:

$$\lim_{y \downarrow x} F(y) = \lim_{y \downarrow x} \mathbb{P}(X \le y)$$

As $y \downarrow x$, $\{X \le y\}$ is a decreasing sequence of events, and has limit:

$$\lim_{y \downarrow x} \{X \le y\} = \bigcap_{y \downarrow x} \{X \le y\} = \{X \le x\}$$$$\implies \lim_{y \downarrow x} \mathbb{P}(X \le y) = \mathbb{P}(X \le x) = F(x)$$

4: Left Limit

Similar to 3, note that:

$$\lim_{y \uparrow x} \{X \le y\} = \bigcup_{y \uparrow x} \{X \le y\} = \{X < x\}$$$$\implies \lim_{y \uparrow x} \mathbb{P}(X \le y) = \mathbb{P}(X < x)$$

5: Jump

Intuitively, this property tells us that the probability of $X$ taking exactly the value $x$ is equal to the size of the “jump” in the distribution function at $x$.

To prove this, we can simply take the difference of the results from Property 3 ($F(x)$) and Property 4 ($F(x-)$).

Formally, note that $\{X = x\} = \{X \le x\} \setminus \{X < x\}$. Since $\{X < x\} \subseteq \{X \le x\}$, we have:

$$\begin{aligned} \mathbb{P}(X=x) &= \mathbb{P}(X \le x) - \mathbb{P}(X < x) \\ &= F(x) - F(x-) \end{aligned}$$

Existence of random variable with given distribution

Theorem: Characterization of Distribution Functions

Statement

If a function $F$ satisfies properties 1-3 above, then there exists a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ and a random variable $X$ on it such that $F$ is the distribution function of $X$.

$$\implies \text{properties 1-3 fully characterize distribution functions.}$$

Proof

1. Construct the Probability Space

   Take $\Omega = (0, 1)$, $\mathcal{F} = \mathcal{B}$ (the Borel $\sigma$-field on $(0, 1)$), and let $\mathbb{P}$ be the Lebesgue measure $\lambda$ restricted to $(0, 1)$.

2. Define the Random Variable

   For $\omega \in (0, 1)$, define $X(\omega)$ as:

   $$X(\omega) = \sup \{ y : F(y) < \omega \}$$

   This is denoted as $F^{-1}(\omega)$ and is called the generalized inverse function.

   Note: If $F$ is strictly increasing, this is just the ordinary inverse. If $F$ has “flat” regions or jumps, this definition handles them.

   Continuous Case

   

   Finding $x$ such that $F(x) = \omega$ is straightforward.

   Discontinuous / Jump Case

   

   A jump in $F(x)$ at $x_0$ means an interval of $\omega$ values map to the same $x_0$.

3. Analyze the condition $\omega \le F(x)$

   For any $\omega \in (0, 1)$ and $x \in \mathbb{R}$: If $\omega \le F(x)$, then for any $y$ such that $F(y) < \omega$, we must have $y \le x$ (otherwise if $y > x$, then by monotonicity $F(y) \ge F(x) \ge \omega$, which is a contradiction).

   This implies that $x$ is an upper bound for the set $\{ y : F(y) < \omega \}$. Since $X(\omega)$ is the supremum of this set, we get:

   $$\omega \le F(x) \implies X(\omega) \le x$$

4. Analyze the condition $\omega > F(x)$

   On the other hand, if $\omega > F(x)$, then since $F$ is right continuous, we can find an $x' > x$ such that we still have $\omega > F(x')$.

   Since $F(x') < \omega$, the value $x'$ belongs to the set $\{ y : F(y) < \omega \}$. Recall $X(\omega) = \sup \{ y : F(y) < \omega \}$. Thus:

   $$X(\omega) \ge x' > x$$

   So we have:

   $$\omega > F(x) \implies X(\omega) > x$$

5. Verify the Distribution

   Combining steps 3 and 4, we have the equivalence:

   $$\{ \omega : X(\omega) \le x \} = \{ \omega : \omega \le F(x) \}$$

   Now we calculate the probability:

   $$\begin{aligned} \mathbb{P}(X \le x) &= \mathbb{P}(\{ \omega \in (0, 1) : X(\omega) \le x \}) \\ &= \mathbb{P}(\{ \omega \in (0, 1) : \omega \le F(x) \}) \\ &= \lambda((0, F(x)] \cap (0, 1)) \end{aligned}$$

   Since $0 \le F(x) \le 1$ (by property 2), the set is simply $(0, F(x)]$.

   $$\lambda((0, F(x)]) = F(x) - 0 = F(x)$$

   Thus, $X$ has distribution function $F$.