Introduction

This section is about measure theoretic probability and some related topics. Most of the technical content is inspired from a course I took at UWaterloo, given by Yi Shen and Durrett’s popular book Probability: Theory and Examples, with some additional tidbits from my personal experience that helped me build a good intuition on the subject.

The whole thing might look a bit abstract at first and it may seem unnecessary to go through all the trouble. When you feel overwhelmed, just remind yourself that it’s just a rigorous framework that formalizes a lot of trivial intuitions, so that the intuitions we build are correct. Without rigor, it is easy to make mistakes and build highly convincing yet false intuitions.

We will start from the basics, formally introducing even the very fundamental things like “events”, “sample space”, “probability space”, “random variables”, etc. With these rigorous definitions aligned with our intuitions, when we later move on to more sophisticated topics, we will be able to witness the true power of rigor that measure theoretic probability brings to the table.

Prerequisites

1. Real analysis:
   • Sequences & series: Convergence, Cauchy sequences, subsequences.
   • Limits & continuity: Epsilon-delta definitions, uniform continuity, properties of continuous functions.
   • Metric spaces: Distance, open/closed balls, completeness.
   • Integration: Riemann integration and its limitations (leading to Lebesgue).
2. Set theory:
   • Union, intersection, complement, De Morgan’s laws, Cartesian products.
   • Power sets & collections of sets: Working with sets of sets.
   • Axiom of choice: Understanding its implications.
3. General topology (helpful but not absolutely necessary):
   • Topological spaces: Open sets, closed sets, neighborhoods.
   • Continuity in topological spaces.

What is Measure Theory?

Measure theory is a mathematical framework that helps us understand what measuring the “size” of subsets of a certain set means. In some sense, it is a formal generalization of geometrical measures (e.g. length for 1-dimensional, area for 2-dimensional, and volume for higher dimensional objects).

Motivation: Why do we need Measure Theory?

To understand why Measure Theory exists, we first have to realize that the Riemann integration we learned in high school has a “blind spot”. It works great for a certain class of functions, but fails when functions get too “jittery” or “broken”.

An analogy

Think about the size of the interval $[0, 1]$ on the real number line. One might intuitively say that the size is $1-0 = 1$. This intuition is correct if it is formalized, and it is easy to see that using that formalization, one can compute the size of any interval $[a, b]$, which is $b-a$.

Note that these intervals are one of the “simplest” subsets of the set of real numbers $\R$. But, there can be subsets of $\R$ that are much more complex. For example, how would you compute the “size” of a collection of scattered points? Or the size of all the irrational numbers between $0$ and $1$? Or what about the size of the Cantor ternary set?

Measure Theory is the rigorous framework for assigning a “size” to subsets of a space in a way that remains consistent even when those subsets are highly complex and incredibly hard to describe. We will see many examples of such subsets later. I promise this will become more intuitive if you keep reading and be patient (especially after going through the Lebesgue measure).

Relation to Probability Theory

After measure theory was developed in the early 20th century, Soviet mathematician Andrey Kolmogorov discovered that it can also be used to provide a rigorous foundation to probability.

While probability involves some extra ideas (relative to the rest of mathematics), the issue was whether it required a new technical ingredient to be added to the rest of mathematics. Kolmogorov’s achievement was the realization that it didn’t. With agreed axioms, mathematicians systematically developed theorem-proof probability. The connection to the rest of theorem-proof mathematics enabled the use of other mathematical tools, especially the limit theorems.