Intuitive RL Series : Contraction Operators

Fri, 11 Sep 2026 00:00:00 +0000

If you are teaching or learning RL and have ever wished an idea showed up as a picture first, this series is for you. I am trying to communicate science the way I always wanted it communicated to me: precision and clarity, with as few words as I can get away with (written or spoken), as much visual structure as I can find, and a clear sense of where each stepping stone sits in the larger picture. I recognize this is not a universal preference, and I am targeting a specific kind of reader. This first post is intentionally simple so I can learn, from your feedback, what actually works for people who share my communication preferences. Main ideas from Reinforcement Learning: Foundations [1]. Videos use Manim [2], 3Blue1Brown’s animation tool.

Let’s start!

Click 💡 whenever you want more intuition.

Contraction operators keep showing up in RL proofs. They are a main tool for reasoning about convergence: if we keep applying the same update, do we settle on a solution?

Contractions are defined using norms.

A norm is a function $\|\cdot\| : \mathbb{R}^d \to [0, \infty)$ such that $\forall\, x, y \in \mathbb{R}^d,\; \forall\, a \in \mathbb{R}$

$\|ax\| = |a| \cdot \|x\|$

💡

Scaling a vector by $a$ scales its size by $|a|$.
$\|x + y\| \le \|x\| + \|y\|$

💡

The size of a sum is at most the sum of the sizes.
$\|x\| = 0 \implies x = 0$

💡

Size zero means the zero vector.

Examples:

the $p$-norm $\|x\|_p = \big(\sum_i |x_i|^p\big)^{1/p}$ for $p \ge 1$
- Euclidean: $p = 2$
the max-norm $\|x\|_\infty = \max_i |x_i|$

An operator is a function between spaces. Here, real vector spaces.

$$ T : \mathbb{R}^d \to \mathbb{R}^d, \qquad T(v) \in \mathbb{R}^d $$

Applied $n$ times:

$$ T^n(v) = T\big(T^{n-1}(v)\big), \qquad n \ge 2, $$

$T$ is a contraction operator (or specifically a $\boldsymbol{\beta}$-contraction operator) w.r.t. $\|\cdot\|$ if

$$ \forall\, v_1, v_2 \in \mathbb{R}^d \qquad \|T(v_1) - T(v_2)\| \le \beta \, \|v_1 - v_2\|. $$ $$ \text{where } \beta \text{ is } \in (0, 1) $$

References

[1] S. Mannor, Y. Mansour, and A. Tamar, Reinforcement Learning: Foundations. Cambridge, U.K.: Cambridge University Press, 2026. [Online]. Available: https://sites.google.com/view/rlfoundations/home

[2] G. Sanderson, Manim, 3Blue1Brown. [Online]. Available: https://github.com/3b1b/manim

pedagogical | Esra'a Saleh

Intuitive RL Series : Contraction Operators

References