Intuitive RL Series : Contraction Operators

Last updated on Sep 12, 2026 Intuitive RL Series

If you are teaching or learning RL and have ever wished an idea showed up as a picture first, this series is for you. I am trying to communicate science the way I always wanted it communicated to me: precision and clarity, with as few words as I can get away with (written or spoken), as much visual structure as I can find, and a clear sense of where each stepping stone sits in the larger picture. I recognize this is not a universal preference, and I am targeting a specific kind of reader. This first post is intentionally simple so I can learn, from your feedback, what actually works for people who share my communication preferences. Main ideas from Reinforcement Learning: Foundations [1]. Videos use Manim [2], 3Blue1Brown’s animation tool.

Let’s start!

Click 💡 whenever you want more intuition.

Contraction operators keep showing up in RL proofs. They are a main tool for reasoning about convergence: if we keep applying the same update, do we settle on a solution?

Contractions are defined using norms.

A norm is a function $\|\cdot\| : \mathbb{R}^d \to [0, \infty)$ such that $\forall\, x, y \in \mathbb{R}^d,\; \forall\, a \in \mathbb{R}$

  1. $\|ax\| = |a| \cdot \|x\|$
    💡

    Scaling a vector by $a$ scales its size by $|a|$.

    OKnot OKx−2xx−2x
  2. $\|x + y\| \le \|x\| + \|y\|$
    💡

    The size of a sum is at most the sum of the sizes.

    OKnot OKxyx+yxyx+y
  3. $\|x\| = 0 \implies x = 0$
    💡

    Size zero means the zero vector.

    OKnot OK‖0‖ = 000‖x‖ = 0x ≠ 0

Examples:

  • the $p$-norm $\|x\|_p = \big(\sum_i |x_i|^p\big)^{1/p}$ for $p \ge 1$
    • Euclidean: $p = 2$
  • the max-norm $\|x\|_\infty = \max_i |x_i|$

An operator is a function between spaces. Here, real vector spaces.

$$ T : \mathbb{R}^d \to \mathbb{R}^d, \qquad T(v) \in \mathbb{R}^d $$

Applied $n$ times:

$$ T^n(v) = T\big(T^{n-1}(v)\big), \qquad n \ge 2, $$

$T$ is a contraction operator (or specifically a $\boldsymbol{\beta}$-contraction operator) w.r.t. $\|\cdot\|$ if

$$ \forall\, v_1, v_2 \in \mathbb{R}^d \qquad \|T(v_1) - T(v_2)\| \le \beta \, \|v_1 - v_2\|. $$ $$ \text{where } \beta \text{ is } \in (0, 1) $$

References

[1] S. Mannor, Y. Mansour, and A. Tamar, Reinforcement Learning: Foundations. Cambridge, U.K.: Cambridge University Press, 2026. [Online]. Available: https://sites.google.com/view/rlfoundations/home

[2] G. Sanderson, Manim, 3Blue1Brown. [Online]. Available: https://github.com/3b1b/manim

Intuitive RL series Reinforcement Learning pedagogical
Esra'a Saleh
Esra'a Saleh