Monge and Kantorovich Optimal Transport

From matchings to probability measures, maps, relaxations, and Wasserstein distances.

Gabriel Peyré

CNRS and École normale supérieure, PSL Université

Optimal Transport for Machine Learners

Why compare distributions?

Modern machine learning repeatedly asks for distances between probability laws:

empirical data versus a model distribution,
source versus target domains,
generated samples versus real samples,
evolving particle clouds during training.

Optimal transport compares distributions by using the geometry of the ambient space.

source: \alpha,x,atarget: \beta,y,btransport: T,\pi,P

Transport turns distribution comparison into geometry.

The fitting problem

Data and model distributions. A model produces samples from \alpha_\theta, while observations come from a target law \beta. A geometric fitting objective has the form \min_\theta D(\textcolor{#c7372f}{\alpha_\theta},\textcolor{#226db4}{\beta}), where D is a discrepancy between probability measures.

In practice, both laws are often empirical: \textcolor{#c7372f}{\alpha_\theta^n} =\frac1n\sum_{i=1}^n\delta_{\textcolor{#c7372f}{x_i(\theta)}}, \qquad \textcolor{#226db4}{\widehat\beta_m} =\frac1m\sum_{j=1}^m\delta_{\textcolor{#226db4}{y_j}}. If x_i(\theta)=g_\theta(z_i) for latent samples z_i\sim\zeta, this is the push-forward model \textcolor{#c7372f}{\alpha_\theta}=(g_\theta)_\#\zeta, \qquad \min_\theta D((g_\theta)_\#\zeta,\textcolor{#226db4}{\beta}).

OT chooses D by asking how much effort is needed to move the mass of \alpha_\theta onto \beta.

Computational OT in practice

OT is not a single formula, but a computational toolbox.

Computational ecosystem. The same geometric principle gives sorting algorithms in one dimension, assignment algorithms for equal weights, linear programs for general discrete measures, semi-discrete solvers for continuous-to-discrete maps, and Sinkhorn iterations for differentiable large-scale losses.

This course uses the POT library and the numerical figures of the OT4ML book to keep these regimes visible throughout.

Roadmap: discrete Monge matching

1. Monge matching

2. Continuous Monge

3. Kantorovich relaxation

4. Metric structure

5. Applications

Finite Monge problem

Let x_1,\ldots,x_n and y_1,\ldots,y_n be two point clouds and let c_{i,j}=c(\textcolor{#c7372f}{x_i},\textcolor{#226db4}{y_j}).

Permutation matching. The finite Monge problem is \min_{\sigma \in \mathfrak S_n} \sum_{i=1}^n c_{i,\sigma(i)}. For c(\textcolor{#c7372f}{x},\textcolor{#226db4}{y})=\|\textcolor{#c7372f}{x}-\textcolor{#226db4}{y}\|^p, it matches points while paying a geometric displacement cost.

Equivalently, one minimizes \langle C,\textcolor{#7c5bbd}{P_\sigma}\rangle over permutation matrices (\textcolor{#7c5bbd}{P_\sigma})_{i,j}=\mathbf 1_{\{j=\sigma(i)\}}.

Quadratic assignment between two point clouds

Why this is hard

Combinatorial nature. The feasible set has n! permutations. A brute force search is impossible except for very small n.

For arbitrary cost matrices, this is the linear assignment problem. Classical methods include:

Hungarian algorithm,
auction algorithm,
linear programming and network simplex variants.

A matching is a permutation matrix.

The one-dimensional miracle

For convex costs on the real line, sorting solves the problem.

Monotone matching. If \textcolor{#c7372f}{x_1} \leq \cdots \leq \textcolor{#c7372f}{x_n} and \textcolor{#226db4}{y_1} \leq \cdots \leq \textcolor{#226db4}{y_n}, then for every convex increasing cost h(|\textcolor{#c7372f}{x}-\textcolor{#226db4}{y}|), \sigma(i)=i is an optimal assignment.

The proof follows from the crossing inequality: replacing two crossing segments by non-crossing ones decreases the cost.

One-dimensional quantile matching

Convex versus concave costs in 1D

One-dimensional assignments for convex and concave powers

Same ordered one-dimensional supports: convex p=2 matchings are monotone, while concave p=1/2 matchings create long crossing exchanges.

For c_p(\textcolor{#c7372f}{x},\textcolor{#226db4}{y})=|\textcolor{#c7372f}{x}-\textcolor{#226db4}{y}|^p:

p>1: convexity gives the monotone rearrangement;
p=1: non-uniqueness can appear;
0<p<1: concavity favors different alternating-chain patterns.

The line is special because order is global. Convex costs are solved by sorting; concave costs still exploit the one-dimensional order, but through recursive local matching indicators rather than monotone pairing.

Live panel: one-dimensional costs

Use the controls to compare how the exponent in |\textcolor{#c7372f}{x}-\textcolor{#226db4}{y}|^p changes the matching geometry.