Numerical experiment · independent Gaussian tokens

Five finite-token approximations to one straight line

Each color uses its own i.i.d. sample of tokens Yi ∼ N(0, 1). Increase n to extend the same five samples, or resample to start again.

Tn(x) = ∑i=1n Yi eτxYi∑i=1n eτxYiT(x) = τx

Population τxDraw 1Draw 2Draw 3Draw 4Draw 5

Attention output

The black line is population attention τx. The five colored curves are empirical attention from independent finite token samples.

Faint bell: query density φ(x), on its own scale. Rug: tokens from draw 1 (only those in [−4, 4]).

Where the query-weighted squared error comes from

The curves show φ(x) times the squared difference between empirical attention and τx.

φ(x) |Tn(x) − τx|². Large errors at rare queries receive less weight; area under each curve contributes to the squared norm below. The vertical scale adapts.

Query-weighted L² error, one value per draw

Draw 1—

Draw 2—

Draw 3—

Draw 4—

Draw 5—

Finite-window quadrature: [∫−88 |Tn(x) − τx|² φ(x) dx]1/2, using composite Simpson quadrature with 256 intervals. These are five realized errors, not an expectation over token samples or a confidence bound.

Computing five independent draws…

Seeded Box–Muller sampling; no symmetric pairing, quantile placement, or fitted curves. Larger samples need not improve every query or every realized norm.