Edit the sentence, then tap a token to make it the query
These words have hand-built meaning vectors, so their attention is intentional. Tap one to add it. Any other word still works, it just gets a neutral placeholder vector (shown with a dashed outline).
key token
Q·K
÷√d
softmax
Attention(Q,K,V) = softmax(Q Kᵀ / √dₖ) V
Semantic head
Positional head
Not in word bank
Line thickness and brightness scale with the attention weight: the share of "attention" the query token pays to each other token. Nothing here calls a real model. Words in the word bank carry small hand-built meaning vectors so their attention is intentional; any other word gets a neutral placeholder vector, so its semantic arcs are not meaningful (the positional head still works on it). The arithmetic is the same shape a real transformer runs, shrunk to numbers you can read.