How attention works: search, feeds, and LLMs
Last updated: July 17, 2026
The interactive below shows how attention works in three different systems: a search engine ranking pages, a social feed ranking posts, and a language model attending over words. Switch tabs to move between them, and open the compare tab to line them up side by side. Click the fullscreen button in the top right of the demo for a closer look.
Demo
Attention across three systems
One interactive that runs the same score, normalize, allocate loop through search PageRank, a social feed, and LLM self-attention, so you can see exactly how they differ.
What does the demo show?
The demo has five tabs. The pattern is the shared skeleton, one diagram you can morph between the three systems. Search runs real PageRank on a small web of pages. Social runs a live feed that sorts posts by predicted engagement. Attention runs real scaled dot-product attention over a sentence. Compare lines all three up against the five differences that matter.
A few things to try:
- Add a link in the search tab. Click one page, then another, to draw a link between them. The ranking recomputes and the page you pointed at climbs. Authority is conferred by who points at you.
- Let the social feed run. Watch the sensational posts climb while the careful ones sink, then press reveal true quality to see the gap between what the feed surfaces and what is actually good.
- Swap big for small in the attention tab. The sentence is "the trophy did not fit in the suitcase because it was too big." Toggle the predicate and watch what the word "it" points to flip between the trophy and the suitcase.
- Toggle the attention heads. One head tracks meaning, another tracks position. Real models run many at once, which is .
What do search, social, and LLMs have in common?
Underneath the different interfaces, all three run the same three steps.
- Score. Give every candidate a number for how relevant it is. Pages get a PageRank score from their inbound links. Posts get a predicted engagement score from their features. Tokens get a score from the query dotted with each key.
- Normalize. Turn the raw scores into comparable weights. Search and the feed sort them into a ranking. Attention runs a , which squashes the scores into weights that are all positive and sum to 1.
- Allocate. Spend a scarce resource by those weights. Search spends slots on the results page. The feed spends impressions. Attention spends its output on a weighted blend of value vectors.
Routing inside a mixture-of-experts model is the same move again: a gate scores the experts, a softmax normalizes, and each token is sent to the top few. If you have seen how mixture-of-experts routing works, this pattern will feel familiar.
How does attention differ across the three systems?
The steps are shared. The differences are in what gets scored, what objective the score serves, how often it is recomputed, and what the output looks like. Five axes separate them, and the compare tab lays them out in full.
Search: authority, scored recursively
Search scores pages by authority. A page is important if important pages link to it, which is circular on purpose. PageRank resolves the circularity by iterating until the numbers stop moving. It is structural and slow: authority only shifts when the link graph shifts. The output is a hard ranked list, and the classic failure mode is people gaming that graph with link farms. PageRank is the classic core of search ranking, but modern engines layer hundreds of additional learned signals on top, so treat it as the foundation, not the whole system.
Social: engagement, scored by a proxy
A feed scores posts by predicted engagement, not by quality. It runs live: the current ranking decides who gets impressions, those impressions generate real engagement, and that engagement re-sorts the feed for the next round. That feedback loop is what makes the feed different from the other two. It produces rich-get-richer dynamics, and because engagement is only a proxy for what you want, the feed can drift away from quality while its own metric keeps climbing.
Attention: relevance, scored fresh per input
scores against each other for one job: predicting the next token. Each token emits a query, a key, and a value (the ), the query dots with every key to make scores, and the softmax turns those scores into weights that sum to 1. The output for each token is a blend of the value vectors, not a single winner. It is recomputed from scratch for every input, and it all serves next-token prediction. There is no human feedback loop inside the forward pass.
Why is engagement the dangerous objective?
Here is the honest part. The same score, normalize, allocate machinery is harmless when the target is relevance or prediction, and corrosive when the target is engagement.
Change the objective and you change the whole system. That is why "the algorithm" is not one thing. A search ranker, a feed, and a transformer share a skeleton, but the goal you point that skeleton at is what decides whether it serves you or feeds on you.
Frequently asked questions
What do search engines, social feeds, and language models have in common?
All three run the same three-step pattern: they score every candidate for relevance, normalize the scores into weights, then allocate a scarce resource by those weights. Search scores pages, a feed scores posts, and a language model scores tokens. The machinery is shared. The objective each one optimizes is what makes them behave so differently.
Is LLM attention the same as a search ranking?
No. A search ranking ends in a hard ordered list where the top results win and the rest are cut below the fold. Attention ends in a soft blend: a softmax gives every token a weight, the weights sum to 1, and the output mixes their values. Attention never crowns a single winner, and it serves next-token prediction rather than authority.
Why do social feeds show sensational content?
Because they optimize predicted engagement, and sensational, novel posts reliably win clicks, likes, and shares. Engagement is a proxy for what you want, so the feed can surface low-quality content that happens to be engaging. A live feedback loop makes it worse: the posts on top get the most impressions, which gathers the most engagement, which keeps them on top.
Does PageRank still power Google search?
PageRank is the classic core idea, that a page is important if important pages link to it. It is still a foundation, but modern search layers hundreds of additional learned signals on top of it, so no serious ranking today is PageRank alone.
What does the softmax do in attention?
The softmax turns raw relevance scores into weights that are all positive and sum to 1. That is the normalize step. It lets attention treat the scores as a distribution and produce a weighted average of the value vectors instead of a single pick.
Key takeaways
- All three systems run the identical skeleton: score every candidate, normalize the scores, allocate a scarce resource. The shape is shared even though the engineering is not.
- The objective is the whole story. Link authority, predicted engagement, and next-token prediction produce completely different behavior from the same machinery.
- Search and a feed both end in a hard ranked cut. Attention never picks a winner. It outputs a blend whose weights sum to 1.
- The feed is the corrosive one, not because its math is worse, but because engagement is a proxy for what you want, and its feedback loop amplifies the gap between the two.
- PageRank is only the classic core of search, and attention is one layer among many in a real model. Both are foundations, not the whole system.