Transformer

What Transformer is

The transformer replaced recurrence with attention, so a model can look directly at any earlier token instead of passing information along step by step. That change made training fully parallel and long-range dependencies tractable.

How it works

Each block contains multi-head self-attention followed by a feed-forward network, with residual connections and layer normalisation around both. Positional encodings inject word order because attention itself is order-agnostic. Decoder-only stacks power most chat models; encoder-decoder variants remain common in translation.

Why it matters

Nearly every frontier model in text, vision, audio and protein modelling is a transformer. Its scaling behaviour is what turned model size into a strategy.

Common uses

• →Language models
• →Vision transformers
• →Speech and music models
• →Protein and genomics models