Vision Transformer

What Vision Transformer is

Vision transformers showed that the attention architecture generalises beyond text, matching or beating convolutional networks when trained on enough data.

How it works

The image is divided into fixed-size patches, each linearly embedded and given a positional encoding, then processed by standard transformer blocks. Self-supervised pretraining on unlabelled images is common.

Why it matters

ViT backbones underpin most multimodal models, which is how a chat assistant can look at a screenshot and reason about it.

Common uses

• →Multimodal model image encoders
• →Large-scale image classification
• →Medical imaging research