Language model · Established · Advanced

BERT (2)

A bidirectional encoder transformer that reads a whole sequence at once, dominant for classification and retrieval before generative models took over.

What BERT (2) is

BERT is trained with masked language modelling, so it sees context from both directions. It produces representations rather than free text, which suits classification, tagging and reranking.

How it works

A pretrained BERT is fine-tuned with a small task head on labelled data. Distilled and multilingual variants are widely deployed in search and production NLP.

Why it matters

Encoder models remain the efficient choice for high-volume classification and reranking where generation is unnecessary.

Common uses

→Search relevance and reranking
→Text classification at scale
→Entity recognition
→Sentence embeddings

Strengths

✓Cheap and fast at inference
✓Strong with modest labelled data

Watch for

✓Cannot generate text
✓Short input limit

Continue exploring

Transformer Embeddings Reranking Natural Language Processing