Moyan AI Training Institution LogoMoyan AI

Practice · Established · Intermediate

Fine-Tuning

Continuing training of a pretrained model on a smaller, targeted dataset to specialise its behaviour, format or domain.

What Fine-Tuning is

Fine-tuning is best at teaching style, structure and task patterns. It is a poor and expensive way to inject facts, which is what retrieval is for.

How it works

Full fine-tuning updates all weights; parameter-efficient methods such as LoRA train small adapter matrices instead, cutting cost by orders of magnitude. Datasets of a few hundred to a few thousand high-quality examples often suffice.

Why it matters

Choosing correctly between prompting, retrieval and fine-tuning is one of the highest-leverage architecture decisions in an AI project.

Common uses

  • →Consistent output formats
  • →Brand voice and tone
  • →Domain jargon and classification
  • →Reducing prompt length and cost

Strengths

  • ✓Reliable formatting and style
  • ✓Shorter prompts at inference
  • ✓Can beat larger models on narrow tasks

Watch for

  • ✓Data preparation effort
  • ✓Model must be retrained as needs change
  • ✓Catastrophic forgetting of general ability

Continue exploring

More in this collection

Browse all AI Concepts