Algorithms · Established · Intermediate

Gradient Boosting

An ensemble technique that adds models sequentially, each one correcting the residual errors of the combination so far.

What Gradient Boosting is

Gradient boosting builds a strong learner from many weak ones, typically shallow trees, by fitting each new tree to the gradient of the loss.

How it works

Implementations such as XGBoost, LightGBM and CatBoost add histogram binning, regularisation and efficient handling of categorical features and missing values.

Why it matters

It is the practical champion for tabular prediction and the default first model most data teams should try before anything deeper.

Common uses

→Fraud and credit scoring
→Click-through prediction
→Demand forecasting
→Insurance pricing

Strengths

✓Best-in-class tabular accuracy
✓Handles missing values natively

Watch for

✓Sensitive to hyperparameters
✓Sequential training is harder to parallelise

Continue exploring

Decision Tree Feature Engineering Model Evaluation