Learning Paradigms · Established · Advanced

Policy Gradient Methods

Reinforcement learning algorithms that adjust the action-selection distribution directly using gradients of expected reward.

What Policy Gradient Methods is

Instead of learning values and deriving a policy, policy-gradient methods parameterise the policy itself, which handles continuous actions and stochastic behaviour naturally.

How it works

The gradient of expected return is estimated from sampled trajectories and used to increase the probability of actions that led to above-average return. PPO adds a clipped objective that prevents destructively large updates.

Why it matters

PPO in particular became the default optimiser for RLHF, so this family sits underneath a large share of deployed assistants.

Common uses

→Continuous control in robotics
→Language model preference optimisation
→Traffic and resource scheduling

Strengths

✓Handles continuous actions
✓Directly optimises the deployed policy

Watch for

✓High variance gradients
✓Sensitive to hyperparameters

Continue exploring

Reinforcement Learning from Human Feedback Q-Learning