Learning Paradigms · Established · Advanced

Q-Learning

A value-based reinforcement learning algorithm that learns the expected long-term reward of taking an action in a given state.

What Q-Learning is

Q-learning maintains an estimate, Q(state, action), of the discounted future reward. Acting greedily with respect to Q yields a policy without ever modelling the environment explicitly.

How it works

Each experience updates the Q estimate towards the observed reward plus the discounted best value of the next state. Deep Q-networks replace the lookup table with a neural network so the method scales to high-dimensional inputs like pixels.

Why it matters

It is the classic entry point into RL and the basis of the deep RL results that first drew mainstream attention to the field.

Common uses

→Game-playing agents
→Inventory and queueing control
→Simple robotic policies

Strengths

✓Model-free
✓Converges under clear conditions in tabular settings

Watch for

✓Unstable with function approximation
✓Struggles with continuous action spaces

Continue exploring

Policy Gradient Methods Exploration vs Exploitation