Architectures · Established · Advanced

Long Short-Term Memory

Also known as: LSTM

A recurrent cell with input, forget and output gates that lets useful information persist across many time steps.

What Long Short-Term Memory is

LSTMs were designed to fix the vanishing gradient problem in plain RNNs by giving the network an explicit, gated memory channel.

How it works

A cell state runs through the sequence with only linear interactions; gates decide what is written, erased and read at each step, so gradients survive longer paths.

Why it matters

For a decade LSTMs powered translation, speech and handwriting recognition, and they still appear in resource-constrained sequence models.

Common uses

→Speech and handwriting recognition
→Sensor and financial time series
→On-device sequence models

Strengths

✓Handles medium-range dependencies well
✓Small memory footprint

Watch for

✓Slow to train relative to attention
✓Superseded on most language tasks

Continue exploring

Recurrent Neural Network Transformer