Architectures · Established · Intermediate

Recurrent Neural Network

Also known as: RNN

A network that processes sequences one step at a time while carrying a hidden state forward.

What Recurrent Neural Network is

RNNs and their gated variants LSTM and GRU were the standard for language and time series before transformers, because they naturally handle variable-length sequences.

How it works

At each step the hidden state is updated from the previous state and the current input. Gates in LSTMs control what is remembered and forgotten, mitigating vanishing gradients over long sequences.

Why it matters

They explain why long-range dependency was hard, which is exactly the problem attention solved. Compact recurrent and state-space models remain relevant where streaming and low memory matter.

Common uses

→Time-series forecasting
→Streaming speech recognition
→Legacy sequence tagging systems

Strengths

✓Constant memory per step
✓Natural fit for streaming

Watch for

✓Sequential training is slow
✓Struggles with very long context

Continue exploring

Transformer Long Short-Term Memory Time Series Forecasting State Space Model