Architectures · Emerging · Advanced

State Space Model

Also known as: SSM, Mamba-style models

A sequence architecture that maintains a compressed continuous state, offering near-linear scaling with sequence length.

What State Space Model is

State space models revisit recurrence with modern mathematics, aiming to match transformer quality while avoiding attention's quadratic cost on long inputs.

How it works

A learned linear dynamical system propagates state across the sequence, with selective mechanisms deciding what to retain. Efficient parallel scan implementations make training practical on accelerators.

Why it matters

They are one of the leading research directions for very long context and efficient inference, and hybrid attention-SSM designs are appearing in released models.

Common uses

→Long-document and genomics sequences
→Efficient long-context language models
→Streaming audio

Strengths

✓Scales better than attention on long inputs
✓Constant memory inference

Watch for

✓Ecosystem and tooling less mature than transformers

Continue exploring

Transformer Attention Mechanism Context Window Recurrent Neural Network