Infrastructure · Fast-moving · Intermediate

Edge AI

Running AI models directly on devices — phones, cameras, vehicles, sensors — instead of sending data to a server.

What Edge AI is

Edge deployment keeps data local, works offline and responds in milliseconds, at the cost of tight memory, power and compute budgets.

How it works

Models are compressed with quantisation, pruning and distillation, then compiled to device runtimes and neural accelerators. Hybrid designs run a small local model and escalate hard cases to the cloud.

Why it matters

Privacy regulation, latency requirements and connectivity gaps all push inference towards the device, and capable small models finally make it viable.

Common uses

→On-device assistants and dictation
→Smart cameras and doorbells
→In-vehicle perception
→Industrial sensors without connectivity

Strengths

✓No data leaves the device
✓Works offline
✓Very low latency

Watch for

✓Limited model size
✓Fragmented hardware targets
✓Updates are harder to roll out

Continue exploring

Quantization Small Language Model AI and the Internet of Things Data Privacy in AI