Local runtime · Established · Advanced
llama.cpp (2)
A C++ inference engine that runs quantised language models efficiently on CPUs and consumer GPUs.
What llama.cpp (2) is
It made local model running practical on ordinary hardware, and its GGUF quantised format became a de facto standard for distributing local models.
How it works
Models are converted to GGUF at a chosen precision and executed with hardware-specific optimisations across desktop, mobile and embedded targets.
Why it matters
Much of the local AI ecosystem, including higher-level tools, is built on it.
Common uses
- →Running models on laptops and phones
- →Embedded deployments
- →Privacy-first applications
Strengths
- ✓Minimal dependencies
- ✓Broad hardware support
Watch for
- ✓Lower-level than managed runtimes
- ✓Quality drops at aggressive quantisation
Continue exploring
More in this collection
Browse all AI Technology