caniopen-sourceit

the catalog · ✨ AI assistants

llama.cpp

★ 122k

C/C++ LLM inference engine powering most local AI tools; runs GGUF models on any hardware

llama.cpp does the job of Cloud LLM APIs, self-hosted. It is MIT-licensed, so you can do essentially what you like with it, and setting it up is an afternoon, not a weekend. Cloud LLM APIs bills $20+ for it. 122k stars on GitHub.

Open-source replacement for Cloud LLM APIs.

Licence
MIT
Self-hosting
Medium
The paid one costs
$20+
GitHub stars
122k
view the code →official site →

Is it actually a replacement?

We have not written a verdict on Cloud LLM APIs yet, so this page says what llama.cpp is rather than judging what you would give up. When that product gets a page, this one will link to it.

llama.cpp questions

Is llama.cpp open source?

Yes — it is released under MIT.

Can I self-host llama.cpp?

Yes — we rate it medium to self-host.

More local model runner tools