Llama.cpp

Llama.cpp is a C++ implementation designed to run large language models efficiently on standard hardware. It takes Meta's Llama models and other compatible LLMs and optimises them for speed and low resource consumption, making it possible to run these models locally on consumer-grade machines without specialised hardware. The tool is particularly useful for developers, researchers, and anyone wanting to run LLMs privately without relying on cloud APIs. Because it's written in pure C++ with minimal dependencies, it runs fast across different operating systems. Llama.cpp powers many other applications and interfaces in the open source ecosystem that need efficient local inference. Since it's open source, you can inspect the code, modify it, and use it without restrictions. The project is actively maintained and has become one of the most popular ways to run Llama models and compatible models locally.