llama-cpp-agents

A robust Python framework for local LLM agents: it delivers structured function calling, chat, and output generation via `llama-cpp-python`.

Llama-cpp-agents is your go-to framework for building high-performance, local AI agents. It operates on top of `llama-cpp-python`, providing a streamlined interface for complex LLM interactions (e.g., function calling, structured output). The core mechanism is a custom GGML-BNF grammar generator: this ensures the agent’s output precisely matches user-defined structures, supporting nested objects, dictionaries, and lists. Developers can quickly implement complex agents, including `FunctionCallingAgent` and `StructuredOutputAgent`, ensuring efficient, reliable performance on local hardware with models like Mistral 7B.

https://llama-cpp-agent.readthedocs.io/en/latest/