Llama.cpp

llama.cpp python library is a simple Python bindings for @ggerganov llama.cpp.

This package provides:

• Low-level access to C API via ctypes interface.
• High-level Python API for text completion
  • OpenAI-like API
  • LangChain compatibility
  • LlamaIndex compatibility
• OpenAI compatible web server
  • Local Copilot replacement
  • Function Calling support
  • Vision API support
  • Multiple Models

Installation and Setup

• Install the Python package

  pip install llama-cpp-python
• Download one of the supported models and convert them to the llama.cpp format per the instructions

Chat models

See a usage example.

from langchain_community.chat_models import ChatLlamaCpp

API Reference:ChatLlamaCpp

LLMs

See a usage example.

from langchain_community.llms import LlamaCpp

API Reference:LlamaCpp

Embedding models

See a usage example.

from langchain_community.embeddings import LlamaCppEmbeddings

API Reference:LlamaCppEmbeddings