LlamaCppEmbeddings#

class langchain_community.embeddings.llamacpp.LlamaCppEmbeddings[source]#

Bases: BaseModel, Embeddings

llama.cpp embedding models.

To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: abetlen/llama-cpp-python

Example

from langchain_community.embeddings import LlamaCppEmbeddings
llama = LlamaCppEmbeddings(model_path="/path/to/model.bin")

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param device: str | None = None#

Device type to use and pass to the model

param f16_kv: bool = False#

Use half-precision for key/value cache.

param logits_all: bool = False#

Return logits for all tokens, not just the last token.

param model_path: str [Required]#
param n_batch: int | None = 512#

Number of tokens to process in parallel. Should be a number between 1 and n_ctx.

param n_ctx: int = 512#

Token context window.

param n_gpu_layers: int | None = None#

Number of layers to be loaded into gpu memory. Default None.

param n_parts: int = -1#

Number of parts to split the model into. If -1, the number of parts is automatically determined.

param n_threads: int | None = None#

Number of threads to use. If None, the number of threads is automatically determined.

param seed: int = -1#

Seed. If -1, a random seed is used.

param use_mlock: bool = False#

Force system to keep model in RAM.

param verbose: bool = True#

Print verbose output to stderr.

param vocab_only: bool = False#

Only load the vocabulary, no weights.

async aembed_documents(texts: List[str]) List[List[float]]#

Asynchronous Embed search docs.

Parameters:

texts (List[str]) – List of text to embed.

Returns:

List of embeddings.

Return type:

List[List[float]]

async aembed_query(text: str) List[float]#

Asynchronous Embed query text.

Parameters:

text (str) – Text to embed.

Returns:

Embedding.

Return type:

List[float]

embed_documents(texts: List[str]) List[List[float]][source]#

Embed a list of documents using the Llama model.

Parameters:

texts (List[str]) – The list of texts to embed.

Returns:

List of embeddings, one for each text.

Return type:

List[List[float]]

embed_query(text: str) List[float][source]#

Embed a query using the Llama model.

Parameters:

text (str) – The text to embed.

Returns:

Embeddings for the text.

Return type:

List[float]

Examples using LlamaCppEmbeddings