XinferenceEmbeddings#

class langchain_community.embeddings.xinference.XinferenceEmbeddings(server_url: str | None = None, model_uid: str | None = None)[source]#

Xinference embedding models.

To use, you should have the xinference library installed:

pip install xinference

If you’re simply using the services provided by Xinference, you can utilize the xinference_client package:

pip install xinference_client

Check out: xorbitsai/inference To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers.

Example

To start a local instance of Xinference, run

$ xinference

You can also deploy Xinference in a distributed cluster. Here are the steps:

Starting the supervisor:

$ xinference-supervisor

If you’re simply using the services provided by Xinference, you can utilize the xinference_client package:

pip install xinference_client

Starting the worker:

$ xinference-worker

Then, launch a model using command line interface (CLI).

Example:

$ xinference launch -n orca -s 3 -q q4_0

It will return a model UID. Then you can use Xinference Embedding with LangChain.

Example:

from langchain_community.embeddings import XinferenceEmbeddings

xinference = XinferenceEmbeddings(
    server_url="http://0.0.0.0:9997",
    model_uid = {model_uid} # replace model_uid with the model UID return from launching the model
)

Attributes

Methods

`__init__`([server_url, model_uid])
`aembed_documents`(texts)	Asynchronous Embed search docs.
`aembed_query`(text)	Asynchronous Embed query text.
`embed_documents`(texts)	Embed a list of documents using Xinference.
`embed_query`(text)	Embed a query of documents using Xinference.

Parameters:

server_url (str | None)
model_uid (str | None)

__init__(server_url: str | None = None, model_uid: str | None = None)[source]#

Parameters:

server_url (str | None)
model_uid (str | None)

async aembed_documents(texts: list[str]) → list[list[float]]#

Asynchronous Embed search docs.

Parameters:: texts (list[str]) – List of text to embed.
Returns:: List of embeddings.
Return type:: list[list[float]]

async aembed_query(text: str) → list[float]#

Asynchronous Embed query text.

Parameters:: text (str) – Text to embed.
Returns:: Embedding.
Return type:: list[float]

embed_documents(texts: List[str]) → List[List[float]][source]#

Embed a list of documents using Xinference. :param texts: The list of texts to embed.

Returns:: List of embeddings, one for each text.
Parameters:: texts (List[str])
Return type:: List[List[float]]

embed_query(text: str) → List[float][source]#

Embed a query of documents using Xinference. :param text: The text to embed.

Returns:: Embeddings for the text.
Parameters:: text (str)
Return type:: List[float]

Examples using XinferenceEmbeddings

Xorbits inference (Xinference)