VertexFSVectorStore#

class langchain_google_community.bq_storage_vectorstores.featurestore.VertexFSVectorStore[source]#

Bases: BaseBigQueryVectorStore

A vector store implementation that utilizes BigQuery Storage and Vertex AI Feature Store.

This class provides efficient storage, using BigQuery as the underlining source of truth and retrieval of documents with vector embeddings within Vertex AI Feature Store. It is particularly indicated for low latency serving. It supports similarity search, filtering and getting nearest neighbor by id. Optionally, this class can leverage a BigQuery Vector Search for batch serving through the to_bq_vector_store method.

embedding#: Embedding model for generating and comparing embeddings.

project_id#: Google Cloud Project ID where BigQuery resources are located.

dataset_name#: BigQuery dataset name.

table_name#: BigQuery table name.

location#: BigQuery region/location.

content_field#: Name of the column storing document content (default: “content”).

embedding_field#: Name of the column storing text embeddings (default: “embedding”).

doc_id_field#: Name of the column storing document IDs (default: “doc_id”).

credentials#: Optional Google Cloud credentials object.

embedding_dimension#: Dimension of the embedding vectors (inferred if not provided).

online_store_name#

Name of the Vertex AI Feature Store online store. Defaults to the dataset name.

Type:: str, optional

online_store_location#

Location of the online store. Default to “location” parameter.

Type:: str, optional

view_name#

Name of the Feature View. Defaults to the table name.

Type:: str, optional

cron_schedule#

Cron schedule for data syncing.

Type:: str, optional

algorithm_config#

Algorithm configuration for indexing.

Type:: Any, optional

filter_columns#

Columns to use for filtering.

Type:: List[str], optional

crowding_column#

Column to use for crowding.

Type:: str, optional

distance_measure_type#

Distance measure type (default: DOT_PRODUCT_DISTANCE).

Type:: str, optional

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param algorithm_config: Any | None = None#

param content_field: str = 'content'#

param credentials: Any | None = None#

param cron_schedule: str | None = None#

param crowding_column: str | None = None#

param dataset_name: str [Required]#

param distance_measure_type: str | None = None#

param doc_id_field: str = 'doc_id'#

param embedding: Embeddings [Required]#

param embedding_dimension: int | None = None#

param embedding_field: str = 'embedding'#

param extra_fields: Dict[str, str] | None = None#

param feature_view: Any = None#

param filter_columns: List[str] | None = None#

param location: str [Required]#

param online_store_location: str | None = None#

param online_store_name: str | None = None#

param project_id: str [Required]#

param table_name: str [Required]#

param table_schema: Any = None#

param view_name: str | None = None#

async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]#

Async run more documents through the embeddings and add to the vectorstore.

Parameters:

documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – Additional keyword arguments.

Returns:

List of IDs of the added texts.

Raises:

ValueError – If the number of IDs does not match the number of documents.

Return type:

List[str]

async aadd_texts(texts: Iterable[str], metadatas: List[dict] | None = None, **kwargs: Any) → List[str]#

Async run more texts through the embeddings and add to the vectorstore.

Parameters:

texts (Iterable[str]) – Iterable of strings to add to the vectorstore.
metadatas (List[dict] | None) – Optional list of metadatas associated with the texts. Default is None.
**kwargs (Any) – vectorstore specific parameters.

Returns:

List of ids from adding the texts into the vectorstore.

Raises:

ValueError – If the number of metadatas does not match the number of texts.
ValueError – If the number of ids does not match the number of texts.

Return type:

List[str]

add_documents(documents: List[Document], **kwargs: Any) → List[str]#

Add or update documents in the vectorstore.

Parameters:

documents (List[Document]) – Documents to add to the vectorstore.
kwargs (Any) – Additional keyword arguments. if kwargs contains ids and documents contain ids, the ids in the kwargs will receive precedence.

Returns:

List of IDs of the added texts.

Raises:

ValueError – If the number of ids does not match the number of documents.

Return type:

List[str]

add_texts(texts: List[str], metadatas: List[dict] | None = None, **kwargs: Any) → List[str]#

Run more texts through the embeddings and add to the vectorstore.

Parameters:

texts (List[str]) – List of strings to add to the vectorstore.
metadatas (List[dict] | None) – Optional list of metadata records associated with the texts. (ie [{“url”: “www.myurl1.com”, “title”: “title1”}, {“url”: “www.myurl2.com”, “title”: “title2”}])
kwargs (Any) –

Returns:

List of ids from adding the texts into the vectorstore.

Return type:

List[str]

add_texts_with_embeddings(texts: List[str], embs: List[List[float]], metadatas: List[dict] | None = None) → List[str]#

Add precomputed embeddings and relative texts / metadatas to the vectorstore.

Parameters:

ids – List of unique ids in string format
texts (List[str]) – List of strings to add to the vectorstore.
embs (List[List[float]]) – List of lists of floats with text embeddings for texts.
metadatas (List[dict] | None) – Optional list of metadata records associated with the texts. (ie [{“url”: “www.myurl1.com”, “title”: “title1”}, {“url”: “www.myurl2.com”, “title”: “title2”}])

Returns:

List of ids from adding the texts into the vectorstore.

Return type:

List[str]

async adelete(ids: List[str] | None = None, **kwargs: Any) → bool | None#

Delete by vector ID or other criteria.

Parameters:

ids (List[str] | None) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.

Returns:

True if deletion is successful, False otherwise, None if not implemented.

Return type:

Optional[bool]

async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST#

Async return VectorStore initialized from documents and embeddings.

Parameters:

documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – Additional keyword arguments.

Returns:

VectorStore initialized from documents and embeddings.

Return type:

VectorStore

async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: List[dict] | None = None, **kwargs: Any) → VST#

Async return VectorStore initialized from texts and embeddings.

Parameters:

texts (List[str]) – Texts to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
metadatas (List[dict] | None) – Optional list of metadatas associated with the texts. Default is None.
kwargs (Any) – Additional keyword arguments.

Returns:

VectorStore initialized from texts and embeddings.

Return type:

VectorStore

async aget_by_ids(ids: Sequence[str], /) → List[Document]#

Async get documents by their IDs.

The returned documents are expected to have the ID field set to the ID of the document in the vector store.

Fewer documents may be returned than requested if some IDs are not found or if there are duplicated IDs.

Users should not assume that the order of the returned documents matches the order of the input IDs. Instead, users should rely on the ID field of the returned documents.

This method should NOT raise exceptions if no documents are found for some IDs.

Parameters:: ids (Sequence[str]) – List of ids to retrieve.
Returns:: List of Documents.
Return type:: List[Document]

New in version 0.2.11.

async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]#

Async return docs selected using the maximal marginal relevance.

Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents.

Parameters:

query (str) – Text to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm. Default is 20.
lambda_mult (float) – Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5.
kwargs (Any) –

Returns:

List of Documents selected by maximal marginal relevance.

Return type:

List[Document]

async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]#

Async return docs selected using the maximal marginal relevance.

Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents.

Parameters:

embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm. Default is 20.
lambda_mult (float) – Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Documents selected by maximal marginal relevance.

Return type:

List[Document]

as_retriever(**kwargs: Any) → VectorStoreRetriever#

Return VectorStoreRetriever initialized from this VectorStore.

Parameters:

**kwargs (Any) –

Keyword arguments to pass to the search function. Can include: search_type (Optional[str]): Defines the type of search that

the Retriever should perform. Can be “similarity” (default), “mmr”, or “similarity_score_threshold”.

search_kwargs (Optional[Dict]): Keyword arguments to pass to the

search function. Can include things like:

k: Amount of documents to return (Default: 4) score_threshold: Minimum relevance threshold

for similarity_score_threshold

fetch_k: Amount of documents to pass to MMR algorithm: (Default: 20)
lambda_mult: Diversity of results returned by MMR;: 1 for minimum diversity and 0 for maximum. (Default: 0.5)

filter: Filter by document metadata

Returns:

Retriever class for VectorStore.

Return type:

VectorStoreRetriever

Examples:

# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)

# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)

# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)

# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})

# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)

async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]#

Async return docs most similar to query using a specified search type.

Parameters:

query (str) – Input text.
search_type (str) – Type of search to perform. Can be “similarity”, “mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Documents most similar to the query.

Raises:

ValueError – If search_type is not one of “similarity”, “mmr”, or “similarity_score_threshold”.

Return type:

List[Document]

async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]#

Async return docs most similar to query.

Parameters:

query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Documents most similar to the query.

Return type:

List[Document]

async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]#

Async return docs most similar to embedding vector.

Parameters:

embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Documents most similar to the query vector.

Return type:

List[Document]

async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]#

Async return docs and relevance scores in the range [0, 1].

0 is dissimilar, 1 is most similar.

Parameters:

query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) –
kwargs to be passed to similarity search. Should include: score_threshold: Optional, a floating point value between 0 to 1 to

filter the resulting set of retrieved docs

Returns:

List of Tuples of (doc, similarity_score)

Return type:

List[Tuple[Document, float]]

async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]#

Async run similarity search with distance.

Parameters:

*args (Any) – Arguments to pass to the search method.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Tuples of (doc, similarity_score).

Return type:

List[Tuple[Document, float]]

delete(ids: List[str] | None = None, **kwargs: Any) → bool | None#

Delete documents by record IDs

Parameters:

ids (List[str] | None) – List of ids to delete.
**kwargs (Any) – Other keyword arguments that subclasses might use.

Returns:

True if deletion is successful, False otherwise, None if not implemented.

Return type:

Optional[bool]

classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST#

Return VectorStore initialized from documents and embeddings.

Parameters:

documents (List[Document]) – List of Documents to add to the vectorstore.
embedding (Embeddings) – Embedding function to use.
kwargs (Any) – Additional keyword arguments.

Returns:

VectorStore initialized from documents and embeddings.

Return type:

VectorStore

classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: List[dict] | None = None, **kwargs: Any) → VertexFSVectorStore[source]#

Return VectorStore initialized from input texts

Parameters:

texts (List[str]) – List of strings to add to the vectorstore.
embedding (Embeddings) – An embedding model instance for text to vector transformations.
metadatas (List[dict] | None) – Optional list of metadata records associated with the texts. (ie [{“url”: “www.myurl1.com”, “title”: “title1”}, {“url”: “www.myurl2.com”, “title”: “title2”}])
kwargs (Any) –

Returns:

List of ids from adding the texts into the vectorstore.

Return type:

VertexFSVectorStore

get_by_ids(ids: Sequence[str], /) → List[Document]#

Get documents by their IDs.

The returned documents are expected to have the ID field set to the ID of the document in the vector store.

Fewer documents may be returned than requested if some IDs are not found or if there are duplicated IDs.

Users should not assume that the order of the returned documents matches the order of the input IDs. Instead, users should rely on the ID field of the returned documents.

This method should NOT raise exceptions if no documents are found for some IDs.

Parameters:: ids (Sequence[str]) – List of ids to retrieve.
Returns:: List of Documents.
Return type:: List[Document]

New in version 0.2.11.

get_documents(ids: List[str] | None, filter: Dict[str, Any] | None = None, **kwargs: Any) → List[Document][source]#

Search documents by their ids or metadata values. :param ids: List of ids of documents to retrieve from the vectorstore. :param filter: Filter on metadata properties, e.g.

{
“str_property”: “foo”, “int_property”: 123

}

Returns:

List of ids from adding the texts into the vectorstore.

Parameters:

ids (List[str] | None) –
filter (Dict[str, Any] | None) –
kwargs (Any) –

Return type:

List[Document]

max_marginal_relevance_search(query: str, k: int = 5, fetch_k: int = 25, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]#

Return docs selected using the maximal marginal relevance.

Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents.

Parameters:

**kwargs (Any) –
query (str) – search query text.
filter –
Filter on metadata properties, e.g. {

”str_property”: “foo”, “int_property”: 123

}
k (int) – Number of Documents to return. Defaults to 5.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5.

Returns:

List of Documents selected by maximal marginal relevance.

Return type:

List[Document]

max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 5, fetch_k: int = 25, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]#

Return docs selected using the maximal marginal relevance.

Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents.

Parameters:

embedding (List[float]) – Embedding to look up documents similar to.
filter –
Filter on metadata properties, e.g. {

”str_property”: “foo”, “int_property”: 123

}
k (int) – Number of Documents to return. Defaults to 5.
fetch_k (int) – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult (float) – Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5.
kwargs (Any) –

Returns:

List of Documents selected by maximal marginal relevance.

Return type:

List[Document]

search(query: str, search_type: str, **kwargs: Any) → List[Document]#

Return docs most similar to query using a specified search type.

Parameters:

query (str) – Input text
search_type (str) – Type of search to perform. Can be “similarity”, “mmr”, or “similarity_score_threshold”.
**kwargs (Any) – Arguments to pass to the search method.

Returns:

List of Documents most similar to the query.

Raises:

ValueError – If search_type is not one of “similarity”, “mmr”, or “similarity_score_threshold”.

Return type:

List[Document]

search_neighbors_by_ids(ids: List[str], filter: Dict[str, Any] | None = None, **kwargs: Any) → List[List[List[Any]]][source]#

Searches for neighboring entities in a Vertex Feature Store based on: their IDs and optional filter on metadata

Parameters:

ids (List[str]) – A list of string identifiers representing the entities to search for.
filter (Dict[str, Any] | None) –
(Optional) A dictionary specifying filter criteria for document on metadata properties, e.g.

{
“str_property”: “foo”, “int_property”: 123

}
kwargs (Any) –

Return type:

List[List[List[Any]]]

similarity_search(query: str, k: int = 5, **kwargs: Any) → List[Document]#

Search for top k docs most similar to input query.

Parameters:

query (str) – search query to search documents with.
filter – (Optional) A dictionary specifying filtering criteria for the documents. Ie. {“title”: “mytitle”}
k (int) – (Optional) The number of top-ranking similar documents to return per embedding. Defaults to 5.
kwargs (Any) –

Returns:

Return docs most similar to input query.

Return type:

List[Document]

similarity_search_by_vector(embedding: List[float], k: int = 5, **kwargs: Any) → List[Document]#

Return docs most similar to embedding vector.

Parameters:

embedding (List[float]) – Embedding to look up documents similar to.
filter – (Optional) A dictionary specifying filtering criteria for the documents. Ie. {“title”: “mytitle”}
k (int) – (Optional) The number of top-ranking similar documents to return per embedding. Defaults to 5.
kwargs (Any) –

Returns:

Return docs most similar to embedding vector.

Return type:

List[Document]

similarity_search_by_vector_with_score(embedding: List[float], filter: Dict[str, Any] | None = None, k: int = 5) → List[Tuple[Document, float]]#

Return docs most similar to embedding vector with scores.

Parameters:

embedding (List[float]) – Embedding to look up documents similar to.
filter (Dict[str, Any] | None) – (Optional) A dictionary specifying filtering criteria for the documents. Ie. {“title”: “mytitle”}
k (int) – (Optional) The number of top-ranking similar documents to return per embedding. Defaults to 5.

Returns:

Return docs most similar to embedding vector.

Return type:

List[Tuple[Document, float]]

similarity_search_by_vectors(embeddings: List[List[float]], filter: Dict[str, Any] | None = None, k: int = 5, with_scores: bool = False, with_embeddings: bool = False, **kwargs: Any) → Any#

Core similarity search function. Handles a list of embedding vectors,: optionally returning scores and embeddings.

Parameters:

embeddings (List[List[float]]) – A list of embedding vectors, where each vector is a list of floats.
filter (Dict[str, Any] | None) – (Optional) A dictionary specifying filtering criteria for the documents. Ie. {“title”: “mytitle”}
k (int) – (Optional) The number of top-ranking similar documents to return per embedding. Defaults to 5.
with_scores (bool) – (Optional) If True, include similarity scores in the result for each matched document. Defaults to False.
with_embeddings (bool) – (Optional) If True, include the matched document’s embedding vector in the result. Defaults to False.
kwargs (Any) –

Returns:

A list of k documents for each embedding in embeddings

Return type:

Any

similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]#

Return docs and relevance scores in the range [0, 1].

0 is dissimilar, 1 is most similar.

Parameters:

query (str) – Input text.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs (Any) –
kwargs to be passed to similarity search. Should include: score_threshold: Optional, a floating point value between 0 to 1 to

filter the resulting set of retrieved docs.

Returns:

List of Tuples of (doc, similarity_score).

Return type:

List[Tuple[Document, float]]

similarity_search_with_score(query: str, filter: Dict[str, Any] | None = None, k: int = 5, **kwargs: Any) → List[Tuple[Document, float]]#

Search for top k docs most similar to input query, returns both docs and: scores.

Parameters:

query (str) – search query to search documents with.
filter (Dict[str, Any] | None) – (Optional) A dictionary specifying filtering criteria for the documents. Ie. {“title”: “mytitle”}
k (int) – (Optional) The number of top-ranking similar documents to return per embedding. Defaults to 5.
kwargs (Any) –

Returns:

Return docs most similar to input query along with scores.

Return type:

List[Tuple[Document, float]]

sync_data() → None[source]#

Sync the data from the BigQuery source into the Executor source

Return type:: None

to_bq_vector_store(**kwargs: Any) → Any[source]#

Converts the current object’s parameters into a BigQueryVectorStore instance.

This method combines the base parameters of the current object to create a: BigQueryVectorStore object.

Parameters:

**kwargs (Any) – Additional keyword arguments to be passed to the ` BigQueryVectorStore` constructor. These override any matching parameters in the base object.

Returns:

An initialized BigQueryVectorStore object ready: for vector search operations.

Return type:

BigQueryVectorStore

Raises:

ValueError – If any of the combined parameters are invalid for initializing a BigQueryVectorStore.

property embeddings: Embeddings | None#: Access the query embedding object if available.

property full_table_id: str#