NeedleLoader#

class langchain_community.document_loaders.needle.NeedleLoader(needle_api_key: str | None = None, collection_id: str | None = None)[source]#

NeedleLoader is a document loader for managing documents stored in a collection.

Setup:

Install the needle-python library and set your Needle API key.

pip install needle-python
export NEEDLE_API_KEY="your-api-key"
Key init args:

  • needle_api_key (Optional[str]): API key for authenticating with Needle.

  • collection_id (str): Needle collection to load documents from.

Usage:
from langchain_community.document_loaders.needle import NeedleLoader

loader = NeedleLoader(
    needle_api_key="your-api-key",
    collection_id="your-collection-id"
)

# Load documents
documents = loader.load()
for doc in documents:
    print(doc.metadata)

# Lazy load documents
for doc in loader.lazy_load():
    print(doc.metadata)

Initializes the NeedleLoader with API key and collection ID.

Parameters:

  • needle_api_key (Optional[str]) – API key for authenticating with Needle.

  • collection_id (Optional[str]) – Identifier for the Needle collection.

Raises:

  • ImportError – If the needle-python library is not installed.

  • ValueError – If the collection ID is not provided.

Methods

__init__([needle_api_key, collection_id])

Initializes the NeedleLoader with API key and collection ID.

add_files(files)

Adds files to the Needle collection.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

Lazily loads documents from the Needle collection.

load()

Loads all documents from the Needle collection.

load_and_split([text_splitter])

Load Documents and split into chunks.
__init__(needle_api_key: str | None = None, collection_id: str | None = None) → None[source]#

Initializes the NeedleLoader with API key and collection ID.

Parameters:

  • needle_api_key (Optional[str]) – API key for authenticating with Needle.

  • collection_id (Optional[str]) – Identifier for the Needle collection.

Raises:

  • ImportError – If the needle-python library is not installed.

  • ValueError – If the collection ID is not provided.

Return type:

None

add_files(files: Dict[str, str]) → None[source]#

Adds files to the Needle collection.

Parameters:

files (Dict[str, str]) – Dictionary where keys are file names and values are file URLs.

Raises:

  • ImportError – If the needle-python library is not installed.

  • ValueError – If the collection is not properly initialized.

Return type:

None

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → list[Document]#

Load data into Document objects.

Return type:

list[Document]

lazy_load() → Iterator[Document][source]#

Lazily loads documents from the Needle collection.

Yields:

Iterator[Document] – An iterator over the documents.

Return type:

Iterator[Document]

load() → List[Document][source]#

Loads all documents from the Needle collection.

Returns:

A list of documents from the collection.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → list[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

list[Document]