UnstructuredLoader#

class langchain_unstructured.document_loaders.UnstructuredLoader(file_path: str | Path | list[str] | list[pathlib.Path] | None = None, *, file: IO[bytes] | list[IO[bytes]] | None = None, partition_via_api: bool = False, post_processors: list[Callable[[str], str]] | None = None, api_key: str | None = None, client: UnstructuredClient | None = None, url: str | None = None, **kwargs: Any)[source]#

Unstructured document loader interface.

Setup:

Install langchain-unstructured and set environment variable UNSTRUCTURED_API_KEY.

Instantiate:
Lazy load:
docs = []
docs_lazy = loader.lazy_load()

# async variant:
# docs_lazy = await loader.alazy_load()

for doc in docs_lazy:
    docs.append(doc)
print(docs[0].page_content[:100])
print(docs[0].metadata)

1 2 0 2
{'source': './example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': './example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2024-07-25T21:28:58', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText', 'element_id': 'd3ce55f220dfb75891b4394a18bcb973'}
Async load:
docs = await loader.aload()
print(docs[0].page_content[:100])
print(docs[0].metadata)

1 2 0 2
{'source': './example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': './example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2024-07-25T21:28:58', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText', 'element_id': 'd3ce55f220dfb75891b4394a18bcb973'}

References

https://docs.unstructured.io/api-reference/api-services/sdk https://docs.unstructured.io/api-reference/api-services/overview https://docs.unstructured.io/open-source/core-functionality/partitioning https://docs.unstructured.io/open-source/core-functionality/chunking

Initialize loader.

Methods

__init__([file_path, file, ...])

Initialize loader.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

Load file(s) to the _UnstructuredBaseLoader.

load()

Load data into Document objects.

load_and_split([text_splitter])

Load Documents and split into chunks.
Parameters:

  • file_path (Optional[str | Path | list[str] | list[Path]]) –

  • file (Optional[IO[bytes] | list[IO[bytes]]]) –

  • partition_via_api (bool) –

  • post_processors (Optional[list[Callable[[str], str]]]) –

  • api_key (Optional[str]) –

  • client (Optional[UnstructuredClient]) –

  • url (Optional[str]) –

  • kwargs (Any) –

__init__(file_path: str | Path | list[str] | list[pathlib.Path] | None = None, *, file: IO[bytes] | list[IO[bytes]] | None = None, partition_via_api: bool = False, post_processors: list[Callable[[str], str]] | None = None, api_key: str | None = None, client: UnstructuredClient | None = None, url: str | None = None, **kwargs: Any)[source]#

Initialize loader.

Parameters:

  • file_path (str | Path | list[str] | list[pathlib.Path] | None) –

  • file (IO[bytes] | list[IO[bytes]] | None) –

  • partition_via_api (bool) –

  • post_processors (list[Callable[[str], str]] | None) –

  • api_key (str | None) –

  • client (UnstructuredClient | None) –

  • url (str | None) –

  • kwargs (Any) –

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

lazy_load() → Iterator[Document][source]#

Load file(s) to the _UnstructuredBaseLoader.

Return type:

Iterator[Document]

load() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → List[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

List[Document]

On this page