UnstructuredMarkdownLoader#

class langchain_community.document_loaders.markdown.UnstructuredMarkdownLoader(file_path: str | List[str] | Path | List[Path], *, mode: str = 'single', **unstructured_kwargs: Any)[source]#

Load Markdown files using Unstructured.

You can run the loader in one of two modes: “single” and “elements”. If you use “single” mode, the document will be returned as a single langchain Document object. If you use “elements” mode, the unstructured library will split the document into elements such as Title and NarrativeText. You can pass in additional unstructured kwargs after mode to apply different unstructured settings.

Setup:

Install langchain-community.

pip install -U langchain-community
Instantiate:
from langchain_community.document_loaders import UnstructuredMarkdownLoader

loader = UnstructuredMarkdownLoader(
    "./example_data/example.md",
    mode="elements",
    strategy="fast",
)
Lazy load:
docs = []
docs_lazy = loader.lazy_load()

# async variant:
# docs_lazy = await loader.alazy_load()

for doc in docs_lazy:
    docs.append(doc)
print(docs[0].page_content[:100])
print(docs[0].metadata)

Sample Markdown Document
{'source': './example_data/example.md', 'category_depth': 0, 'last_modified': '2024-08-14T15:04:18', 'languages': ['eng'], 'filetype': 'text/markdown', 'file_directory': './example_data', 'filename': 'example.md', 'category': 'Title', 'element_id': '3d0b313864598e704aa26c728ecb61e5'}
Async load:
docs = await loader.aload()
print(docs[0].page_content[:100])
print(docs[0].metadata)

Sample Markdown Document
{'source': './example_data/example.md', 'category_depth': 0, 'last_modified': '2024-08-14T15:04:18', 'languages': ['eng'], 'filetype': 'text/markdown', 'file_directory': './example_data', 'filename': 'example.md', 'category': 'Title', 'element_id': '3d0b313864598e704aa26c728ecb61e5'}

References

https://unstructured-io.github.io/unstructured/core/partition.html#partition-md

Initialize with file path.

Methods

__init__(file_path, *[, mode])

Initialize with file path.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

Load file.

load()

Load data into Document objects.

load_and_split([text_splitter])

Load Documents and split into chunks.
Parameters:

  • file_path (Union[str, List[str], Path, List[Path]]) –

  • mode (str) –

  • unstructured_kwargs (Any) –

__init__(file_path: str | List[str] | Path | List[Path], *, mode: str = 'single', **unstructured_kwargs: Any)#

Initialize with file path.

Parameters:

  • file_path (str | List[str] | Path | List[Path]) –

  • mode (str) –

  • unstructured_kwargs (Any) –

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

lazy_load() → Iterator[Document]#

Load file.

Return type:

Iterator[Document]

load() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → List[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

List[Document]

Examples using UnstructuredMarkdownLoader

On this page