generate_size_based_batches#

langchain_community.utilities.pebblo.generate_size_based_batches(
docs: List[Document],
max_batch_size: int = 102400,
) → List[List[Document]][source]#

Generate batches of documents based on page_content size. :param docs: List of documents to be batched. :param max_batch_size: Maximum size of each batch in bytes. Defaults to 100*1024(100KB)

Returns:

List of batches of documents

Return type:

List[List[Document]]

Parameters:

  • docs (List[Document])

  • max_batch_size (int)