generate_size_based_batches#

langchain_community.utilities.pebblo.generate_size_based_batches(

docs: List[Document],

max_batch_size: int = 102400,

) → List[List[Document]][source]#

Generate batches of documents based on page_content size. :param docs: List of documents to be batched. :param max_batch_size: Maximum size of each batch in bytes. Defaults to 100*1024(100KB)

Returns:

List of batches of documents

Return type:

List[List[Document]]

Parameters:

• docs (List[Document])

• max_batch_size (int)