generate_size_based_batches

langchain_community.utilities.pebblo.generate_size_based_batches(docs: List[Document], max_batch_size: int = 102400) → List[List[Document]]

Generate batches of documents based on page_content size. :param docs: List of documents to be batched. :param max_batch_size: Maximum size of each batch in bytes. Defaults to 100*1024(100KB)

Returns:

List of batches of documents

Return type:

List[List[Document]]

Parameters:

• docs (List[Document]) –

• max_batch_size (int) –