WikipediaLoader#
- class langchain_community.document_loaders.wikipedia.WikipediaLoader(query: str, lang: str = 'en', load_max_docs: int | None = 25, load_all_available_meta: bool | None = False, doc_content_chars_max: int | None = 4000)[source]#
Load from Wikipedia.
The hard limit on the length of the query is 300 for now.
Each wiki page represents one Document.
Initializes a new instance of the WikipediaLoader class.
- Parameters:
query (str) β The query string to search on Wikipedia.
lang (str, optional) β The language code for the Wikipedia language edition. Defaults to βenβ.
load_max_docs (int, optional) β The maximum number of documents to load. Defaults to 100.
load_all_available_meta (bool, optional) β Indicates whether to load all available metadata for each document. Defaults to False.
doc_content_chars_max (int, optional) β The maximum number of characters for the document content. Defaults to 4000.
Methods
__init__
(query[,Β lang,Β load_max_docs,Β ...])Initializes a new instance of the WikipediaLoader class.
A lazy loader for Documents.
aload
()Load data into Document objects.
Loads the query result from Wikipedia into a list of Documents.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
- __init__(query: str, lang: str = 'en', load_max_docs: int | None = 25, load_all_available_meta: bool | None = False, doc_content_chars_max: int | None = 4000)[source]#
Initializes a new instance of the WikipediaLoader class.
- Parameters:
query (str) β The query string to search on Wikipedia.
lang (str, optional) β The language code for the Wikipedia language edition. Defaults to βenβ.
load_max_docs (int, optional) β The maximum number of documents to load. Defaults to 100.
load_all_available_meta (bool, optional) β Indicates whether to load all available metadata for each document. Defaults to False.
doc_content_chars_max (int, optional) β The maximum number of characters for the document content. Defaults to 4000.
- async alazy_load() AsyncIterator[Document] #
A lazy loader for Documents.
- Return type:
AsyncIterator[Document]
- lazy_load() Iterator[Document] [source]#
Loads the query result from Wikipedia into a list of Documents.
- Returns:
- A list of Document objects representing the loaded
Wikipedia pages.
- Return type:
Iterator[Document]
- load_and_split(text_splitter: TextSplitter | None = None) list[Document] #
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
- Parameters:
text_splitter (Optional[TextSplitter]) β TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns:
List of Documents.
- Return type:
list[Document]
Examples using WikipediaLoader