RSSFeedLoader#

class langchain_community.document_loaders.rss.RSSFeedLoader(urls: Sequence[str] | None = None, opml: str | None = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any)[source]#

Load news articles from RSS feeds using Unstructured.

Parameters:

  • urls (Sequence[str] | None) – URLs for RSS feeds to load. Each articles in the feed is loaded into its own document.

  • opml (str | None) – OPML file to load feed urls from. Only one of urls or opml should be provided. The value

  • string (can be a URL) –

  • string. (or OPML markup contents as byte or) –

  • continue_on_failure (bool) – If True, continue loading documents even if loading fails for a particular URL.

  • show_progress_bar (bool) – If True, use tqdm to show a loading progress bar. Requires tqdm to be installed, pip install tqdm.

  • **newsloader_kwargs (Any) – Any additional named arguments to pass to NewsURLLoader.

Example

from langchain_community.document_loaders import RSSFeedLoader

loader = RSSFeedLoader(
    urls=["<url-1>", "<url-2>"],
)
docs = loader.load()

The loader uses feedparser to parse RSS feeds. The feedparser library is not installed by default so you should install it if using this loader: https://pythonhosted.org/feedparser/

If you use OPML, you should also install listparser: https://pythonhosted.org/listparser/

Finally, newspaper is used to process each article: https://newspaper.readthedocs.io/en/latest/

Initialize with urls or OPML.

Methods

__init__([urls, opml, continue_on_failure, ...])

Initialize with urls or OPML.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

A lazy loader for Documents.

load()

Load data into Document objects.

load_and_split([text_splitter])

Load Documents and split into chunks.
__init__(urls: Sequence[str] | None = None, opml: str | None = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any) → None[source]#

Initialize with urls or OPML.

Parameters:

  • urls (Sequence[str] | None) –

  • opml (str | None) –

  • continue_on_failure (bool) –

  • show_progress_bar (bool) –

  • newsloader_kwargs (Any) –

Return type:

None

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

lazy_load() → Iterator[Document][source]#

A lazy loader for Documents.

Return type:

Iterator[Document]

load() → List[Document][source]#

Load data into Document objects.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → List[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

List[Document]

Examples using RSSFeedLoader

On this page