SeleniumURLLoader#

class langchain_community.document_loaders.url_selenium.SeleniumURLLoader(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: str | None = None, executable_path: str | None = None, headless: bool = True, arguments: List[str] = [])[source]#

Load HTML pages with Selenium and parse with Unstructured.

This is useful for loading pages that require javascript to render.

Parameters:

  • urls (List[str])

  • continue_on_failure (bool)

  • browser (Literal['chrome', 'firefox'])

  • binary_location (str | None)

  • executable_path (str | None)

  • headless (bool)

  • arguments (List[str])

urls#

List of URLs to load.

Type:

List[str]

continue_on_failure#

If True, continue loading other URLs on failure.

Type:

bool

browser#

The browser to use, either ‘chrome’ or ‘firefox’.

Type:

str

binary_location#

The location of the browser binary.

Type:

Optional[str]

executable_path#

The path to the browser executable.

Type:

Optional[str]

headless#

If True, the browser will run in headless mode.

Type:

bool

arguments[List, str]#

List of arguments to pass to the browser.

Load a list of URLs using Selenium and unstructured.

Methods

__init__(urls[, continue_on_failure, ...])

Load a list of URLs using Selenium and unstructured.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

A lazy loader for Documents.

load()

Load the specified URLs using Selenium and create Document instances.

load_and_split([text_splitter])

Load Documents and split into chunks.
__init__(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: str | None = None, executable_path: str | None = None, headless: bool = True, arguments: List[str] = [])[source]#

Load a list of URLs using Selenium and unstructured.

Parameters:

  • urls (List[str])

  • continue_on_failure (bool)

  • browser (Literal['chrome', 'firefox'])

  • binary_location (str | None)

  • executable_path (str | None)

  • headless (bool)

  • arguments (List[str])

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → list[Document]#

Load data into Document objects.

Return type:

list[Document]

lazy_load() → Iterator[Document]#

A lazy loader for Documents.

Return type:

Iterator[Document]

load() → List[Document][source]#

Load the specified URLs using Selenium and create Document instances.

Returns:

A list of Document instances with loaded content.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → list[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

list[Document]

Examples using SeleniumURLLoader