ArthurCallbackHandler#

class langchain_community.callbacks.arthur_callback.ArthurCallbackHandler(arthur_model: ArthurModel)[source]#

Callback Handler that logs to Arthur platform.

Arthur helps enterprise teams optimize model operations and performance at scale. The Arthur API tracks model performance, explainability, and fairness across tabular, NLP, and CV models. Our API is model- and platform-agnostic, and continuously scales with complex and dynamic enterprise needs. To learn more about Arthur, visit our website at https://www.arthur.ai/ or read the Arthur docs at https://docs.arthur.ai/

Initialize callback handler.

Attributes

ignore_agent

Whether to ignore agent callbacks.

ignore_chain

Whether to ignore chain callbacks.

ignore_chat_model

Whether to ignore chat model callbacks.

ignore_custom_event

Ignore custom event.

ignore_llm

Whether to ignore LLM callbacks.

ignore_retriever

Whether to ignore retriever callbacks.

ignore_retry

Whether to ignore retry callbacks.

raise_error

Whether to raise an error if an exception occurs.

run_inline

Whether to run the callback inline.

Methods

__init__(arthur_model)

Initialize callback handler.

from_credentials(model_id[, arthur_url, ...])

Initialize callback handler from Arthur credentials.

on_agent_action(action, **kwargs)

Do nothing when agent takes a specific action.

on_agent_finish(finish, **kwargs)

Do nothing

on_chain_end(outputs, **kwargs)

On chain end, do nothing.

on_chain_error(error, **kwargs)

Do nothing when LLM chain outputs an error.

on_chain_start(serialized, inputs, **kwargs)

On chain start, do nothing.

on_chat_model_start(serialized, messages, *, ...)

Run when a chat model starts running.

on_custom_event(name, data, *, run_id[, ...])

Override to define a handler for a custom event.

on_llm_end(response, **kwargs)

On LLM end, send data to Arthur.

on_llm_error(error, **kwargs)

Do nothing when LLM outputs an error.

on_llm_new_token(token, **kwargs)

On new token, pass.

on_llm_start(serialized, prompts, **kwargs)

On LLM start, save the input prompts

on_retriever_end(documents, *, run_id[, ...])

Run when Retriever ends running.

on_retriever_error(error, *, run_id[, ...])

Run when Retriever errors.

on_retriever_start(serialized, query, *, run_id)

Run when the Retriever starts running.

on_retry(retry_state, *, run_id[, parent_run_id])

Run on a retry event.

on_text(text, **kwargs)

Do nothing

on_tool_end(output[, observation_prefix, ...])

Do nothing when tool ends.

on_tool_error(error, **kwargs)

Do nothing when tool outputs an error.

on_tool_start(serialized, input_str, **kwargs)

Do nothing when tool starts.
Parameters:

arthur_model (ArthurModel)

__init__(arthur_model: ArthurModel) → None[source]#

Initialize callback handler.

Parameters:

arthur_model (ArthurModel)

Return type:

None

classmethod from_credentials(model_id: str, arthur_url: str | None = 'https://app.arthur.ai', arthur_login: str | None = None, arthur_password: str | None = None) → ArthurCallbackHandler[source]#

Initialize callback handler from Arthur credentials.

Parameters:

  • model_id (str) – The ID of the arthur model to log to.

  • arthur_url (str, optional) – The URL of the Arthur instance to log to. Defaults to “https://app.arthur.ai”.

  • arthur_login (str, optional) – The login to use to connect to Arthur. Defaults to None.

  • arthur_password (str, optional) – The password to use to connect to Arthur. Defaults to None.

Returns:

The initialized callback handler.

Return type:

ArthurCallbackHandler

on_agent_action(action: AgentAction, **kwargs: Any) → Any[source]#

Do nothing when agent takes a specific action.

Parameters:
Return type:

Any

on_agent_finish(finish: AgentFinish, **kwargs: Any) → None[source]#

Do nothing

Parameters:
Return type:

None

on_chain_end(outputs: Dict[str, Any], **kwargs: Any) → None[source]#

On chain end, do nothing.

Parameters:

  • outputs (Dict[str, Any])

  • kwargs (Any)

Return type:

None

on_chain_error(error: BaseException, **kwargs: Any) → None[source]#

Do nothing when LLM chain outputs an error.

Parameters:

  • error (BaseException)

  • kwargs (Any)

Return type:

None

on_chain_start(serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) → None[source]#

On chain start, do nothing.

Parameters:

  • serialized (Dict[str, Any])

  • inputs (Dict[str, Any])

  • kwargs (Any)

Return type:

None

on_chat_model_start(serialized: dict[str, Any], messages: list[list[BaseMessage]], *, run_id: UUID, parent_run_id: UUID | None = None, tags: list[str] | None = None, metadata: dict[str, Any] | None = None, **kwargs: Any) → Any#

Run when a chat model starts running.

ATTENTION: This method is called for chat models. If you’re implementing

a handler for a non-chat model, you should use on_llm_start instead.

Parameters:

  • serialized (Dict[str, Any]) – The serialized chat model.

  • messages (List[List[BaseMessage]]) – The messages.

  • run_id (UUID) – The run ID. This is the ID of the current run.

  • parent_run_id (UUID) – The parent run ID. This is the ID of the parent run.

  • tags (Optional[List[str]]) – The tags.

  • metadata (Optional[Dict[str, Any]]) – The metadata.

  • kwargs (Any) – Additional keyword arguments.

Return type:

Any

on_custom_event(name: str, data: Any, *, run_id: UUID, tags: list[str] | None = None, metadata: dict[str, Any] | None = None, **kwargs: Any) → Any#

Override to define a handler for a custom event.

Parameters:

  • name (str) – The name of the custom event.

  • data (Any) – The data for the custom event. Format will match the format specified by the user.

  • run_id (UUID) – The ID of the run.

  • tags (list[str] | None) – The tags associated with the custom event (includes inherited tags).

  • metadata (dict[str, Any] | None) – The metadata associated with the custom event (includes inherited metadata).

  • kwargs (Any)

Return type:

Any

Added in version 0.2.15.

on_llm_end(response: LLMResult, **kwargs: Any) → None[source]#

On LLM end, send data to Arthur.

Parameters:
Return type:

None

on_llm_error(error: BaseException, **kwargs: Any) → None[source]#

Do nothing when LLM outputs an error.

Parameters:

  • error (BaseException)

  • kwargs (Any)

Return type:

None

on_llm_new_token(token: str, **kwargs: Any) → None[source]#

On new token, pass.

Parameters:

  • token (str)

  • kwargs (Any)

Return type:

None

on_llm_start(serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) → None[source]#

On LLM start, save the input prompts

Parameters:

  • serialized (Dict[str, Any])

  • prompts (List[str])

  • kwargs (Any)

Return type:

None

on_retriever_end(documents: Sequence[Document], *, run_id: UUID, parent_run_id: UUID | None = None, **kwargs: Any) → Any#

Run when Retriever ends running.

Parameters:

  • documents (Sequence[Document]) – The documents retrieved.

  • run_id (UUID) – The run ID. This is the ID of the current run.

  • parent_run_id (UUID) – The parent run ID. This is the ID of the parent run.

  • kwargs (Any) – Additional keyword arguments.

Return type:

Any

on_retriever_error(error: BaseException, *, run_id: UUID, parent_run_id: UUID | None = None, **kwargs: Any) → Any#

Run when Retriever errors.

Parameters:

  • error (BaseException) – The error that occurred.

  • run_id (UUID) – The run ID. This is the ID of the current run.

  • parent_run_id (UUID) – The parent run ID. This is the ID of the parent run.

  • kwargs (Any) – Additional keyword arguments.

Return type:

Any

on_retriever_start(serialized: dict[str, Any], query: str, *, run_id: UUID, parent_run_id: UUID | None = None, tags: list[str] | None = None, metadata: dict[str, Any] | None = None, **kwargs: Any) → Any#

Run when the Retriever starts running.

Parameters:

  • serialized (Dict[str, Any]) – The serialized Retriever.

  • query (str) – The query.

  • run_id (UUID) – The run ID. This is the ID of the current run.

  • parent_run_id (UUID) – The parent run ID. This is the ID of the parent run.

  • tags (Optional[List[str]]) – The tags.

  • metadata (Optional[Dict[str, Any]]) – The metadata.

  • kwargs (Any) – Additional keyword arguments.

Return type:

Any

on_retry(retry_state: RetryCallState, *, run_id: UUID, parent_run_id: UUID | None = None, **kwargs: Any) → Any#

Run on a retry event.

Parameters:

  • retry_state (RetryCallState) – The retry state.

  • run_id (UUID) – The run ID. This is the ID of the current run.

  • parent_run_id (UUID) – The parent run ID. This is the ID of the parent run.

  • kwargs (Any) – Additional keyword arguments.

Return type:

Any

on_text(text: str, **kwargs: Any) → None[source]#

Do nothing

Parameters:

  • text (str)

  • kwargs (Any)

Return type:

None

on_tool_end(output: Any, observation_prefix: str | None = None, llm_prefix: str | None = None, **kwargs: Any) → None[source]#

Do nothing when tool ends.

Parameters:

  • output (Any)

  • observation_prefix (str | None)

  • llm_prefix (str | None)

  • kwargs (Any)

Return type:

None

on_tool_error(error: BaseException, **kwargs: Any) → None[source]#

Do nothing when tool outputs an error.

Parameters:

  • error (BaseException)

  • kwargs (Any)

Return type:

None

on_tool_start(serialized: Dict[str, Any], input_str: str, **kwargs: Any) → None[source]#

Do nothing when tool starts.

Parameters:

  • serialized (Dict[str, Any])

  • input_str (str)

  • kwargs (Any)

Return type:

None

Examples using ArthurCallbackHandler