callbacks#

Callback handlers allow listening to events in LangChain.

Class hierarchy:

BaseCallbackHandler --> <name>CallbackHandler  # Example: AimCallbackHandler

Classes

callbacks.aim_callback.AimCallbackHandler([...])

Callback Handler that logs to Aim.

callbacks.aim_callback.BaseMetadataCallbackHandler()

Callback handler for the metadata and associated function states for callbacks.

callbacks.argilla_callback.ArgillaCallbackHandler(...)

Callback Handler that logs into Argilla.

callbacks.arize_callback.ArizeCallbackHandler([...])

Callback Handler that logs to Arize.

callbacks.arthur_callback.ArthurCallbackHandler(...)

Callback Handler that logs to Arthur platform.

callbacks.bedrock_anthropic_callback.BedrockAnthropicTokenUsageCallbackHandler()

Callback Handler that tracks bedrock anthropic info.

callbacks.clearml_callback.ClearMLCallbackHandler([...])

Callback Handler that logs to ClearML.

callbacks.comet_ml_callback.CometCallbackHandler([...])

Callback Handler that logs to Comet.

callbacks.confident_callback.DeepEvalCallbackHandler(metrics)

Callback Handler that logs into deepeval.

callbacks.context_callback.ContextCallbackHandler([...])

Callback Handler that records transcripts to the Context service.

callbacks.fiddler_callback.FiddlerCallbackHandler(...)

Initialize Fiddler callback handler.

callbacks.flyte_callback.FlyteCallbackHandler()

Callback handler that is used within a Flyte task.

callbacks.human.AsyncHumanApprovalCallbackHandler(...)

Asynchronous callback for manually validating values.

callbacks.human.HumanApprovalCallbackHandler(...)

Callback for manually validating values.

callbacks.human.HumanRejectedException

Exception to raise when a person manually review and rejects a value.

callbacks.infino_callback.InfinoCallbackHandler([...])

Callback Handler that logs to Infino.

callbacks.labelstudio_callback.LabelStudioCallbackHandler([...])

Label Studio callback handler.

callbacks.labelstudio_callback.LabelStudioMode(value)

Label Studio mode enumerator.

callbacks.llmonitor_callback.LLMonitorCallbackHandler([...])

Callback Handler for LLMonitor`.

callbacks.llmonitor_callback.UserContextManager(user_id)

Context manager for LLMonitor user context.

callbacks.mlflow_callback.MlflowCallbackHandler([...])

Callback Handler that logs metrics and artifacts to mlflow server.

callbacks.mlflow_callback.MlflowLogger(**kwargs)

Callback Handler that logs metrics and artifacts to mlflow server.

callbacks.openai_info.OpenAICallbackHandler()

Callback Handler that tracks OpenAI info.

callbacks.promptlayer_callback.PromptLayerCallbackHandler([...])

Callback handler for promptlayer.

callbacks.sagemaker_callback.SageMakerCallbackHandler(run)

Callback Handler that logs prompt artifacts and metrics to SageMaker Experiments.

callbacks.streamlit.mutable_expander.ChildRecord(...)

Child record as a NamedTuple.

callbacks.streamlit.mutable_expander.ChildType(value)

Enumerator of the child type.

callbacks.streamlit.mutable_expander.MutableExpander(...)

Streamlit expander that can be renamed and dynamically expanded/collapsed.

callbacks.streamlit.streamlit_callback_handler.LLMThought(...)

A thought in the LLM's thought stream.

callbacks.streamlit.streamlit_callback_handler.LLMThoughtLabeler()

Generates markdown labels for LLMThought containers.

callbacks.streamlit.streamlit_callback_handler.LLMThoughtState(value)

Enumerator of the LLMThought state.

callbacks.streamlit.streamlit_callback_handler.StreamlitCallbackHandler(...)

Callback handler that writes to a Streamlit app.

callbacks.streamlit.streamlit_callback_handler.ToolRecord(...)

Tool record as a NamedTuple.

callbacks.tracers.comet.CometTracer(**kwargs)

Comet Tracer.

callbacks.tracers.wandb.WandbRunArgs

Arguments for the WandbTracer.

callbacks.tracers.wandb.WandbTracer(...)

Callback Handler that logs to Weights and Biases.

callbacks.trubrics_callback.TrubricsCallbackHandler([...])

Callback handler for Trubrics.

callbacks.upstash_ratelimit_callback.UpstashRatelimitError(...)

Upstash Ratelimit Error

callbacks.upstash_ratelimit_callback.UpstashRatelimitHandler(...)

Callback to handle rate limiting based on the number of requests or the number of tokens in the input.

callbacks.uptrain_callback.UpTrainCallbackHandler(*)

Callback Handler that logs evaluation results to uptrain and the console.

callbacks.uptrain_callback.UpTrainDataSchema(...)

The UpTrain data schema for tracking evaluation results.

callbacks.utils.BaseMetadataCallbackHandler()

Handle the metadata and associated function states for callbacks.

callbacks.wandb_callback.WandbCallbackHandler([...])

Callback Handler that logs to Weights and Biases.

callbacks.whylabs_callback.WhyLabsCallbackHandler(...)

Callback Handler for logging to WhyLabs.

Functions

callbacks.aim_callback.import_aim()

Import the aim python package and raise an error if it is not installed.

callbacks.clearml_callback.import_clearml()

Import the clearml python package and raise an error if it is not installed.

callbacks.comet_ml_callback.import_comet_ml()

Import comet_ml and raise an error if it is not installed.

callbacks.context_callback.import_context()

Import the getcontext package.

callbacks.fiddler_callback.import_fiddler()

Import the fiddler python package and raise an error if it is not installed.

callbacks.flyte_callback.analyze_text(text)

Analyze text using textstat and spacy.

callbacks.flyte_callback.import_flytekit()

Import flytekit and flytekitplugins-deck-standard.

callbacks.infino_callback.get_num_tokens(...)

Calculate num tokens for OpenAI with tiktoken package.

callbacks.infino_callback.import_infino()

Import the infino client.

callbacks.infino_callback.import_tiktoken()

Import tiktoken for counting tokens for OpenAI models.

callbacks.labelstudio_callback.get_default_label_configs(mode)

Get default Label Studio configs for the given mode.

callbacks.llmonitor_callback.identify(user_id)

Builds an LLMonitor UserContextManager

callbacks.manager.get_bedrock_anthropic_callback()

Get the Bedrock anthropic callback handler in a context manager.

callbacks.manager.get_openai_callback()

Get the OpenAI callback handler in a context manager.

callbacks.manager.wandb_tracing_enabled([...])

Get the WandbTracer in a context manager.

callbacks.mlflow_callback.analyze_text(text)

Analyze text using textstat and spacy.

callbacks.mlflow_callback.construct_html_from_prompt_and_generation(...)

Construct an html element from a prompt and a generation.

callbacks.mlflow_callback.get_text_complexity_metrics()

Get the text complexity metrics from textstat.

callbacks.mlflow_callback.import_mlflow()

Import the mlflow python package and raise an error if it is not installed.

callbacks.mlflow_callback.mlflow_callback_metrics()

Get the metrics to log to MLFlow.

callbacks.openai_info.get_openai_token_cost_for_model(...)

Get the cost in USD for a given model and number of tokens.

callbacks.openai_info.standardize_model_name(...)

Standardize the model name to a format that can be used in the OpenAI API.

callbacks.sagemaker_callback.save_json(data, ...)

Save dict to local file path.

callbacks.tracers.comet.import_comet_llm_api()

Import comet_llm api and raise an error if it is not installed.

callbacks.tracers.wandb.build_tree(runs)

Builds a nested dictionary from a list of runs. :param runs: The list of runs to build the tree from. :return: The nested dictionary representing the langchain Run in a tree structure compatible with WBTraceTree.

callbacks.tracers.wandb.flatten_run(run)

Utility to flatten a nest run object into a list of runs.

callbacks.tracers.wandb.modify_serialized_iterative(runs)

Utility to modify the serialized field of a list of runs dictionaries. removes any keys that match the exact_keys and any keys that contain any of the partial_keys. recursively moves the dictionaries under the kwargs key to the top level. changes the "id" field to a string "_kind" field that tells WBTraceTree how to visualize the run. promotes the "serialized" field to the top level. :param runs: The list of runs to modify. :param exact_keys: A tuple of keys to remove from the serialized field. :param partial_keys: A tuple of partial keys to remove from the serialized field. :return: The modified list of runs.

callbacks.tracers.wandb.truncate_run_iterative(runs)

Utility to truncate a list of runs dictionaries to only keep the specified

callbacks.uptrain_callback.import_uptrain()

Import the uptrain package.

callbacks.utils.flatten_dict(nested_dict[, ...])

Flatten a nested dictionary into a flat dictionary.

callbacks.utils.hash_string(s)

Hash a string using sha1.

callbacks.utils.import_pandas()

Import the pandas python package and raise an error if it is not installed.

callbacks.utils.import_spacy()

Import the spacy python package and raise an error if it is not installed.

callbacks.utils.import_textstat()

Import the textstat python package and raise an error if it is not installed.

callbacks.utils.load_json(json_path)

Load json file to a string.

callbacks.wandb_callback.analyze_text(text)

Analyze text using textstat and spacy.

callbacks.wandb_callback.construct_html_from_prompt_and_generation(...)

Construct an html element from a prompt and a generation.

callbacks.wandb_callback.import_wandb()

Import the wandb python package and raise an error if it is not installed.

callbacks.wandb_callback.load_json_to_dict(...)

Load json file to a dictionary.

callbacks.whylabs_callback.import_langkit([...])

Import the langkit python package and raise an error if it is not installed.