AgentTrajectoryEvaluator#

class langchain.evaluation.schema.AgentTrajectoryEvaluator[source]#

Interface for evaluating agent trajectories.

Attributes

`requires_input`	Whether this evaluator requires an input string.
`requires_reference`	Whether this evaluator requires a reference label.

Methods

`__init__`()
`aevaluate_agent_trajectory`(*, prediction, ...)	Asynchronously evaluate a trajectory.
`evaluate_agent_trajectory`(*, prediction, ...)	Evaluate a trajectory.

__init__()#

async aevaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: str | None = None, **kwargs: Any) → dict[source]#

Asynchronously evaluate a trajectory.

Parameters:

prediction (str) – The final predicted response.
agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.
input (str) – The input to the agent.
reference (Optional[str]) – The reference answer.
kwargs (Any) –

Returns:

The evaluation result.

Return type:

dict

evaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: str | None = None, **kwargs: Any) → dict[source]#

Evaluate a trajectory.

Parameters:

prediction (str) – The final predicted response.
agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.
input (str) – The input to the agent.
reference (Optional[str]) – The reference answer.
kwargs (Any) –

Returns:

The evaluation result.

Return type:

dict