resolve_criteria

langchain.evaluation.criteria.eval_chain.resolve_criteria(

criteria: Mapping[str, str] | Criteria | ConstitutionalPrinciple | str | None,

) → dict[str, str]

Resolve the criteria to evaluate.

Parameters:

• criteria (CRITERIA_TYPE) –

  The criteria to evaluate the runs against. It can be:

  • a mapping of a criterion name to its description

  • a single criterion name present in one of the default criteria

  • a single ConstitutionalPrinciple instance

• Returns

• -------

• Dict[str – A dictionary mapping criterion names to descriptions.

• str] – A dictionary mapping criterion names to descriptions.

• Examples

• --------

• "relevance" (>>> criterion =)

• CriteriaEvalChain.resolve_criteria(criteria) (>>>)

• {'relevance' ('Is the submission referring to a real quote from the text?'})

Return type:

dict[str, str]