ExactMatchStringEvaluator#

class langchain.evaluation.exact_match.base.ExactMatchStringEvaluator(*, ignore_case: bool = False, ignore_punctuation: bool = False, ignore_numbers: bool = False, **kwargs: Any)[source]#

Compute an exact match between the prediction and the reference.

Examples

>>> evaluator = ExactMatchChain()
>>> evaluator.evaluate_strings(
        prediction="Mindy is the CTO",
        reference="Mindy is the CTO",
    )  # This will return {'score': 1.0}

>>> evaluator.evaluate_strings(
        prediction="Mindy is the CTO",
        reference="Mindy is the CEO",
    )  # This will return {'score': 0.0}

Attributes

evaluation_name

Get the evaluation name.

input_keys

Get the input keys.

requires_input

This evaluator does not require input.

requires_reference

This evaluator requires a reference.

Methods

__init__(*[, ignore_case, ...])

aevaluate_strings(*, prediction[, ...])

Asynchronously evaluate Chain or LLM output, based on optional input and label.

evaluate_strings(*, prediction[, reference, ...])

Evaluate Chain or LLM output, based on optional input and label.
Parameters:

  • ignore_case (bool)

  • ignore_punctuation (bool)

  • ignore_numbers (bool)

  • kwargs (Any)

__init__(*, ignore_case: bool = False, ignore_punctuation: bool = False, ignore_numbers: bool = False, **kwargs: Any)[source]#
Parameters:

  • ignore_case (bool)

  • ignore_punctuation (bool)

  • ignore_numbers (bool)

  • kwargs (Any)

async aevaluate_strings(*, prediction: str, reference: str | None = None, input: str | None = None, **kwargs: Any) → dict#

Asynchronously evaluate Chain or LLM output, based on optional input and label.

Parameters:

  • prediction (str) – The LLM or chain prediction to evaluate.

  • reference (Optional[str], optional) – The reference label to evaluate against.

  • input (Optional[str], optional) – The input to consider during evaluation.

  • kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.

Returns:

The evaluation results containing the score or value.

Return type:

dict

evaluate_strings(*, prediction: str, reference: str | None = None, input: str | None = None, **kwargs: Any) → dict#

Evaluate Chain or LLM output, based on optional input and label.

Parameters:

  • prediction (str) – The LLM or chain prediction to evaluate.

  • reference (Optional[str], optional) – The reference label to evaluate against.

  • input (Optional[str], optional) – The input to consider during evaluation.

  • kwargs (Any) – Additional keyword arguments, including callbacks, tags, etc.

Returns:

The evaluation results containing the score or value.

Return type:

dict