uqlm.black_box.match.MatchScorer

class uqlm.black_box.match.MatchScorer

Bases: SimilarityScorer

__init__()

Class for computing exact match rates between original responses and candidates. This method is based on Cole et al.(2023) [1].

Methods

__init__()	Class for computing exact match rates between original responses and candidates.
evaluate(responses, sampled_responses)	This method computes exact match rates for the provided pairs of texts.

evaluate(responses, sampled_responses)

This method computes exact match rates for the provided pairs of texts.

Return type:

List[float]

Parameters:

• responses (list of strings) – Original LLM response

• sampled_responses (list of list of strings) – Candidate responses to be compared to the original response

Returns:

Exact match rates

Return type:

List of float

References

[1]

Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. 2023. URL: https://arxiv.org/abs/2305.14613, arXiv:2305.14613.