uqlm.black_box.cosine.CosineScorer#
- class uqlm.black_box.cosine.CosineScorer(transformer='all-MiniLM-L6-v2')#
Bases:
SimilarityScorer- __init__(transformer='all-MiniLM-L6-v2')#
Compute cosine similarity betwee original and candidate responses.
- Parameters:
transformer (str (HuggingFace sentence transformer), default='all-MiniLM-L6-v2') – Specifies which huggingface sentence transformer to use when computing cosine distance. See https://huggingface.co/sentence-transformers?sort_models=likes#models for more information. The recommended sentence transformer is ‘all-MiniLM-L6-v2’.
Methods
__init__([transformer])Compute cosine similarity betwee original and candidate responses.
evaluate(responses, sampled_responses)This method computes model-based text similarity metrics values for the provided pairs of texts.
- evaluate(responses, sampled_responses)#
This method computes model-based text similarity metrics values for the provided pairs of texts.
- Return type:
List[float]- Parameters:
responses (list of strings) – Original LLM response
sampled_responses (list of list of strings) – Candidate responses to be compared to the original response
- Returns:
Mean cosine similarity values
- Return type:
List of float
References