uqlm.nli.cluster.SemanticClusterer#
- class uqlm.nli.cluster.SemanticClusterer(nli=None, length_normalize=False)#
Bases:
object- __init__(nli=None, length_normalize=False)#
Methods
__init__([nli, length_normalize])best_response_selection(clustered_responses, ...)Select the best response from the clustered responses based on the cluster probabilities
cluster_responses(responses[, prompt])This method create clusters from a list of responses based on the semantic meaning of each response.
Compute cluster probabilities
compute_response_probabilities(logprobs_results)Compute response probabilities
evaluate(responses[, prompt, ...])Evaluate the cluster of responses.
length_norm_sequence_prob(logprobs[, ...])Compute length normalized sequence logprob
- static best_response_selection(clustered_responses, cluster_probabilities)#
Select the best response from the clustered responses based on the cluster probabilities
- Return type:
str
- cluster_responses(responses, prompt=None)#
This method create clusters from a list of responses based on the semantic meaning of each response.
- Return type:
Any- Parameters:
responses (list of str, default=None) – A list of model responses
prompt (str, default=None) – A prompt for the responses.
- Return type:
A list of lists, where each list represents a cluster.
- compute_cluster_probabilities(response_probabilities, cluster_indices)#
Compute cluster probabilities
- Return type:
List[float]
- compute_response_probabilities(logprobs_results, num_responses=None)#
Compute response probabilities
- Return type:
List[float]
- evaluate(responses, prompt=None, response_probabilities=None)#
Evaluate the cluster of responses.
- Return type:
Tuple[str,List[List[str]],List[float],Dict[Tuple[str,str],float]]
- static length_norm_sequence_prob(logprobs, length_normalize=True)#
Compute length normalized sequence logprob
- Return type:
float
References