uqlm.white_box.single_logprobs.SingleLogprobsScorer#

class uqlm.white_box.single_logprobs.SingleLogprobsScorer(scorers=['normalized_probability', 'min_probability', 'sequence_probability'], length_normalize=True)#

Bases: LogprobsScorer

__init__(scorers=['normalized_probability', 'min_probability', 'sequence_probability'], length_normalize=True)#

Class for computing WhiteBox UQ scores with a single generation

Parameters:

  • scorers (List[str], default=SAMPLED_LOGPROBS_SCORER_NAMES) – Specifies which scorers to compute. Must be a subset of [“semantic_negentropy”, “semantic_density”, “monte_carlo_probability”, “consistency_and_confidence”].

  • length_normalize (bool, default=True) – Specifies whether to length normalize the logprobs. This attribute affect the response probability computation for three scorers (semantic_negentropy, semantic_density, and monte_carlo_probability).

Methods

__init__([scorers, length_normalize])

Class for computing WhiteBox UQ scores with a single generation

evaluate(logprobs_results)

Compute scores from logprobs results

extract_logprobs(single_response_logprobs)

Extract log probabilities from token data

extract_probs(single_response_logprobs)

Extract probabilities from token data

extract_top_logprobs(single_response_logprobs)

Extract top log probabilities for each token
evaluate(logprobs_results)#

Compute scores from logprobs results

Return type:

Dict[str, List[float]]

static extract_logprobs(single_response_logprobs)#

Extract log probabilities from token data

Return type:

ndarray

extract_probs(single_response_logprobs)#

Extract probabilities from token data

Return type:

ndarray

static extract_top_logprobs(single_response_logprobs)#

Extract top log probabilities for each token

Return type:

List[ndarray]

References