uqlm.longform.uad.uad.UncertaintyAwareDecoder#

class uqlm.longform.uad.uad.UncertaintyAwareDecoder(reconstructor_llm)#

Bases: object

__init__(reconstructor_llm)#

Class for decomposing responses into individual claims or sentences. This class is used as an intermediate step for longform UQ methods.

Parameters:

reconstructor_llm (langchain BaseChatModel, default=None) – A langchain llm BaseChatModel. User is responsible for specifying temperature and other relevant parameters to the constructor of their llm object.

Methods

__init__(reconstructor_llm)

Class for decomposing responses into individual claims or sentences.

reconstruct_responses(claim_sets, claim_scores)

async reconstruct_responses(claim_sets, claim_scores, responses=None, threshold=0.3333333333333333, progress_bar=None)#
Return type:

List[str]

Parameters:

  • claim_sets (List[List[str]]) – List of original responses decomposed into lists of claims

  • claim_scores (List[List[float]]) – List of lists of claim-level confidence scores to be used for uncertainty-aware filtering

  • threshold (float, default=1/3) – Threshold used for uncertainty-aware filtering

  • progress_bar (rich.progress.Progress, default=None) – If provided, displays a progress bar while scoring responses

References