{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 🎯 LLM-as-a-Judge\n", "\n", "
\n", "

\n", " LLM-as-a-Judge scorers use one or more LLMs to evaluate the reliability of the original LLM's response. They offer high customizability through prompt engineering and the choice of judge LLM(s). Below is a list of the available scorers:\n", "

\n", "\n", "* Categorical LLM-as-a-Judge ([Manakul et al., 2023](https://arxiv.org/abs/2303.08896); [Chen & Mueller, 2023](https://arxiv.org/abs/2308.16175); [Luo et al., 2023](https://arxiv.org/pdf/2303.15621))\n", "* Continuous LLM-as-a-Judge ([Xiong et al., 2024](https://arxiv.org/pdf/2306.13063))\n", "* Panel of LLM Judges ([Verga et al., 2024](https://arxiv.org/abs/2404.18796))\n", " \n", "
\n", "\n", "## 📊 What You'll Do in This Demo\n", "\n", "\n", "
\n", "
1
\n", "
\n", "

Set up LLM and prompts.

\n", "

Set up LLM instance and load example data prompts.

\n", "
\n", "
\n", "\n", "
\n", "
2
\n", "
\n", "

Generate LLM Responses and Confidence Scores

\n", "

Generate and score LLM responses to the example questions using the LLMPanel() class.

\n", "
\n", "
\n", "\n", "
\n", "
3
\n", "
\n", "

Evaluate Hallucination Detection Performance

\n", "

Compute precision, recall, and F1-score of hallucination detection.

\n", "
\n", "
\n", "\n", "## ⚖️ Advantages & Limitations\n", "\n", "
\n", "
\n", "

Pros

\n", " \n", "
\n", " \n", "
\n", "

Cons

\n", " \n", "
\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [] }, "outputs": [], "source": [ "import os\n", "from uqlm import LLMPanel\n", "from uqlm.judges import LLMJudge\n", "from uqlm.utils import load_example_dataset, math_postprocessor" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## 1. Set up LLM and Prompts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this demo, we will illustrate this approach using a set of math questions from the [SVAMP benchmark](https://arxiv.org/abs/2103.07191). To implement with your use case, simply **replace the example prompts with your data**. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loading dataset - svamp...\n", "Processing dataset...\n", "Dataset ready!\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
questionanswer
0There are 87 oranges and 290 bananas in Philip...145
1Marco and his dad went strawberry picking. Mar...19
2Edward spent $ 6 to buy 2 books each book cost...3
3Frank was reading through his favorite book. T...198
4There were 78 dollars in Olivia's wallet. She ...63
\n", "
" ], "text/plain": [ " question answer\n", "0 There are 87 oranges and 290 bananas in Philip... 145\n", "1 Marco and his dad went strawberry picking. Mar... 19\n", "2 Edward spent $ 6 to buy 2 books each book cost... 3\n", "3 Frank was reading through his favorite book. T... 198\n", "4 There were 78 dollars in Olivia's wallet. She ... 63" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load example dataset (SVAMP)\n", "svamp = load_example_dataset(\"svamp\", n=75)\n", "svamp.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Define prompts\n", "MATH_INSTRUCTION = \"When you solve this math problem only return the answer with no additional text.\\n\"\n", "prompts = [MATH_INSTRUCTION + prompt for prompt in svamp.question]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, we use `ChatVertexAI` and `AzureChatOpenAI` to instantiate our LLMs, but any [LangChain Chat Model](https://js.langchain.com/docs/integrations/chat/) may be used. Be sure to **replace with your LLM of choice.**" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "tags": [] }, "outputs": [], "source": [ "# import sys\n", "# !{sys.executable} -m pip install python-dotenv\n", "# !{sys.executable} -m pip install langchain-openai\n", "\n", "# # User to populate .env file with API credentials\n", "from dotenv import load_dotenv, find_dotenv\n", "from langchain_openai import AzureChatOpenAI\n", "\n", "load_dotenv(find_dotenv())\n", "original_llm = AzureChatOpenAI(\n", " deployment_name=os.getenv(\"DEPLOYMENT_NAME\"),\n", " openai_api_key=os.getenv(\"API_KEY\"),\n", " azure_endpoint=os.getenv(\"API_BASE\"),\n", " openai_api_type=os.getenv(\"API_TYPE\"),\n", " openai_api_version=os.getenv(\"API_VERSION\"),\n", " temperature=1, # User to set temperature\n", ")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "tags": [] }, "outputs": [], "source": [ "# import sys\n", "# !{sys.executable} -m pip install langchain-google-vertexai\n", "from langchain_google_vertexai import ChatVertexAI\n", "\n", "gemini_pro = ChatVertexAI(model_name=\"gemini-1.5-pro\")\n", "gemini_flash = ChatVertexAI(model_name=\"gemini-1.5-flash\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## 2. Generate responses and confidence scores" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### `LLMPanel()` - Class for aggregating multiple instances of LLMJudge using average, min, max, or majority voting\n", "\n", "![Sample Image](https://raw.githubusercontent.com/cvs-health/uqlm/develop/assets/images/judges_graphic.png)\n", "\n", "#### 📋 Class Attributes\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ParameterType & DefaultDescription
judgeslist of LLMJudge or BaseChatModel
Judges to use. If BaseChatModel, LLMJudge is instantiated using default parameters.
llmBaseChatModel
default=None		A langchain llm `BaseChatModel`. User is responsible for specifying temperature and other relevant parameters to the constructor of the provided `llm` object.
system_promptstr or None
default=\"You are a helpful assistant.\"		Optional argument for user to provide custom system prompt for the LLM.
max_calls_per_minint
default=None		Specifies how many API calls to make per minute to avoid rate limit errors. By default, no limit is specified.
scoring_templatesint
default=None		Specifies which off-the-shelf template to use for each judge. Four off-the-shelf templates offered: incorrect/uncertain/correct (0/0.5/1), incorrect/correct (0/1), continuous score (0 to 1), and likert scale score (1-5 scale, normalized to 0/0.25/0.5/0.75/1). These templates are respectively specified as 'true_false_uncertain', 'true_false', 'continuous', and 'likert'. If specified, must be of equal length to `judges` list. Defaults to 'true_false_uncertain' template used by Chen and Mueller (2023) for each judge.
\n", "\n", "#### 🔍 Parameter Groups\n", "\n", "
\n", "
\n", "

🧠 LLM-Specific

\n", " \n", "
\n", "
\n", "

📊 Confidence Scores

\n", " \n", "
\n", "
\n", "

⚡ Performance

\n", " \n", "
\n", "
\n", "\n", "#### 💻 Usage Examples\n", "\n", "```python\n", "# Basic usage with single self-judge parameters\n", "panel = LLMPanel(llm=llm, judges=[llm])\n", "\n", "# Using two judges with default parameters\n", "panel = LLMPanel(llm=llm, judges=[llm, llm2])\n", "\n", "# Using two judges, one with continuous template\n", "panel = LLMPanel(\n", " llm=llm, judges=[llm, llm2], scoring_templates=['true_false_uncertain', 'continuous']\n", ")\n", "```" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "tags": [] }, "outputs": [], "source": [ "panel = LLMPanel(llm=original_llm, judges=[original_llm, gemini_pro, gemini_flash])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🔄 Class Methods\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MethodDescription & Parameters
LLMPanel.generate_and_score\n", "

Generate responses to provided prompts and use panel to of judges to score responses for correctness.

\n", "

Parameters:

\n", "
    \n", "
  • prompts - (list of str) A list of input prompts for the model.
    • \n", "
\n", "

Returns: UQResult containing data (prompts, responses, sampled responses, and confidence scores) and metadata

\n", "
\n", " 💡 Best For: Complete end-to-end uncertainty quantification when starting with prompts.\n", "
\n", "
LLMPanel.score\n", "

Use panel to of judges to score provided responses for correctness. Use if responses are already generated. Otherwise, use `generate_and_score`.

\n", "

Parameters:

\n", "
    \n", "
  • prompts - (list of str) A list of input prompts for the model.
    • \n", "
  • responses - (list of str) A list of LLM responses for the prompts.
    • \n", "
\n", "

Returns: UQResult containing data (responses and confidence scores) and metadata

\n", "
\n", " 💡 Best For: Computing uncertainty scores when responses are already generated elsewhere.\n", "
\n", "
" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Generating responses...\n", "Generating LLMJudge scores...\n", "Generating LLMJudge scores...\n", "Generating LLMJudge scores...\n" ] } ], "source": [ "result = await panel.generate_and_score(prompts=prompts)\n", "\n", "# option 2: provide pre-generated responses with score method\n", "# result = await panel.score(prompts=prompts, responses=responses)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
promptresponsejudge_1judge_2judge_3avgmaxminmedian
0When you solve this math problem only return t...1451.01.01.01.01.01.01.0
1When you solve this math problem only return t...19 pounds1.01.01.01.01.01.01.0
2When you solve this math problem only return t...$31.01.01.01.01.01.01.0
3When you solve this math problem only return t...1981.01.01.01.01.01.01.0
4When you solve this math problem only return t...631.01.01.01.01.01.01.0
\n", "
" ], "text/plain": [ " prompt response judge_1 \\\n", "0 When you solve this math problem only return t... 145 1.0 \n", "1 When you solve this math problem only return t... 19 pounds 1.0 \n", "2 When you solve this math problem only return t... $3 1.0 \n", "3 When you solve this math problem only return t... 198 1.0 \n", "4 When you solve this math problem only return t... 63 1.0 \n", "\n", " judge_2 judge_3 avg max min median \n", "0 1.0 1.0 1.0 1.0 1.0 1.0 \n", "1 1.0 1.0 1.0 1.0 1.0 1.0 \n", "2 1.0 1.0 1.0 1.0 1.0 1.0 \n", "3 1.0 1.0 1.0 1.0 1.0 1.0 \n", "4 1.0 1.0 1.0 1.0 1.0 1.0 " ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result_df = result.to_df()\n", "result_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## 3. Evaluate Hallucination Detection Performance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To evaluate hallucination detection performance, we 'grade' the responses against an answer key. Note the `math_postprocessor` is specific to our use case (math questions). **If you are using your own prompts/questions, update the grading method accordingly**." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
promptresponsejudge_1judge_2judge_3avgmaxminmediananswerresponse_correct
0When you solve this math problem only return t...1451.01.01.01.01.01.01.0145True
1When you solve this math problem only return t...19 pounds1.01.01.01.01.01.01.019True
2When you solve this math problem only return t...$31.01.01.01.01.01.01.03True
3When you solve this math problem only return t...1981.01.01.01.01.01.01.0198True
4When you solve this math problem only return t...631.01.01.01.01.01.01.063True
\n", "
" ], "text/plain": [ " prompt response judge_1 \\\n", "0 When you solve this math problem only return t... 145 1.0 \n", "1 When you solve this math problem only return t... 19 pounds 1.0 \n", "2 When you solve this math problem only return t... $3 1.0 \n", "3 When you solve this math problem only return t... 198 1.0 \n", "4 When you solve this math problem only return t... 63 1.0 \n", "\n", " judge_2 judge_3 avg max min median answer response_correct \n", "0 1.0 1.0 1.0 1.0 1.0 1.0 145 True \n", "1 1.0 1.0 1.0 1.0 1.0 1.0 19 True \n", "2 1.0 1.0 1.0 1.0 1.0 1.0 3 True \n", "3 1.0 1.0 1.0 1.0 1.0 1.0 198 True \n", "4 1.0 1.0 1.0 1.0 1.0 1.0 63 True " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Populate correct answers and grade responses\n", "result_df[\"answer\"] = svamp.answer\n", "result_df[\"response_correct\"] = [math_postprocessor(r) == a for r, a in zip(result_df[\"response\"], svamp[\"answer\"])]\n", "result_df.head(5)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Judge 1 precision: 0.8620689655172413\n", "Judge 1 recall: 0.78125\n", "Judge 1 f1-score: 0.819672131147541\n", " \n", "Judge 2 precision: 0.9375\n", "Judge 2 recall: 0.9375\n", "Judge 2 f1-score: 0.9375\n", " \n", "Judge 3 precision: 0.9090909090909091\n", "Judge 3 recall: 0.9375\n", "Judge 3 f1-score: 0.9230769230769231\n", " \n" ] } ], "source": [ "# evaluate precision, recall, and f1-score of Semantic Entropy's predictions of correctness\n", "from sklearn.metrics import precision_score, recall_score, f1_score\n", "\n", "for ind in [1, 2, 3]:\n", " y_pred = [(s > 0) * 1 for s in result_df[f\"judge_{str(ind)}\"]]\n", " y_true = result_df.response_correct\n", " print(f\"Judge {ind} precision: {precision_score(y_true=y_true, y_pred=y_pred)}\")\n", " print(f\"Judge {ind} recall: {recall_score(y_true=y_true, y_pred=y_pred)}\")\n", " print(f\"Judge {ind} f1-score: {f1_score(y_true=y_true, y_pred=y_pred)}\")\n", " print(\" \")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## 5. Scorer Definitions\n", "Under the LLM-as-a-Judge approach, either the same LLM that was used for generating the original responses or a different LLM is asked to form a judgment about a pre-generated response. Below, we define two LLM-as-a-Judge scorer templates. \n", "### Categorical Judge Template (`true_false_uncertain`)\n", "We follow the approach proposed by [Chen & Mueller, 2023](https://arxiv.org/abs/2308.16175) in which an LLM is instructed to score a question-answer concatenation as either *incorrect*, *uncertain*, or *correct* using a carefully constructed prompt. These categories are respectively mapped to numerical scores of 0, 0.5, and 1. We denote the LLM-as-a-judge scorers as $J: \\mathcal{Y} \\xrightarrow[]{} \\{0, 0.5, 1\\}$. Formally, we can write this scorer function as follows:\n", "\n", "\\begin{equation}\n", "J(y_i) = \\begin{cases}\n", " 0 & \\text{LLM states response is incorrect} \\\\\n", " 0.5 & \\text{LLM states that it is uncertain} \\\\\n", " 1 & \\text{LLM states response is correct}.\n", "\\end{cases}\n", "\\end{equation}\n", "\n", "### Continuous Judge Template (`continuous`)\n", "For the continuous template, the LLM is asked to directly score a question-answer concatenation's correctness on a scale of 0 to 1. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "© 2025 CVS Health and/or one of its affiliates. All rights reserved." ] } ], "metadata": { "environment": { "kernel": "uqlm_my_test", "name": "workbench-notebooks.m126", "type": "gcloud", "uri": "us-docker.pkg.dev/deeplearning-platform-release/gcr.io/workbench-notebooks:m126" }, "kernelspec": { "display_name": "uqlm_my_test", "language": "python", "name": "uqlm_my_test" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.12" } }, "nbformat": 4, "nbformat_minor": 4 }