# Apple 提出 LLM 函数调用不确定性量化方法

- 来源：Apple Machine Learning Research（RSS）
- 发布时间：2026-07-15 08:00
- AIHOT 分数：52
- AIHOT 链接：https://aihot.virxact.com/items/cmrmll0tt04z1biulyhcdujjt
- 原文链接：https://machinelearning.apple.com/research/uncertainty-quantification-function-calling

## AI 摘要

Apple 机器学习研究团队提出一种针对大语言模型函数调用的不确定性量化方法，旨在提升 LLM 在自主任务执行中的可靠性。该方法通过量化模型对函数调用参数和决策的置信度，帮助识别潜在错误调用。该研究目前处于学术探索阶段，未公布具体模型版本或开源计划。

## 正文

AuthorsZihuiwen Ye†*, Lukas Aichberger†*, Michael Kirchhof, Sinead Williamson, Luca Zappella, Yarin Gal†, Arno Blaas‡, Adam Goliński‡

Large Language Models (LLMs) are increasingly deployed to autonomously solve real-world tasks. A key ingredient for this is the LLM Function-Calling paradigm, a widely used approach for equipping LLMs with tool-use capabilities. However, an LLM calling functions incorrectly can have severe implications, especially when their effects are irreversible, e.g., transferring money or deleting data. Hence, it is of paramount importance to consider the LLM’s confidence that a function call solves the task correctly prior to executing it. Uncertainty Quantification (UQ) methods can be used to quantify this confidence and prevent potentially incorrect function calls. In this work, we present what is, to our knowledge, the first evaluation of UQ methods for LLM Function-Calling (FC). While multi-sample UQ methods, such as Semantic Entropy, show strong performance for natural language Q&A tasks, we find that in the FC setting, it offers no clear advantage over simple single-sample UQ methods. Additionally, we find that the particularities of FC outputs can be leveraged to improve the performance of existing UQ methods in this setting. Specifically, multi-sample UQ methods benefit from clustering FC outputs based on their abstract syntax tree parsing, while single-sample UQ methods can be improved by selecting only semantically meaningful tokens when calculating logit-based uncertainty scores.

† University of Oxford

* Equal contribution

‡ Joint senior authorship

Related readings and updates.

Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results

Uncertainty Quantification (UQ) in Language Models (LMs) is key to improving their safety and reliability. Evaluations often use metrics like AUROC to assess how well UQ methods (e.g., negative sequence probabilities) correlate with task correctness functions (e.g., ROUGE-L). We show that mutual biases—when both UQ methods and correctness functions are biased by the same factors—systematically distort evaluation. First, we formally prove that…

Efficient and Effective Uncertainty Quantification in LLMs

This paper was accepted at the Safe Generative AI Workshop (SGAIW) 2024 at NeurIPS 2024.

Uncertainty quantification (UQ) is crucial for ensuring the safe deployment of large language model, particularly in high-stakes applications where hallucinations can be harmful. However, existing UQ methods often demand substantial computational resources, e.g., multi-sample methods such as Semantic Entropy (Kuhn et al., 2023) usually require 5-10 inference…

Discover opportunities in Machine Learning.