# Ventor-QTest：面向第三方托管大模型 API 的威胁模型驱动审计

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-17 08:00
- AIHOT 分数：54
- AIHOT 链接：https://aihot.virxact.com/items/cmsy2y6810imrroz0nx7hvgfd
- 原文链接：https://arxiv.org/abs/2608.16391

## AI 摘要

Ventor-QTest 提出一种无需目标 API 概率信息的复合黑盒审计方法，通过重复请求与长序列组件分别报告平均保真度损失（AFL）和极端保真度损失（EFL）。在七组路由快照中，EFL 随任务暴露增加与 Terminal-Bench 通过率下降同步出现，提示审计长程智能体任务时应联合报告 AFL 与 EFL。实现已开源。

## 正文

As large language models become increasingly widespread, third-party providers that deploy open-weight models have become an important part of the ecosystem. Auditing the quality of their inference APIs is therefore an open problem. We formalize hosted model routing as a stochastic process and propose \textbf{Ventor-QTest}, a composite black-box audit that requires no probability information from the target API. Its repeated-request component sends each frozen constrained context to the target multiple times, reconstructs a categorical output distribution from the returned text counts, and reports average fidelity loss (AFL) as a null-bias-corrected, within-window mean coarsened-KL statistic. Its long-sequence component uses independent runs to report extreme fidelity loss (EFL) through the empirical upper tail of a run-level reference-centered-surprisal statistic. Across three logprob-capable route conditions, AFL shows strong linear descriptive agreement with a logprob-derived coarsened-KL comparator. Across seven route snapshots, 20-run sequence probes reveal route-specific EFL variation. AFL and EFL have little detectable route-level association with GPQA-Diamond accuracy. In contrast, pronounced EFL coincides with a decline in Terminal-Bench pass rate as task exposure increases. This pattern may arise because correctness in long-horizon tasks is more sensitive to extreme fidelity loss. These results motivate reporting AFL and EFL jointly, particularly when auditing long-horizon agentic tasks. The open-source implementation is available at https://github.com/Tencent/AI-Infra-Guard/tree/main/services/api_checker/ventor_qtest.