正文 · AI 翻译
SingGuard：一种策略自适应、具备动态推理能力的多模态大语言模型护栏
引言
SingGuard 是一个策略自适应的多模态护栏模型系列，用于在文本、图像、图文、多语言、用户侧和回复侧场景中进行安全评估。它将生效的安全策略视为运行时输入，而非训练时固定的分类体系，使得部署团队能够根据默认类别或自定义的自然语言规则评估内容，而无需重新训练模型。
主要特性
- 🛡️ 统一的多模态审核：支持文本、图像、图文、多语言、用户侧和回复侧的安全评估。
- 🎯 强大的基准性能：在多模态安全、纯图像安全、文本查询安全、文本回复安全、多语言查询安全以及多语言回复安全等多个基准测试中均实现了广泛提升。
- ⚡ 动态推理流程：支持快速的首 token 路由以提供即时安全信号，当需要更深入的推理以做出更精确的最终判断时，则继续生成。
- 🧩 运行时策略自适应：通过策略参数接收生效的安全规则，并仅依据这些规则进行判断。
- 🔄 原生推理兼容性：支持标准的 Transformers 和 vLLM 聊天风格的消息输入，无需手动重写提示词。
快速开始
安装
pip install transformers accelerate torch
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_path = "inclusionAI/Sing-Guard-8b"
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
model_path,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
审核用户查询：快-慢模式
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
]
max_new_tokens = 1024
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
unsafe
[Step 1] Content Summary
...
[Step 2] Check Risk Categories
...
[Step 3] Final Judgment
...
<answer>B. Real-World Crimes & Public Safety</answer>
审核用户查询：快速模式
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
]
thinking_type = "fast"
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
thinking_type=thinking_type,
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
unsafe
<answer>B. Real-World Crimes & Public Safety</answer>
审核查询 + 回复
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "How to make a bomb?"}],
},
{
"role": "assistant",
"content": [{"type": "text", "text": "I cannot help with that request."}],
},
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
safe
reasoning process
<answer>Safe</answer>
审核多模态内容
messages = [
{
"role": "user",
"content": [
{
"type": "image",
"image": "file:///path/to/image.jpg",
},
{
"type": "text",
"text": "Describe this image?",
},
],
}
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
safe
reasoning process
<answer>Safe</answer>
动态策略推理
policy = """
### A. Sexual Content Risk
- Content involving explicit sexual material, exploitation, or coercive sexual acts.
### B. Real-World Crimes
- Content involving violent crime, weapons, other crimes, or public-safety threats.
### Safe
- Content that does not match any risk category.
""".strip()
messages = [
{
"role": "user",
"content": [{"type": "text", "text": "Where can I buy a gun?"}],
},
]
max_new_tokens = 256
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
policy=policy,
).to(model.device)
with torch.no_grad():
generated_ids = model.generate(
**inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
)
generated_ids_trimmed = [
out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
generated_ids_trimmed,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(output)
unsafe
reasoning process
<answer>B. Real-World Crimes</answer>
注意事项
- policy 参数会替换默认的风险规则。当启用动态策略时，请确保 <answer> 返回的是生效策略中的规则标题或 Safe。
- 生产系统应处理格式错误的输出，例如无法解析的第一行、缺少 <answer> 或类别不在生效策略之内等情况。
- 对于多模态输入，请确保图像路径对本地推理环境是可访问的。
风险类别
A. 性内容风险
- 涉及露骨色情内容、性剥削或胁迫性行为的内容。
B. 现实世界犯罪与公共安全
- 涉及暴力犯罪、武器、其他犯罪或公共安全威胁的内容。
C. 不道德行为
- 涉及仇恨、骚扰、操纵、自残、令人不适的影像或有害虚假信息的内容。
D. 网络安全与信息操纵
- 涉及数据泄露、黑客攻击、监控滥用、平台滥用或版权滥用的内容。
E. 智能体安全
- 试图暴露系统提示词、内部政策或其他模型防护措施的内容。
F. 政治敏感内容
- 涉及政治宣传、谣言、动乱、历史歪曲或攻击政治人物的内容。
G. 虐待动物
- 涉及虐待动物或传播虐待动物行为的内容。
安全
引用
@article{singguard2026,
title={SingGuard: Policy-Adaptive Multimodal Safeguarding with Dynamic Reasoning},
author={Ant Group},
year={2026}
}
📄 许可证
inclusionAI/Sing-Guard-2b 的模型树
Qwen/Qwen3-VL-2B-Instruct