RedTeam Arena 是一个开源的大语言模型红队测试平台，由 LMSYS 与 Pliny 及 BASI 社区联合推出。平台首发游戏 Bad Words 已吸引数千用户参与，玩家需诱导模型说出特定敏感词汇。团队提出新型统计方法 Extended Elo，通过逻辑回归分别计算玩家、模型和提示词的独立评分，相比传统 Elo 算法实现样本量的二次方级节省。所有代码已开源，数据将在短期延迟后向公众开放。
原文 · 未翻译
Contents
The Leaderboard: Extended Elo
What’s next?
RedTeam Arena: An Open-Source, Community-driven Jailbreaking Platform
We are excited to launch RedTeam Arena, a community-driven redteaming platform, built in collaboration with Pliny and the BASI community!
Figure 1: RedTeam Arena with Bad Words at redarena.ai
RedTeam Arena is an open-source red-teaming platform for LLMs. Our plan is to provide games that people can play to have fun, while sharpening their red-teaming skills. The first game we created is called Bad Words, challenging players to convince models to say target "bad words”. It already has strong community adoption, with thousands of users participating and competing for the top spot on the jailbreaker leaderboard.
We plan to open the data after a short responsible disclosure delay. We hope this data will help the community determine the boundaries of AI models—how they can be controlled and convinced.
This is not a bug bounty program, and it is not your grandma’s jailbreak arena. Our goal is to serve and grow the redteaming community. To make this one of the most massive crowdsourced red teaming initiatives of all time. From our perspective, models that are easily persuaded are not worse: they are just more controllable, and less resistant to persuasion. This can be good or bad depending on your use-case; it’s not black-and-white.
We need your help. Join our jailbreaking game at redarena.ai. All the code is open-sourced on Github. You can open issues and also send feedback on Discord. You are welcome to propose new games, or new bad words on X (just tag @lmsysorg and @elder_plinius so we see it)!
The Leaderboard: Extended Elo
Figure 2. Leaderboard screenshot. Latest version at redarena.ai/leaderboard
People have been asking how we compute the leaderboard of players, models, and prompts. The idea is to treat every round of Bad Words as a 1v1 game between a player and a (prompt, model) combination, and calculate the corresponding Elo score. Doing this naively is sample-inefficient and would result in slow convergence, so we instead designed a new statistical method for this purpose (writeup coming!) and we’ll describe it below.
RedTeam Arena 是一个开源的大语言模型红队测试平台，由 LMSYS 与 Pliny 及 BASI 社区联合推出。平台首发游戏 Bad Words 已吸引数千用户参与，玩家需诱导模型说出特定敏感词汇。团队提出新型统计方法 Extended Elo，通过逻辑回归分别计算玩家、模型和提示词的独立评分，相比传统 Elo 算法实现样本量的二次方级节省。所有代码已开源，数据将在短期延迟后向公众开放。
原文 · 保持原样，未翻译
Contents
The Leaderboard: Extended Elo
What’s next?
RedTeam Arena: An Open-Source, Community-driven Jailbreaking Platform
We are excited to launch RedTeam Arena, a community-driven redteaming platform, built in collaboration with Pliny and the BASI community!
Figure 1: RedTeam Arena with Bad Words at redarena.ai
RedTeam Arena is an open-source red-teaming platform for LLMs. Our plan is to provide games that people can play to have fun, while sharpening their red-teaming skills. The first game we created is called Bad Words, challenging players to convince models to say target "bad words”. It already has strong community adoption, with thousands of users participating and competing for the top spot on the jailbreaker leaderboard.
Observation model. Let TTT be the number of battles (“time-steps”), MMM be the number of models, PPP be the number of players, and RRR be the number of prompts. For each battle i∈[n]i \in [n]i∈[n], we have a player, a model, and a prompt, encoded as following:
XiModel∈{0,1}MX_i^{\rm Model} \in \{0,1\}^MXiModel∈{0,1}M, a one-hot vector with 1 on the entry of the model sampled in battle iii.
XiPlayer∈{0,1}PX_i^{\rm Player} \in \{0,1\}^PXiPlayer∈{0,1}P, a one-hot vector with 1 on the entry of the player in battle iii.
XiPrompt∈{0,1}RX_i^{\rm Prompt} \in \{0,1\}^RXiPrompt∈{0,1}R, a one-hot vector with 1 on the entry of the prompt sampled in battle iii.
Yi∈{0,1}Y_i \in \{0,1\}Yi∈{0,1}, a binary outcome taking the value 1 if the player won (or forfeited) and 0 otherwise.
We then model the win probability of the player as
This form might look familiar, since it is the same type of model as the Arena Score: a logistic model. This is just a logistic model with a different, additive structure—the model scores βModel\beta^{\rm Model}βModel and prompt scores βPrompt\beta^{\rm Prompt}βPrompt combine additively to generate a notion of total strength for the model-prompt pair. The player scores βPlayer\beta^{\rm Player}βPlayer have a similar interpretation as the standard Elo score, and we let β\betaβ denote the concatenation (βPlayer,βModel,βPrompt)(\beta^{\rm Player}, \beta^{\rm Model}, \beta^{\rm Prompt})(βPlayer,βModel,βPrompt). For lack of a better term, we call this model “Extended Elo”.
What problem is this new model solving that the old Elo algorithm couldn’t? The answer is in the efficiency of estimation. The standard Elo algorithm could apply in our setting by simply calling every model-prompt pair a distinct “opponent” for the purposes of calculating the leaderboard. However, this approach has two issues: It cannot disentangle the effectiveness of the prompt versus that of the model. There is a single coefficient for the pair. Instead, extended Elo can assign strength to each subpart. There are M×RM\times RM×R model-prompt pairs, and only M+RM+RM+R distinct models and prompts. Therefore, asymptotically if MMM and RRR grow proportionally, the extended Elo procedure has a quadratic sample-size saving over the standard Elo procedure.
Now, we solve this logistic regression problem online. That is, letting ℓ(x,y;β)\ell(x,y;\beta)ℓ(x,y;β) be the binary cross-entropy loss, we use the iteration
for some learning rate η\etaη. This is a generalization of the Elo update. In fact, if one removes the prompt coefficient, it reduces exactly to the Elo update between players and models, as if these were 1-1 games.
That’s it! After updating the model coefficients in this way, we report them in the tables in the RedTeam Arena. We also have more plans for this approach: extended Elo can be used not just for 1v2 leaderboards, like this one, but any NNNvMMM-player leaderboards in order to attribute notions of strength to each subpart using binary human preference feedback.
What’s next?
RedTeam Arena is a community-driven project, and we’re eager to grow it further with your help! Whether through raising Github issues, creating PRs here, or providing feedback on Discord, we welcome all your contributions!
We plan to open the data after a short responsible disclosure delay. We hope this data will help the community determine the boundaries of AI models—how they can be controlled and convinced.
This is not a bug bounty program, and it is not your grandma’s jailbreak arena. Our goal is to serve and grow the redteaming community. To make this one of the most massive crowdsourced red teaming initiatives of all time. From our perspective, models that are easily persuaded are not worse: they are just more controllable, and less resistant to persuasion. This can be good or bad depending on your use-case; it’s not black-and-white.
We need your help. Join our jailbreaking game at redarena.ai. All the code is open-sourced on Github. You can open issues and also send feedback on Discord. You are welcome to propose new games, or new bad words on X (just tag @lmsysorg and @elder_plinius so we see it)!
The Leaderboard: Extended Elo
Figure 2. Leaderboard screenshot. Latest version at redarena.ai/leaderboard
People have been asking how we compute the leaderboard of players, models, and prompts. The idea is to treat every round of Bad Words as a 1v1 game between a player and a (prompt, model) combination, and calculate the corresponding Elo score. Doing this naively is sample-inefficient and would result in slow convergence, so we instead designed a new statistical method for this purpose (writeup coming!) and we’ll describe it below.
Observation model. Let TTT be the number of battles (“time-steps”), MMM be the number of models, PPP be the number of players, and RRR be the number of prompts. For each battle i∈[n]i \in [n]i∈[n], we have a player, a model, and a prompt, encoded as following:
XiModel∈{0,1}MX_i^{\rm Model} \in \{0,1\}^MXiModel∈{0,1}M, a one-hot vector with 1 on the entry of the model sampled in battle iii.
XiPlayer∈{0,1}PX_i^{\rm Player} \in \{0,1\}^PXiPlayer∈{0,1}P, a one-hot vector with 1 on the entry of the player in battle iii.
XiPrompt∈{0,1}RX_i^{\rm Prompt} \in \{0,1\}^RXiPrompt∈{0,1}R, a one-hot vector with 1 on the entry of the prompt sampled in battle iii.
Yi∈{0,1}Y_i \in \{0,1\}Yi∈{0,1}, a binary outcome taking the value 1 if the player won (or forfeited) and 0 otherwise.
We then model the win probability of the player as
This form might look familiar, since it is the same type of model as the Arena Score: a logistic model. This is just a logistic model with a different, additive structure—the model scores βModel\beta^{\rm Model}βModel and prompt scores βPrompt\beta^{\rm Prompt}βPrompt combine additively to generate a notion of total strength for the model-prompt pair. The player scores βPlayer\beta^{\rm Player}βPlayer have a similar interpretation as the standard Elo score, and we let β\betaβ denote the concatenation (βPlayer,βModel,βPrompt)(\beta^{\rm Player}, \beta^{\rm Model}, \beta^{\rm Prompt})(βPlayer,βModel,βPrompt). For lack of a better term, we call this model “Extended Elo”.
What problem is this new model solving that the old Elo algorithm couldn’t? The answer is in the efficiency of estimation. The standard Elo algorithm could apply in our setting by simply calling every model-prompt pair a distinct “opponent” for the purposes of calculating the leaderboard. However, this approach has two issues: It cannot disentangle the effectiveness of the prompt versus that of the model. There is a single coefficient for the pair. Instead, extended Elo can assign strength to each subpart. There are M×RM\times RM×R model-prompt pairs, and only M+RM+RM+R distinct models and prompts. Therefore, asymptotically if MMM and RRR grow proportionally, the extended Elo procedure has a quadratic sample-size saving over the standard Elo procedure.
Now, we solve this logistic regression problem online. That is, letting ℓ(x,y;β)\ell(x,y;\beta)ℓ(x,y;β) be the binary cross-entropy loss, we use the iteration
for some learning rate η\etaη. This is a generalization of the Elo update. In fact, if one removes the prompt coefficient, it reduces exactly to the Elo update between players and models, as if these were 1-1 games.
That’s it! After updating the model coefficients in this way, we report them in the tables in the RedTeam Arena. We also have more plans for this approach: extended Elo can be used not just for 1v2 leaderboards, like this one, but any NNNvMMM-player leaderboards in order to attribute notions of strength to each subpart using binary human preference feedback.
What’s next?
RedTeam Arena is a community-driven project, and we’re eager to grow it further with your help! Whether through raising Github issues, creating PRs here, or providing feedback on Discord, we welcome all your contributions!