症状:模型在自己的内心独白里打转

长推理里反复出现 其实… 等等… 嗯…让我重新想想… 这类自我推翻的过渡句。 模型困在自己的内心独白里打转,迟迟不去读文件、跑命令、采数据——这就是社区所说的“雷霆思考”:思考强度拉满,但效率暴跌。

复现:tau-bench 量化测试

用 tau-bench(retail,12 任务 × 5 轮,temperature 0.0)量化:

引擎 触发振荡的任务 振荡标记总数
vLLM AWQ 7/12 26
llama Q6 7/11 15

两个引擎触发率相当——问题与引擎,是概率性的,但6bit量化症状比4bit稍轻

根因:缺信息时不停空想

根据个人的使用体验,当模型拿到工具/代码实据时果断,信息缺少时容易钻牛角尖,陷入循环。模型在缺信息时不停空想是主要诱因,振荡句(等等 其实 重新想)只是症状,但 token 级压制症状仍然有效,比提示词限制效果可靠地多。

解决方案:opencode 配置注入 logit_bias

llama.cpp 的 /v1 端点接受请求级 logit_bias(token id → 偏置)。AI SDK 的 openai-compatible provider 会把 models.<id>.options 里的未知字段透传进请求体,所以只需在 opencode 里给模型加一条 options,无需改服务端:

"qwen3.8-27b-uncensored-q6-mtp-oscfix": {
  "tool_call": true, "reasoning": true,
  "limit": { "context": 524288, "output": 32768 },
  "options": {
    // logit_bias 按 token id 指定;-999 = 禁用。只压句首大写过渡词
    "logit_bias": {
      "13784": -999, // Wait
      "50821": -999, // Actually
      "77264": -999, // Hmm
      "45847": -999  // Hold
    },
    "dry_multiplier": 2.0
  }
}

实测:logit_bias 精确到达请求体,句首 Wait/Actually/Hmm 被根除。dry_multiplier 额外打断重复序列。注意偏置对整个响应生效(含最终回答),无法限定在 reasoning 阶段。

实测对比:从空转到直接行动

以下均为 tau-bench retail 任务的模型原始 reasoning(temperature 0.0)。

案例 1:T恤库存查询——从 22 次自我推翻到 4 次直接行动

:同一轮里连续自我推翻,"到底先鉴权还是先答产品问题"打转:

Wait, the policy says I need to authenticate the user identity first. But the user is just asking for product information. Let me re-read the policy...

Hmm, but the user is just asking a general product question. However, the policy says...

Actually, wait - let me reconsider... Actually, let me re-read: ... So I need to authenticate first. Let me ask for their email. But wait - the user said 'I'll keep it brief...'

Hmm, but actually, maybe I should just look up the product first...

Wait, let me reconsider again.

:直接行动,不再空转:

Let me start by listing all product types to find the tshirt product id.

I found the T-Shirt product with product id "9523456873". Now let me get the product details to see how many options/items are available.

案例 2:T恤浏览——禁用词被替代,但行为倾向仍在

(振荡出现的那轮,原文):

I should note that the user hasn't been authenticated yet. But the user is just asking a general product question, not about their profile or orders. Let me re-read the policy...

"At the beginning of the conversation, you have to authenticate the user identity by locating their user id via email, or via name + zip code."

Hmm, but the user is just asking a general product question. The policy says "At the beginning of the conversation, you have to authenticate the user identity." But the user hasn't given me their email or zip code yet...

Actually, looking at this more carefully, the user is asking a general p...

(振荡词被禁用后出现换词替代 But wait/However/Let me reconsider——压掉特定词但行为倾向仍在):

...However, the policy says "At the beginning of the conversation, you have to authenticate the user identity by locating their user id via email, or via name + zip code."

But wait - the user is asking a general product question. Let me re-read the policy... The user is asking about t-shirt options - this is a general product question, not about their own profile or orders. But the policy says authentication is required at the beginning of the conversation.

Let me think about this more carefully... So I should ask for their email or name + zip to authenticate first. But actually, let me reconsider - the user is asking a general product question... (这一轮振荡标记 14 个,禁用词被 But/However/reconsider 替代,未减少)

案例 3:换水瓶+台灯——从 357 词到 180 词,振荡标记减半

Let me think about this...

Actually, the user id format is like 'sara_doe_496'. So "mei_kovacs_8020" looks like a user id. But the policy says I need to authenticate via email or name + zip code... (前段全轨迹 357 词 / 振荡标记 13)

The user wants to exchange two items from a recent order: a water bottle and a desk lamp. Before I can help, I need to:

  1. First, authenticate the user by finding their user ID
  2. Then look up their order details
  3. Then proceed with the exchange

Let me start by asking for their email to authenticate them. (后段全轨迹 180 词 / 振荡标记 5,同配置同轮数严格配对)

注意事项:

  1. token 变体wait wait(小写带前导空格)Wait 是三个不同 token id。只压大写无空格形式会漏网,模型换词替代(But wait)。如果严格必须列全变体,但只禁句首已经可以显著提升。
  2. vLLM 尚不支持 logit_bias--spec-method mtp 下直接 400 拒绝。