Skip to content

fix(agent): 恢复流式工具参数截断检测 + 空参数告警,强化 execute_code 描述 - #560

Open
ncw1992120 wants to merge 2 commits into
mateaix:devfrom
ncw1992120:fix/sanitize-tool-args-truncation-detection
Open

fix(agent): 恢复流式工具参数截断检测 + 空参数告警,强化 execute_code 描述#560
ncw1992120 wants to merge 2 commits into
mateaix:devfrom
ncw1992120:fix/sanitize-tool-args-truncation-detection

Conversation

@ncw1992120

Copy link
Copy Markdown
Contributor

背景

#557 的后续修复(服务端参数处理 + 模型引导),与文档修复 PR #559 分离为独立关注点。

#557 的生产日志定性显示 No code supplied 失败有两个并列根因:

  • 80% 模型端:模型发出合法 JSON 但缺 code(如 {"language":"python"}
  • 20% 服务端吞参:流式聚合把非法/空白参数静默替换成 {}

本 PR 同时针对这两点。

改动

1. 恢复流式截断检测(#557 评论里的「第三选项」)

sanitizeToolCallArguments 此前在流式聚合点buildFinalToolCalls)把任何非法/空白参数都替换成 {}。由于 {} 是合法 JSON,工具执行器的截断检测(ToolExecutionExecutor JSON 预校验 → 「缩短重发」指引)在流式路径上从不触发——是死代码,模型只能收到笼统的「缺参数」错误。

现在两个调用点行为分离:

调用点 行为
流式聚合buildFinalToolCallsaggregationPoint=true 截断/非法片段原样透传,让执行器检测到截断并向模型返回「缩短重发」指引;空白参数仍归一为 {}新增 WARN(此前静默,是观测盲区)
出站咽喉点normalizeToolCallArgumentsaggregationPoint=false 不变——历史里的工具调用仍在每次请求前归一为 {},aliyun-codingplan 等严格 provider 不会 HTTP 400。持久化的片段在下次出站时被重新归一,历史保护不变

安全性(已核实):执行器在 ToolGuard 与实际执行之前就拦下非法 JSON,透传的片段永远不会被执行;cacheReasoningContent/ConversationWindowManager/ToolLoopGuard 等下游消费者只用 id 或带 try/catch,不受影响。

2. 强化 execute_code 工具描述(治 80% 模型端诱因)

coding-plan 类模型对长 code 参数有省略倾向。在 @Tool description 追加显式 IMPORTANT 警告,并强化 code 参数说明(schema 层 @JsonPropertyDescription 同步),直接干预模型行为。

测试

  • 更新聚合截断测试:断言截断片段原样透传(不再被替换成 {}
  • 新增咽喉点测试:锁定回放的历史截断参数仍归一为 {}(保护不变)
  • NodeStreamingChatHelperToolCallArgsTest 6 例全过;CodeExecuteTool*TestSkillScriptExecutionServiceCodeTest 不受影响

验收标准(上线后同窗口日志)

  1. No code supplied 频次显著下降
  2. 不再有静默 {}——每次参数异常都有带 toolName 的 WARN 可查
  3. execute_code 失败样本中「有 language 缺 code」占比下降

改动范围

 NodeStreamingChatHelper.java            | 54 ++++++++++++++++-------
 CodeExecuteTool.java                    |  9 +++-
 NodeStreamingChatHelperToolCallArgsTest | 43 ++++++++++++++----
 3 files changed, 86 insertions(+), 20 deletions(-)

关联文档修复:#559

…ol args

Two server-side fixes for tool-call argument handling, follow-up to the
research-paper-writing SKILL.md doc fix. Both address the "No code supplied"
failure class where the model's tool-call arguments go missing.

1. Re-enable streaming truncation detection (the "third option" from mateaix#557).
   sanitizeToolCallArguments previously replaced ANY invalid/blank arguments
   with "{}" at the stream-aggregation point (buildFinalToolCalls). Because
   "{}" is valid JSON, the tool executor's truncation detector
   (ToolExecutionExecutor JSON pre-validation -> "shorten and retry" guidance)
   never fired on the streaming path — it was dead code, and the model only
   ever saw a generic per-tool "missing argument" error.

   Now the two call sites behave differently:
   - Stream aggregation (buildFinalToolCalls): a truncated/invalid fragment is
     passed through verbatim so the executor detects the truncation and returns
     the model actionable "shorten and retry" guidance. Blank args still become
     "{}" but are now logged (previously silent — an observation blind spot).
   - Outbound chokepoint (normalizeToolCallArguments): unchanged — history tool
     calls are still harmonized to "{}" before every provider request, so strict
     providers (aliyun-codingplan) never get HTTP 400. The persisted fragment is
     re-normalized on the next send, so history stays protected.

2. Strengthen the execute_code tool description so coding-plan models stop
   omitting the required "code" parameter (the 80% model-side root cause in
   mateaix#557's production-log analysis): an explicit IMPORTANT warning in the @tool
   description plus a stronger "code" parameter description.

Tests: updated the aggregation truncation test to expect raw pass-through, and
added a chokepoint test locking in that replayed truncated args still normalize
to "{}". All 6 NodeStreamingChatHelperToolCallArgsTest cases pass; CodeExecuteTool
and SkillScriptExecutionService code tests unaffected.

Refs: mateaix#557
@ncw1992120

Copy link
Copy Markdown
Contributor Author

评审:实现正确,可合;一条非阻塞备注

#557 评论中「第三选项」的四个验证面逐点核查:

1. 行为分离正确 ✅

  • 聚合点(aggregationPoint=true):非法片段原样透传、WARN 措辞同步;空白参数归一 {} 并新增 WARN——此前的观测盲区(「模型发空」vs「分片全丢」无法区分)已消灭
  • 出站咽喉点(aggregationPoint=false):行为不变,严格 provider 的 HTTP 400 保护保持

2. 架构前提成立 ✅
全仓只有一条 LLM 发送路径:NodeStreamingChatHelper:1187chatModel.stream(prompt),且类 Javadoc 明确要求所有节点统一走此 helper、禁止散落 chatModel.call()。出站归一化(streamCallInternal 入口)不会被绕过,透传的截断片段在下次请求前必被重新归一。

3. 安全声明属实 ✅

  • 执行器 JSON 预校验(ToolExecutionExecutor:535-547)在 ToolGuard 与实际执行之前拦截,透传片段永远不会被执行
  • ToolLoopGuard.canonicalizeArguments 的 Javadoc 明确容忍不可解析 JSON(fallback 到原串 hash),下游消费者声明抽查通过

4. 测试迁移完整 ✅
截断透传断言(聚合点)+ 新增「历史回放截断参数仍归一 {}」的咽喉点锁定测试 + tool_call id 保留断言——正是行为分离需要锁定的三个面。

非阻塞备注:截断指引的模式匹配有覆盖缺口

normalizeToolExecutionError 靠消息子串识别截断(unexpected end-of-input / json parse error / malformed json 等 5 种)。流中截断的主场景(mid-token cut → Jackson "Unexpected end-of-input")能命中;但如 {"a":} 这类畸形片段会产生 "Unexpected character ..." 类消息,不在模式表内——模型将收到通用错误而非「缩短重发」指引。不影响安全(执行器照样在执行前拦截),建议后续小 PR 给模式表补 unexpected character,或改为「JSON 预校验失败即走截断指引」(该 catch 分支本就只有 JSON 解析失败一种入口,无需再靠消息猜类型)。

上线后建议按 #557 的三条验收标准回看同窗口日志。

…d) JSON

Address PR mateaix#560 review note (non-blocking): the JSON pre-validation gate in
ToolExecutionExecutor.execute relied on normalizeToolExecutionError, which
pattern-matches the parser message to detect truncation. A mid-token cut hits
"Unexpected end-of-input" (covered), but a malformed fragment like {"a":}
produces "Unexpected character ..." which is NOT in the pattern table — so the
model got a generic error instead of the actionable "shorten and retry"
guidance.

Since that gate wraps only readTree(arguments), ANY exception reaching it is a
JSON parse failure whose remedy is always "re-emit the call". So:

- Extract the guidance into a TRUNCATED_ARGS_GUIDANCE constant.
- The pre-validation gate now returns it unconditionally (no message guessing),
  closing the gap for every malformation phrasing.
- normalizeToolExecutionError keeps pattern-matching for the genuine
  tool-execution error sites (parallel/single-tool catch blocks), which have
  many unrelated causes; it now reuses the same constant. Made package-private
  static for direct unit testing (same style as capToolCalls).

Add ToolExecutionExecutorTruncationGuidanceTest: guidance is actionable, the
gate covers malformed fragments the pattern table misses, and pattern
regression for the other call sites (covered messages, generic fall-through,
workspace path violation, null handling).

Refs: mateaix#557
@ncw1992120

Copy link
Copy Markdown
Contributor Author

非阻塞备注已落实(commit cab1446f

采用建议的方案 (b)——「JSON 预校验失败即走截断指引」:

  • 提取 TRUNCATED_ARGS_GUIDANCE 常量。
  • 预校验 catch 块execute 里包 readTree(arguments) 的那处)现在无条件返回该指引,不再靠消息子串猜类型。因为该 catch 只可能捕获 JSON 解析失败——mid-token 截断("Unexpected end-of-input")或畸形片段({"a":} → "Unexpected character")都一样,补救方式都是「重发」。这样彻底覆盖了模式表的缺口。
  • normalizeToolExecutionError 保留模式匹配,服务于真正的工具执行错误站点(并行/单工具 catch 块,那些异常有多种无关成因);改为复用同一常量,并改成包私有 static 以便直接单测(同 capToolCalls 风格)。

新增 ToolExecutionExecutorTruncationGuidanceTest(6 例):指引可执行性、预校验对模式表漏掉的畸形片段也返回指引、以及其余站点的模式回归(覆盖消息、通用兜底、workspace 路径违规、null 处理)。全过。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant