Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions src/benchmark/README-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Benchmark

本目录包含与 AI 基准测试(Bench)及 Tokenless 能力测试相关的全部源码,分为以下两大部分。

## AI Bench Agent

这部分源码为主流 AI 基准测试提供执行 Agent 实现。所有 Agent 均已适配 OpenClaw 平台,并针对各 Bench 中的任务类型做了专项优化,以提升 instance 运行的稳定度与通过率。

目前已覆盖的 Bench 包括:

- **SWEBench** — 面向真实软件工程任务的基准测试,Agent 针对代码理解、定位、修复等流程做了流程编排与容错优化。
- **TerminalBench** — 面向终端交互类任务的基准测试,Agent 对命令执行、输出解析、多步交互场景进行了适配与稳定性加固。
- **ClawEval** — OpenClaw 平台自有评测体系,Agent 覆盖其多种任务类型并做了针对性的 prompt 策略与执行逻辑调优。

后续将持续接入更多主流 Bench,新增的 Agent 实现也将遵循相同的适配与优化范式。

## Tokenless 功能与性能测试

这部分源码用于验证 Tokenless 在不同任务阶段提供的上下文压缩能力,测试分为两个层级:

### 模块能力测试

针对 Tokenless 各压缩模块进行独立的功能与性能验证,覆盖不同任务阶段(如上下文构建、中间推理、结果生成等)的压缩效果与正确性,确保每个模块在单元粒度上达到预期表现。

### 端到端全流程测试

选取典型业务场景,串联完整的任务流程进行端到端验证,检验 Tokenless 在真实工作负载下的整体压缩能力、信息保真度与对最终任务结果的影响,确保全链路协同工作符合预期。

28 changes: 28 additions & 0 deletions src/benchmark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Benchmark

This directory contains all source code related to AI benchmark execution and Tokenless capability testing, organized into the following two major sections.

## AI Bench Agents

This section provides agent implementations for mainstream AI benchmarks. All agents have been adapted to the OpenClaw platform and specifically optimized for the task types within each bench to improve instance execution stability and pass rates.

Currently supported benchmarks include:

- **SWEBench** — A benchmark targeting real-world software engineering tasks. The agent is optimized for code comprehension, fault localization, and patch generation workflows with enhanced orchestration and fault tolerance.
- **TerminalBench** — A benchmark focusing on terminal interaction tasks. The agent is adapted for command execution, output parsing, and multi-step interactive scenarios with stability hardening.
- **ClawEval** — The OpenClaw platform's proprietary evaluation suite. The agent covers its diverse task types with tailored prompt strategies and execution logic tuning.

Additional mainstream benchmarks will be continuously integrated, and new agent implementations will follow the same adaptation and optimization paradigm.

## Tokenless Functional & Performance Testing

This section validates the context compression capabilities provided by Tokenless across different task stages. Tests are organized into two tiers:

### Module-Level Testing

Independent functional and performance verification of each Tokenless compression module, covering compression effectiveness and correctness at various task stages (e.g., context construction, intermediate reasoning, result generation). This ensures each module meets expected performance at the unit level.

### End-to-End Testing

Selected representative business scenarios are used to exercise the complete task pipeline end-to-end, verifying Tokenless's overall compression capability, information fidelity, and impact on final task outcomes under realistic workloads. This ensures the full processing chain operates cohesively as expected.

Loading