feat: agentic context tools + three-layer e2e#1
Merged
Conversation
Adds 8 new tools focused on agent decision-making context (not raw data
fetching), completes Hermes and OpenClaw integration for the new tools,
strengthens data-source diagnostics, and introduces three layers of
end-to-end tests plus CI wiring for them.
## New tools (agent context, not data)
- diagnose_data_sources: probe which providers work per market
- get_market_capabilities: return supported/unsupported tools per market
- render_stock_report / render_market_report: j2 templates → markdown
- build_watchlist_context: score/trend/anomalies + next_tools per symbol
- analyze_position_context: stateless pnl/risk given cost + quantity
- check_alert_rules: stateless rule evaluation
- parse_stock_list: extract symbols from freeform text
- backtest run: extended diagnostics field (trade_count_quality,
main_failure_reason, regime_fit, suggested_parameter_changes)
## Host integration
- pi/index.ts, hermes/{tools,schemas,__init__,plugin.yaml},
openclaw/{index.ts,openclaw.plugin.json} all register the same 39 tools
- All hosts round-trip identically; integration tests assert three-way parity
- typings/openclaw: shim Array/Boolean for Type surface
## Diagnostics correctness (P2)
- Finnhub now probes the finnhub package (not requests)
- Longbridge probes longport.openapi
- finnhub-python added to tools/requirements.txt
## Distribution (P1c)
- pyproject: jinja2 as hard dep; force-include templates/ and schemas/
- package.json.files: include templates/
- tools/requirements.txt: add jinja2, finnhub-python
## Three-layer e2e
- Layer 1 (host↔python subprocess): 4 non-network tools × 3 hosts
- tests/e2e/test_host_to_python.py (pytest, covers Hermes handlers)
- tests/e2e/test_pi_e2e.ts, test_openclaw_e2e.ts (real subprocess)
- Layer 2 (real provider network): akshare + yfinance + calendar
- tests/e2e/test_provider_network.py, @pytest.mark.integration_network
- Layer 3 (LLM tool_use): DeepSeek → tool_calls → handler → answer
- tests/e2e/test_hermes_llm.py, test_pi_llm.ts
- @pytest.mark.integration_llm; opt-in via DEEPSEEK_API_KEY
- Model overridable via DEEPSEEK_MODEL (default deepseek-v4-flash)
## Pytest markers + defaults
- pyproject: register integration_network and integration_llm markers
- addopts excludes both by default — unit tests never touch network/LLM
## CI (.github/workflows/ci.yml)
- integration job runs Layer 1 e2e (3 hosts) after registration checks
- e2e-network job: Layer 2, always on push/PR
- e2e-llm job: Layer 3, skipped on forks, needs DEEPSEEK_API_KEY secret
## Unit tests
- 8 new test files for the 8 new tools (~75 tests)
- Layer 1 e2e: 8 more (pytest) + 8 (tsx)
- Layer 2 e2e: 4 (marked)
- Layer 3 e2e: 2 pytest + 2 tsx (marked)
- Total: 423 passed, 6 deselected by default markers (1.8s)
## Notes
- integration_network hits real providers and may fail loud on upstream
outages — that's intentional; skip is worse than red
- integration_llm consumes API tokens; only runs where secret is set
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces several new financial analysis tools and templates across the pi, hermes, and openclaw platforms, alongside comprehensive end-to-end integration tests. The review feedback highlights critical stability and robustness issues, including potential runtime crashes from unhandled None values in backtest diagnostics, missing default parameter objects in TypeScript destructuring, and case-sensitivity bugs in argparse CLI choices. Additionally, the reviewer identifies performance concerns with dynamic module imports, a dangerous silent failure mode in alert rule checks, and type-safety issues in Jinja2 macros and equity curve parsing.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Previous prompt leaked the answer (contained "600519" and "AAPL"), so DeepSeek often skipped the tool and answered directly. tool_calls=[] on CI → red. New tests give the LLM only natural-language company names, never the code. The only way to satisfy the ask is to call parse_stock_list: - test_parse_stock_list_english: "Nvidia and Apple" → NVDA + AAPL Fully offline (US ticker regex only), stable across environments. - test_parse_stock_list_chinese: "贵州茅台和宁德时代" → 600519 + 300750 Requires akshare for name resolution. Hard-asserts the LLM called the tool (main e2e contract), soft-asserts specific codes only when akshare actually resolved them — decouples from upstream data outages. Same rewrite applied to tests/e2e/test_pi_llm.ts.
## Applied from Gemini code review
- backtest.py: guard against `None` for exit.stop_loss / exit.take_profit
before `float()` — real crash path if strategy YAML has explicit nulls
- alert_rules.py: surface data-fetch failures in `meta.warnings` instead
of silently returning "not triggered". An alerting system that fails
silently is worse than one that fails loudly
- capabilities.py + diagnostics.py: `argparse` --market now case-insensitive
via `type=str.upper` — matches the get_capabilities() runtime behavior
- diagnostics.py: probe with `importlib.util.find_spec` instead of
`importlib.import_module`. Avoids executing top-level code of heavy
packages (akshare, yfinance) during a "cheap capability probe"
- _macros.j2: `change_pct` also skips non-number values, not just None
## Baostock stdout pollution (real bug surfaced by e2e-network)
- stock_data.py::_kline_baostock: baostock.login()/logout() print status
lines to stdout, corrupting the tool's JSON output. Silence via
`contextlib.redirect_stdout(io.StringIO())`
## Rejected from Gemini review
- pi/index.ts execute() `= {}` default: pi-agent contract already
guarantees args is Record<string, unknown>, never undefined
- backtest.py isinstance(list, tuple): equity_curve is built by
simulate() as list-of-lists; adding tuple support is speculative
- import_parser.py noise dict expansion: not motivated by any observed
false positive; YAGNI
## Test updates
- test_diagnostics.py: mock `importlib.util.find_spec` to match new probe
- All 423 unit tests still pass
deepseek-v4-flash intermittently returns content=null (empty string in our loop) after tool_use — reproduced in CI as `final=""` while tool_calls was correctly populated with parse_stock_list. That's model-side variability, not a broken plugin contract. Previous assertions on `final` were flaky. Rewrite the agent loop to also capture each tool's parsed JSON result, and rewrite all three test assertions to target the tool_use contract: 1. LLM chose the right tool 2. LLM sent args that carried the natural-language input 3. Our tool successfully computed the expected result Concretely: - test_parse_stock_list_english: assert NVDA + AAPL appear in the tool's extracted items across all parse_stock_list calls (not in the LLM's final) - test_parse_stock_list_chinese: assert LLM called the tool; soft-check codes only when akshare actually resolves them (upstream availability decoupled) - test_capabilities_flow: assert the LLM queried HK and the tool marked get_capital_flow as unsupported (not that the LLM's final said "no") Same rewrite for tests/e2e/test_pi_llm.ts. What we lose: verification that the LLM writes a coherent final answer. What we keep: the actual e2e contract we care about — host↔python↔LLM tool_use plumbing works end to end.
Runs ruff --fix + ruff format on every PR. autofix-ci's GitHub App pushes the fixup commit back to the PR branch (bypassing main's branch protection, which only guards main itself). Scope: - pull_request only. push isn't wired because main is protected (bot can't direct-push) and feature branches without a PR don't need autofix - Currently runs ruff on tools/, skills/, tests/, hermes/. TypeScript formatter (prettier) not enabled — no prettier config in the repo yet
Switch from hand-rolled `git log` release notes to GitHub's built-in `--generate-notes`. Categorization is driven by PR labels via `.github/release.yml`. What changes in v0.1.4+ release pages: - Commits grouped by section (New Features / Bug Fixes / Tests & E2E / CI / Docs / Chore / Other) based on PR labels - Each entry links to its PR and @mentions the author - Footer includes a Full Changelog compare link (auto) - Install commands (npm / pip / OpenClaw) appended below Historical commits landing outside PRs fall into "Other Changes" — acceptable, and no longer a concern since main is now branch-protected and everything must go through a PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
diagnose_data_sources,get_market_capabilities,render_stock_report/render_market_report,build_watchlist_context,analyze_position_context,check_alert_rules,parse_stock_list— focused on helping the host agent make decisions, not fetching more raw dataWhat changed
New tools (agent context, not data)
diagnose_data_sourcesget_market_capabilitiesrender_stock_report/render_market_reportbuild_watchlist_contextnext_toolsanalyze_position_contextcheck_alert_rulesparse_stock_listrun_backtest(extended)diagnosticsfield: trade quality, failure reason, regime fit, parameter suggestionsHost integration (39 tools, 3 hosts)
pi/index.ts— 8pi.registerToolcallshermes/tools.py+schemas.py+__init__.py+plugin.yamlopenclaw/index.ts+openclaw.plugin.jsontypings/openclaw/index.d.ts— shimType.Array/Type.BooleanDiagnostics correctness
finnhubpackage (notrequests)longport.openapifinnhub-pythonadded as opt-in depDistribution packaging
pyproject.toml:jinja2>=3.1.0as hard dep; force-includetemplates/andschemas/package.json:templates/added tofilestools/requirements.txt:jinja2,finnhub-pythonThree-layer e2e
Layer 1 — host↔python subprocess (default CI, no network)
tests/e2e/test_host_to_python.py— 4 tools × Hermes handler + subprocess CLItests/e2e/test_pi_e2e.ts— real Python subprocess through Pi's executortests/e2e/test_openclaw_e2e.ts— same, through OpenClaw's default executorLayer 2 — real provider network (
@pytest.mark.integration_network)tests/e2e/test_provider_network.py— akshare + yfinance + exchange-calendarsLayer 3 — LLM tool_use (
@pytest.mark.integration_llm, needsDEEPSEEK_API_KEY)tests/e2e/test_hermes_llm.py,test_pi_llm.tsDEEPSEEK_MODELenv (defaultdeepseek-v4-flash)CI job graph
test: default unit tests (-m "not integration_network and not integration_llm") — 423 passedintegration: registration checks (Hermes/Pi/OpenClaw) + Layer 1 e2ee2e-network: Layer 2, always on push/PRe2e-llm: Layer 3, gated onsecrets.DEEPSEEK_API_KEYbuild: still gated on[lint, test, integration]Test plan
pytest(default): 423 passed, 6 deselected (network/LLM), ~2spytest -m integration_network: 4 passed against real akshare/yfinancenpx tsx tests/e2e/test_pi_e2e.ts: 4 tools OKnpx tsx tests/e2e/test_openclaw_e2e.ts: 4 tools OKpytest -m integration_llmwithDEEPSEEK_API_KEY: 2 passed (verified locally, 2min)npx tsx tests/e2e/test_pi_llm.tswithDEEPSEEK_API_KEY: 2 passed (verified locally)npx tsc --noEmit: cleanruff check+ruff format --check: cleanNotes
integration_llmjob on CI will fail hard if the secret is set but the key is wrong — this is the intended tradeoff for "no fake green"hermes: 39, pi: 39, openclaw: 39)