feat(v3.2): RPG-style architecture-first planning behind ATLAS_RPG_PLANNING (#120)#124
Merged
Merged
Conversation
added 2 commits
June 10, 2026 11:11
…ANNING (itigges22#120) Adds an opt-in repository-level planning phase that plans the architecture up front and then fills each piece, instead of free-form natural-language planning. It builds on the existing problem-level PlanSearch rather than replacing it: the Repository Planning Graph decides what to build and how the files, signatures, and data flows fit together, and PlanSearch fills in each node whose interface is already pinned. The work draws on two papers. RPG (arXiv:2509.16198) supplies the repository planning graph and the two-stage proposal-then-implementation construction. PlanSearch (arXiv:2409.03733), already implemented in this repo, supplies the per-node algorithmic search. The wavelet substrate is a faithful, dependency free Python port of wavescope-mcp by Dmitri Sotnikov, whose numeric output matches the upstream TypeScript bit for bit under the golden-fixture tests. Everything is gated by ATLAS_RPG_PLANNING and defaults to off. With the flag off, planning and generation behave exactly as before, since the flat planner emits no constraints and the new code paths short-circuit. What landed, by phase: - Wavelet substrate (v3-service/wavelet): structural signal, Ricker CWT, the fine/medium/coarse bands, project decomposition, and the peak-diff drift detector, all pure stdlib. - RPG artifact and two-stage planning (v3-service/rpg.py): capability tree then files, signatures, and edges, with validation, scoring, and a topological projection back onto the existing flat Plan so the agent loop is untouched. - Graph-guided generation: each node's planned interface threads into its /v3/generate call (proxy/rpg.go, proxy/types.go), so PlanSearch fills a node whose architecture is fixed. - Structural verification and drift: the candidate veto now rejects code that does not realize its planned signatures, and on write the proxy runs one bounded corrective regeneration, then surfaces any surviving drift along with the affected downstream subgraph. - Offline evaluation: v3-service/rpg_eval.py scores RPG artifacts for CI and benchmark summaries. The live RepoCraft comparison needs a GPU and is left as a documented runbook; the default stays off until that evidence lands. Design and phase-by-phase status live in docs/reports/RPG_WAVELET_PLANNING_V3_2.md. Tests: 152 Python tests under tests/v3-service and the proxy Go suite, all green. Credit for the idea and framing goes to Dmitri Sotnikov (@yogthos).
A review of the feature turned up two problems that made it inoperative in the default Docker deployment, plus several correctness gaps. This fixes all of them. The flag never reached the v3-service container, so the whole feature was dead when a user set ATLAS_RPG_PLANNING in their .env. It is now forwarded in docker-compose.yml, along with ATLAS_PLAN_THINKING. The proposal stage seeded its coarse band by scanning working_dir off disk, but the v3-service container has no project volume mount, so that scan always found nothing in Docker. It now builds the coarse band from the in-memory project_context the proxy already sends, through a new decompose_file_map entry point, and falls back to a disk scan only when no context was passed. The plan result is a single SSE line, and with RPG on it now carries the whole graph. The reader buffer was 1MB, so a large graph could trip ErrTooLong and silently drop the plan. The buffer is raised to 16MB and a scanner error is now surfaced instead of being reported as a missing result. Signature parsing mishandled Go receiver methods like func (s *S) Load(), and the bare-name fallback would return the keyword func, which produced a false missing-signature veto. Parsing now recognizes receiver methods and never treats a lone declaration keyword as a function name, and defined_names picks up receiver methods too. The edit_file and ast_edit path threaded RPG constraints but never ran the drift retry or reporting that the write path does, so drift on edits was invisible. It now runs the same regenerateOnDrift and reportRPGDrift. planConstraintsForTarget returned the first overlapping step, so a bare-basename step could shadow the deeper path it was not for. It now prefers the most specific match. The corrective regeneration accepted a retry only when the total miss count dropped, which discarded a retry that fixed the targeted signature but traded in a different one; it now accepts a retry that resolves any of the signatures it was correcting for. The topological sort dedups file ids so a duplicate cannot fake acyclicity or drop a file, and node_drift gained a threshold so its drift score actually gates the re-plan decision. Tests: 163 Python tests and the proxy Go suite, all green.
yogthos
marked this pull request as ready for review
June 10, 2026 17:58
Contributor
Author
|
The only thing to note here is that I haven't tested against a small model like 9b. So, while things should work mechanically as expected, it's worth testing to make sure a small model can work with the output well. |
Owner
|
Marking blocked (research-track) per the experimental-isolation policy — not a rejection of the direction. Before merge consideration this needs: rebase on |
Five integration files conflicted; the RPG/wavelet code itself lands clean. Resolutions: - test.yml: dev's matrix already includes tests/v3-service (the PR's intent); dev's comment kept. - docker-compose.yml: dev's v3-service env kept, ATLAS_RPG_PLANNING passthrough added (dev already carries ATLAS_PLAN_THINKING). - proxy/types.go: V3GenerateResponse keeps dev's VerificationEvidence and gains RPGSignatureMissing. - v3-service/Dockerfile: dev's structured_log/graph COPYs kept, wavelet/ + rpg.py COPYs added. - v3-service/main.py: pipeline.run() signature merges dev's build_command/working_dir with the PR's constraints; the RPG signature veto now runs after dev's call-graph veto; the generate handler keeps dev's ClientDisconnected guard and passes constraints. - proxy/rpg.go: callV3GenerateStreaming gained a context.Context first arg on dev (cancellation cascade); the regen call now passes ctx.Ctx.
…t parsing
Adversarial review of the merged branch (three parallel reviewers, every
finding re-verified against the code before fixing):
proxy/rpg.go:
- The drift-regen accept rule could adopt a strictly worse retry: prev
missing [A,B], retry missing [B,C,D] resolved one targeted signature
and was accepted with more total drift. The first branch now also
requires the total not to grow.
- planConstraintsForTarget gains the RPG-artifact trust gate: the flat
planner passes the LLM's plan dict through verbatim, so hallucinated
step constraints must not steer generation while the flag is off.
Also ranks ALL file-producing steps so a deeper unconstrained step is
not shadowed by a shallower constrained one.
- countOverlap counts distinct names (duplicate service entries no
longer double-count); nil-ctx guard; rpg_regen stage gets a matching
EvtStageEnd; rpg_drift is an EvtMetric point event; affectedDownstream
follows data_flow edges only, as documented.
proxy/tools.go:
- Cancellation recheck after the regen retry on both the write_file and
improveContentWithV3 paths — a user cancel during the second V3 window
must not land content on disk.
- Drift reports gate on req.Constraints so a rogue response field alone
cannot fire them.
v3-service/rpg.py:
- Signature-name extraction no longer returns modifiers as names
(public/const/virtual...); C-style 'type name(args)' and JS
arrow/function-expression assignments extract the real name. A wrong
name made every candidate look drifted forever, each false drift
costing a full V3 regeneration.
- defined_names sees arrow functions, function expressions, and C-style
method bodies.
- The no-parseable-definitions escape now keys on extraction confidence:
a confidently-parsed Python file that defines nothing genuinely misses
every planned def (the old escape kept all-wrong candidates while
vetoing half-right ones); regex-path opacity still skips enforcement.
- extract_json_object falls back to the full text when the first fenced
block is not JSON.
v3-service/main.py:
- Signature-veto events are emitted only when the prune actually
applies; the per-candidate check is guarded like the sibling
call-graph veto.
v3-service/wavelet/:
- String masking no longer runs over comment interiors: an apostrophe
in prose ("/* Don't use */") swallowed the terminator and zeroed the
rest of the file's signal. Same for docstring closers, and a line
comment can no longer flip the docstring state machine or python's
block-comment path. Backtick is not a string delimiter in python.
- Project walk: depth guard (RecursionError), gitignore character
classes match, symlink aliases cannot resurrect ignored dirs.
- decompose_file_map/diff_contents enforce the same size caps as the
disk path (the pure-python CWT makes uncapped input a multi-minute
stall); zero-magnitude coefficients are never peaks.
Regression tests for every fix (proxy: 6 new/updated; python: 12 new).
…entry The event-contract gate requires every proxy chat event to have a TUI handler: rpg_drift and rpg_regen render as system rows alongside the asset_lint handler merged from dev. ATLAS_RPG_PLANNING joins the config schema as a bool so the .env.example key validates.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Pre-bind the RPG imports in generate_plan so a partial import failure
can never leave a name unbound on the guarded paths
(py/uninitialized-local-variable x3).
- Bound every ambiguous quantifier in the signature/definition regexes
({0,16} whitespace, {0,200} param bodies) — the arrow-function pattern
was polynomial on long space runs (py/polynomial-redos); verified sub-
10ms on a 40k-char adversarial input with behavior preserved.
- Walker containment guards in wavelet/project.py: normpath + root-
prefix check on joined paths and the gitignore read — free defense-in-
depth in the sanitizer shape taint tracking recognizes
(py/path-injection x6).
- Comment the best-effort empty except (py/empty-except).
The round-1 pass bounded three of the four signature regexes; the polynomial-redos tracker moved to the surviving unbounded one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in repository-level planning phase that plans the architecture up front and then fills each piece, instead of free-form natural-language planning. It builds on the existing problem-level
PlanSearchrather than replacing it: the Repository Planning Graph decides what to build and how the files, signatures, and data flows fit together, andPlanSearchfills in each node whose interface is already pinned.The work draws on two papers. RPG (arXiv:2509.16198) supplies the repository planning graph and the two-stage proposal-then-implementation construction. PlanSearch (arXiv:2409.03733), already implemented in this repo, supplies the per-node algorithmic search. The wavelet substrate is a faithful, dependency free Python port of wavescope-mcp, whose numeric output matches the upstream TypeScript under the fixture tests.
Everything is gated by
ATLAS_RPG_PLANNINGand defaults to off. With the flag off, planning and generation behave exactly as before, since the flat planner emits no constraints and the new code paths short-circuit.What the feature adds:
There is some offline evaluation still needed using v3-service/rpg_eval.py to score RPG artifacts for CI and benchmark summaries.