refactor(jmle): refresh Rust optimizer ownership RED on current main

[seonghobae]

Controlled current-main replacement for #744 / issue #626

Protected main advanced after #744's exact RED and its OpenCode handoff explicitly required the predecessor base to remain unchanged. Fresh compare proves #744 is four protected-main commits behind and has one unique path only, tests/test_jmle_rust_optimizer_ownership.py. This Draft recreates that exact fail-first contract from current protected main without transferring predecessor checks/reviews.

Intentional RED

The test builds the installed Rust/PyO3 package and requires public JMLE adam, lbfgs, and adam_lbfgs execution under backend="rust" not to call the legacy Python _adam or _lbfgs loops. Valid RED must reach those public optimizer boundaries after normal package setup; setup/import/fixture/changelog failure is not evidence.

Exact identity

• protected base at creation: 8a56d7b1b024e18a5a96a94db4ede1bf6648ead1;
• clean replacement head: 386fe8ca52ba9def121e050993cdb5317f37c184;
• changed path: tests/test_jmle_rust_optimizer_ownership.py only.

Minimum GREEN

Move result-affecting Adam state/update/gradient clipping, L-BFGS history/two-loop/curvature/line-search, and adam_lbfgs sequencing/convergence into mlsirm-core; expose thin PyO3 transport and keep Python to bounded validation/marshalling/reporting. Preserve public optimizer/model/seed/restart/objective/trace semantics; missing compatible Rust capability fails closed. Add direct parity plus aligned bias/MAE/RMSE/convergence recovery, docs/rustdoc/APA traceability and changelog after GREEN, then all applicable exact-head Python 3.12/3.14, Rust/PyO3, package/reinstall, GPU/fuzz, Security/SAST and current-head governance gates.

Do not widen to MMLE, hosted state, providers, dependencies/workflows/version/release or parallel canonical docs. Advances #626. Supersedes #744 once established.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 844c7afa-f228-4108-b32c-5c319377232c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@opencode-agent address

Exact bounded source-writer lease for Draft #751 / issue #626. Before any source mutation, refetch this PR and independently resolve protected main; proceed only while the source head is still 386fe8ca52ba9def121e050993cdb5317f37c184, protected main is still 8a56d7b1b024e18a5a96a94db4ede1bf6648ead1, the branch remains a direct one-commit descendant with only tests/test_jmle_rust_optimizer_ownership.py, and no other source writer has appeared. If any identity or writer state moves, stop writing this branch and leave a bounded handoff rather than racing.

This branch is intentional installed-package RED. Preserve the fail-first commit/history. CI/Security/SAST are currently queued and are not acceptance evidence. The RED is valid only if normal Rust/PyO3 build and backend resolution succeed and the three optimizer cases fail because public JMLE still executes legacy Python _adam / _lbfgs arithmetic; setup/import/fixture/changelog failure is invalid evidence.

Implement the minimum GREEN only:

1. Move result-affecting JMLE optimizer arithmetic/control for adam, lbfgs, and adam_lbfgs into mlsirm-core: gradient clipping; Adam moments, bias correction and updates; L-BFGS history/two-loop recursion, curvature handling and line-search/backtracking; and phase sequencing/convergence state needed to preserve public results.
2. Expose the thinnest stable PyO3 transport. Python may perform bounded validation, deterministic seed/input marshalling, immutable result construction and reporting only. Missing/incompatible Rust optimizer capability must fail closed; do not add SciPy or another Python production fallback.
3. Preserve current public optimizer identities, model/parameter blocks, objective/gradient estimand, seeds/restarts, clipping/search constants, statuses/traces and restart ranking unless a separately test-first defect proves a correction is required.
4. Add direct Rust↔public parity/property tests and realistic identified/aligned recovery for all optimizer modes using bias, MAE/RMSE and convergence evidence; correlation alone is insufficient. Preserve caller-array immutability and stable non-reflective failures.
5. Keep CPU f64 as the reference and use bounded low-context-switch execution. Do not add a GPU optimizer path without benchmark evidence; existing GPU objective/gradient support does not prove optimizer parity.
6. Add complete rustdoc/public docstrings and APA-7 equation/source doctoring after GREEN. Add the authoritative changelog fragment only after the feature path is green, then render/check the managed changelog using the repository tool.
7. Do not widen into MMLE, observed-information architecture: migrate observed-information and covariance numerics from Python to Rust #625/test(inference): require Rust-owned observed information #734, fixed-anchor linking architecture: move fixed-anchor linking arithmetic into Rust #662/refactor(linking): refresh fixed-anchor Rust ownership RED on current main #752, ATA architecture: move CAT item information and test-assembly selection into Rust #629/refactor(ata): rebase fixed-form Rust ownership RED on current main #750, hosted state, providers, dependencies/workflows, version/release, or a competing canonical documentation line. docs: establish canonical product and architecture baseline #604/docs: close canonical documentation fitness gaps without parallel authority #621 exclusively own cross-cutting PRD/TRD/Architecture/ADR/UML/ERD/traceability.
8. Before stopping this branch, run focused ownership/parity/recovery tests and the full applicable Python 3.12/3.14, Rust/PyO3, package/reinstall, existing GPU/fuzz, Security Scan and SAST gates on one unchanged head; prove exact changed-production statement/branch coverage and current-head review state. Keep Draft while any gate/finding remains.

Do not merge or mark Ready. The maintainer loop will refetch exact-head/live-base evidence and governance after your final source commit.

[seonghobae]

Superseded by GREEN #760 (Rust-owned Adam/L-BFGS).