Marginal (MMLE-EM) estimation for the latent-space family: multigroup/multilevel/FIPC, QMC-EM, GPU E-step, Rust scoring & fit statistics, paper-batch features#160
Conversation
…ltigroup/multilevel structures and a wgpu E-step Model-design PR (paper basis in docs/mmle_marginal_lsirm_design.md and docs/papers/mmle-lsirm-formula-compilation.md): - crates/mlsirm-core/src/marginal.rs — deterministic Bock-Aitkin-style marginal EM for MIRT/MLS2PLM/MLSRM/ULS2PLM/ULSRM. Person latents (theta, xi) integrated over Gauss-Hermite grids, feasible via the simple-structure conditional factorization (Q_xi^K * sum_d Q_theta, not Q^(1+D+K)). Fisher-preconditioned GEM M-step with Armijo line search; MAP penalties default to the Jeon et al. (2021) priors (PenaltyConfig::lsirm_prior). Multigroup (Bock-Zimowski) group means/SDs with pinned reference group; multilevel (Fox-Glas) random intercept with estimated sigma_u. PCA-aligned zeta/xi for the rotation/reflection invariance. - crates/mlsirm-core/src/quadrature.rs — embedded hermegauss tables (7..41 nodes), bit-identical to the NumPy reference. - crates/mlsirm-core/src/gpu_marginal.rs — wgpu f32 E-step kernels (lp/nbar/item passes, race-free slot ownership); M-step and final EAP stay CPU f64. Measured 110s -> ~5s per multilevel E-step iteration on a 31k x 57 dataset (RTX 3050 Ti). - python/fast_mlsirm/estimators/marginal.py — NumPy mirror; parity with the Rust core at 1e-9 after full EM runs (tests/test_marginal_parity.py). - fit(group_id=..., cluster_id=...) drives the population structures; FitResult.population carries mu/sigma/sigma_u/u_eap/icc/theta_sd and save_fit_result persists them. CLI fit gains --estimator/--group-id/ --cluster-id/--q-theta/--q-xi/--q-u/--tolerance. - Plain ULS2PLM/ULSRM keep the legacy fast path (unchanged behavior); spatial models under estimator="mmle" now fit instead of raising. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nd serving bundle - python/fast_mlsirm/fitstats.py — Orlando-Thissen S-X2 with the Lord-Wingersky recursion generalized to the joint (theta, xi) quadrature grid (per-dimension summed scores, expected-count collapsing, chi-square p via a no-SciPy regularized upper incomplete gamma), Benjamini-Hochberg FDR, Drasgow l_z and Snijders l_z* with the MAP r_0 correction at EAP estimates, infit/outfit at the marginal EAPs, and select_items(): the literature-grounded fit -> flag -> remove -> refit loop (sparse / S-X2-BH / MSQ band / low discrimination / map isolation flags, person-fit screen, per-dimension item floor, full audit trail). - python/fast_mlsirm/serving.py — schema-versioned JSON serving bundle (frozen item parameters + population block + screening audit) and score_respondents(): EAP scoring of new response payloads (dict or dense) against the frozen bundle, the same fixed-parameter pattern as the downstream importance-assessment API. - estimators/marginal.py gains score_eap() (one E-step pass, no updates); public API re-exports; tests for chi2_sf/BH/Lord-Wingersky against enumeration, S-X2 flagging a scrambled item, l_z* calibration and aberrant-person detection, screening pipeline behavior, and bundle round-trip/scoring monotonicity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…set; add `fast-mlsirm score` `fast-mlsirm score --bundle b.json --responses r.json` scores new respondents against a frozen serving bundle (JSON code->0/1 payloads or a .npy matrix), mirroring the downstream fixed-parameter serving flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… FIPC and concurrent calibration All numeric compute now lives in the Rust core; Python keeps thin wrappers plus the NumPy parity references. Paper basis: Part II of docs/papers/mmle-lsirm-formula-compilation.md (Wei-Tanner 1990, Booth-Hobert 1999, Jank 2005, Meng-Schilling 1996, Bock-Mislevy 1982, Thissen et al. 1995, Lord-Wingersky 1984 via Cai 2015, Kim-Cohen 1998, Hanson-Beguin 2002, Kim 2006, Sinharay-Haberman 2014). - mlsirm-core/nodes.rs: shared latent-space node sets — tensor GH, Halton QMC (+ Cranley-Patterson shift), seeded MC; Acklam inverse normal CDF. Bit- mirrored in the NumPy reference (parity ~1e-15 after full EM runs). - mlsirm-core/scoring.rs: ItemBank scoring — EAP, damped-Newton MAP with observed-information SEs, EAPsum tables via Lord-Wingersky; per-dimension N(mean, sd^2) priors cover single/multigroup/multilevel serving. - mlsirm-core/fitstats.rs: S-X2 (+ rms_residual effect size), BH, lz/lz*, infit/outfit; chi-square tail via regularized upper incomplete gamma. - marginal.rs: xi_rule/xi_points/xi_seed; Anchors (FIPC) with frozen items and optional frozen tau; PopulationSpec::SingleFree (free mu/sigma, anchor-identified); GPU guard for oversized QMC node sets. - select_items methodology hardening after the first real-data run over-pruned 45/57 items: S-X2 flag now requires a practical effect size, the MSQ gate uses infit only, the person screen is threshold-configurable and prior-mean centered. - fast-mlsirm score/serving: method="eap"|"map"|"eapsum", prior override for known-team/known-group conditioning, eapsum_tables embedded in the bundle. - Tests: 263 pytest + 43 cargo, incl. QMC/MC backend parity at 1e-9, FIPC anchor freezing + population recovery, concurrent-calibration recovery under structural missingness, EAPsum monotonicity, MAP SEs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dation gates, DIF, Vuong, Q3/GDDM, IRTree expansion, information criteria
Implements the implementable core of the supplied literature set:
- Perumean-Chaney et al. (2013) -> zero-inflated marginal mixture
(FitConfig(zero_inflation=True)): structural-zero class with EM-estimated
pi, responsibilities folded into the E-step weights (GPU kernels
untouched); works under single/multigroup/multilevel populations.
- Debeer & Janssen (2013) -> context-varying item covariate with one
estimated coefficient (fit(covariate={"w", "init_delta"})): the linear
item-position effect with booklet groups as contexts; Newton coordinate in
the M-step; identification guards (single-context covariate rejected).
- Williamson, Xi & Breyer (2012) -> mlsirm_core::agreement + validate_judge:
QWK/kappa, Pearson r, SMD, degradation-vs-human-human and subgroup-SMD
conjunctive acceptance gates with the paper's thresholds.
- Jeon, Rijmen & Rabe-Hesketh (2013) + Makransky & Glas (2013) ->
dif_analysis(): LR DIF screen via group-specific virtual items with all
other items anchored, BH-FDR over studied items, logit effect sizes.
- Schneider, Chalmers, Debelak & Merkle (2019) -> vuong_nonnested(): Vuong
z from casewise logliks with optional Schwarz correction (erfc-based
normal tail in the core).
- Svetina & Levy (2014) -> dimensionality_residuals(): Yen Q3 and GDDM from
EAP residuals in the Rust core.
- Jeon & De Boeck (2016) -> irtree_expand(): mapping-matrix pseudo-item
expansion (their Eq. 9 equivalence makes IRTrees ordinary binary IRT on
the expanded matrix; off-path nodes reuse NaN missingness).
- Kang, Cohen & Sung (2009) -> information_criteria in the core
(AIC/BIC/AICc/SABIC/CAIC + free-parameter counting incl. anchors/ZI/
covariate), surfaced as FitResult.ic and in fit_summary.json.
ZI and covariate are mirrored in the NumPy reference (parity at 1e-9);
Wolkowitz & Skorupski (2013) documented as superseded by marginal-ML MAR
handling; Ferrando et al. (2009) and Joubert et al. (2015) documented as
out-of-scope for binary judge data (specs in docs-research/group_*_specs.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…in 2017) mlsirm_core::oakes::observed_information_oakes assembles the observed (penalized) information at the MML solution as the M-step Hessian at the fixed posterior (central FD of the analytic Q-gradient, no E-steps) plus the Oakes cross term (one CPU f64 E-step per parameter) — the estimator Pritikin (2017) recommends over the supplemented-EM family. SEs cover the item-side parameters and tau, conditional on the population parameters; exposed as fast_mlsirm.oakes_standard_errors(result, responses, factor_id, config). Internal-consistency test: the Oakes assembly matches the full central difference of the marginal score on probe coordinates. Serving bundles now carry pi_zero / covariate_delta in the population block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… specs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…irwise fit, resampling person fit, TCC drift; corpus triage map Implemented from the third supplied corpus (triage in docs/papers/corpus-triage-batch3.md): - Magis (2013): 4PL item information (reduces to 2PL at c=0/d=1); bank_information() item/test information at arbitrary trait points. - Bock & Mislevy (1982) + Wang, Kuo & Chao (2010): cat_next_item() adaptive EAP step over a frozen serving bundle — targets the trait dimension with the largest posterior SD, ranks unadministered items by information. - Marsman et al. (2016): plausible_values() seeded posterior draws from the scoring grid for secondary analyses. - Haberman, Sinharay & Chon (2013): residual_item_fit() standardized EAP-bin residual fit (long-test regime documented; S-X2 for short tests). - Tay & Drasgow (2012): adjusted_chi2_pairs() N=3000-adjusted pairwise chi2/df ratios for local-dependence screening. - Sinharay (2016): person_fit_resampling() parametric-bootstrap empirical p-values for l_z* at the EAP estimates. - Guo, Zheng & Chang (2015): tcc_drift() stepwise TCC drift detection between two same-scale calibrations. All compute in mlsirm-core (scoring.rs / fitstats.rs) with PyO3 wrappers and bundle-level Python APIs; 3PL/4PL estimation and the BIFAC2PLM bifactor variant are scoped as the next model-design PRs in the triage map, and the polytomous/CDM/Bayesian clusters are dispositioned there with reasons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… corpus triage ld_indices() computes the signed standardized pairwise LD X2/G2 against the model-implied joint probabilities on the scoring grid — the classic local-dependence screen, complementing Q3/GDDM and the Tay-Drasgow adjusted ratios. docs/papers/corpus-triage-batch4.md dispositions the fourth reading set and consolidates the explicitly-requested roadmap (BIFAC2PLM bifactor, M2/RMSEA2, 3PL/4PL estimation, polytomous kernels, linking utilities, response times) in priority order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headd1311b2ccc0db7233f20e578e8af9eff15a6280d. -
Head SHA:
d1311b2ccc0db7233f20e578e8af9eff15a6280d -
Workflow run: 29311144089
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (25 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (25 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (8 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (8 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (7 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (7 files)"]
R3 --> V3["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage evidence job did not run or did not publish coverage evidence. Changed-File Evidence Mapflowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
|
…zi et al. 2015); batch-5 corpus triage empirical_reliability() reports the marginal Var(EAP)/(Var(EAP)+mean(SE^2)) per trait dimension for marginal fits, with the sources' caveat that the coefficient presumes a well-fitting model. corpus-triage-batch5.md dispositions the fifth reading set and settles the BIFAC2PLM design as an inner-product interaction kind on the existing eta plumbing (tables/GPU kernels unchanged) — the next model-design PR, now requested across three batches, followed by M2, general mixtures, 3PL/4PL, and polytomous kernels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head35d343f22785b15e90a729f0bbcd375fc642837a. -
Head SHA:
35d343f22785b15e90a729f0bbcd375fc642837a -
Workflow run: 29311601402
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (25 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (25 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (9 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (9 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (7 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (7 files)"]
R3 --> V3["targeted test run"]
Implements the full-information dichotomous bifactor model (Gibbons &
Hedeker 1992; Cai, Yang & Hansen 2011) — requested across three literature
batches — as a third interaction kind on the marginal engine:
- InteractionKind::{None, Distance, Inner} parameterizes every eta site
(tables, M-step gradients with d eta/d zeta_k = x_k, scoring EAP/MAP/
information, fit statistics, Oakes SEs); tau is inert for the inner kind
and excluded from the parameter vector/counts.
- BIFAC2PLM = inner kind at latent_dim >= 1: the general factor rides the
existing conditional-factorization E-step (the Gibbons-Hedeker dimension
reduction), so tables and the wgpu kernels carry over unchanged; at
latent_dim = 1, zeta_i is the general-factor loading lambda_i.
- Positive-manifold loading initialization (a mixed-sign circle init locks
sign-split local optima); marginal estimator only — the JML path guards.
- Recovery tests (Rust + Python) and 1e-9 Rust/NumPy parity; empirical
reliability, EAPsum, CAT, plausible values, and the fit-statistic stack
all operate on BIFAC2PLM banks through the shared kind dispatch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head4e8a143e70c8e21e7cb99c40387ce5479ebe1594. -
Head SHA:
4e8a143e70c8e21e7cb99c40387ce5479ebe1594 -
Workflow run: 29312112905
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (25 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (25 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (9 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (9 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (7 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (7 files)"]
R3 --> V3["targeted test run"]
Add M2 (Maydeu-Olivares & Joe 2005/2006; Cai & Hansen 2013) on the univariate + bivariate residual margins: statistic, df, chi-square p-value, the RMSEA2 approximate-fit index with a 90% noncentral-chi-square confidence interval, and the bivariate SRMSR (Maydeu-Olivares 2013). Every model-implied margin and the up-to-4th-order entries of the multinomial residual covariance Xi_2 are exact via the local-independence factorization over the (theta, xi) node set -- pi_S = sum_c w_c prod P_i(c) -- the same factorization the E-step uses (Cai-Hansen dimension reduction). Delta_2 is central-differenced from the node moments; the quadratic form is evaluated through one Cholesky of Xi_2 (never an explicit inverse): M2 = N ( e'Xi^-1 e - g'(D'Xi^-1 D)^-1 g ), g = D'Xi^-1 e. Kind-aware Rust core (mlsirm_core::fitstats::m2_rmsea2) is the compute path; a NumPy reference (fast_mlsirm.fitstats.m2 / _m2_numpy) is held to 1e-6 parity. Calibration tests in both suites contrast a well-specified fit (RMSEA2 < 0.03) against injected local dependence (RMSEA2 > 0.08). Batch-6 corpus triage; M2/RMSEA2 moves from the roadmap to done. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add mlsirm_core::linking (+ fast_mlsirm.irt_link): put a separately- calibrated new form onto the reference scale from common items, returning theta_old = A*theta_new + B. Covers the moment methods (mean/mean, mean/sigma) and the characteristic-curve methods of Haebara (1980) and Stocking & Lord (1983) for the unidimensional 2PL/1PL common-item case those procedures reduce to (Kolen & Brennan 2014). New-form items transform onto the old scale in the engine's eta = a*theta+b form as a* = a_new/A, b* = b_new - (a_new/A)*B; the characteristic-curve loss is minimized by a self-contained Nelder-Mead from the mean/sigma start, integrated over a standard-normal Gauss-Hermite grid. Rust compute path; four-method recovery tests (Rust + Python) recover a known transform. Motivated by the corpus linking papers (Kim & Lee 2006; Yao & Boughton 2009; Brossman & Lee 2013). Complements link_fixed_item_parameters and the FIPC serving path. Batch-6 triage updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head7130c3de86611ce91b8ce4734fc9ac70a8a19abe. -
Head SHA:
7130c3de86611ce91b8ce4734fc9ac70a8a19abe -
Workflow run: 29314322214
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (27 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (27 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (10 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (10 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (7 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (7 files)"]
R3 --> V3["targeted test run"]
Add branch tests for the new code so its error/edge paths are exercised under the lib test suite (not only the integration recovery test): - linking: LinkMethod::parse (all arms + unknown), the moment zero-spread fallback, the cc-objective non-positive/NaN guard, Nelder-Mead contraction/shrink on a non-smooth objective, and the irt_link input guards (bad slopes, empty/mismatched grid). - M2: the too-few-items / length-mismatch / non-positive-df / too-few-complete-cases guards, plus a small hand-built-bank run that exercises the Cholesky, Delta, Xi, CI, and SRMSR body directly. Docstrings added to the new Python helpers (M2Result, _MutBank, _m2_numpy, _ncchi2_cdf, _nc_lambda_for, n_dims_of, IrtLinkResult) for the review gate's docstring-evidence check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address the Strix scan findings on PR #160 — DoS / data-poisoning guards for serving/fitting entry points that may take untrusted data: - VULN-0001: compact + validate population labels (group_id/cluster_id) in fit.py and inference.py. n_groups/n_clusters is now the distinct-label count (<= n_persons), not max+1, so sparse ids like [0, 1e9] can't force billion-row allocations; negative/non-integer/non-finite/wrong-length ids are rejected. - VULN-0004/0005: FitConfig.validate bounds latent_dim (<= 8) and xi_points (<= 1_000_000). - VULN-0006/0007: load_serving_bundle parses JSON strictly (no NaN/Infinity) and runs _validate_bundle (consistent, bounded dimensions; in-range factor_id; finite alpha/b/zeta/tau/eps_distance; supported quadrature); score_respondents and plausible_values validate the bundle at entry. - VULN-0002: plausible_values enforces the 0/1 finite response domain that score_respondents already required. - VULN-0003: validate_judge validates labels (1-D, equal length, finite, integer, 0<=label<k) before the uint32 conversion. Regression tests in tests/test_security_hardening.py (19) cover every finding; existing serving/scoring/estimator suites unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add unit tests for previously-uncovered defensive branches toward the 100%-line coverage gate: gh_rule None (unsupported size) and the Halton high-latent-dim / shift paths in nodes; XiRuleKind::parse (all arms + unknown) in marginal; validate_bank's four error branches and the y/observed length guard via score_eap in scoring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second Strix pass on PR #160. Close the remaining input-validation gaps (the earlier pass bounded only latent_dim/xi_points): - VULN-0004: FitConfig.validate now rejects non-finite learning_rate, init_gamma, eps_distance, tolerance, and gradient_clip (a bare `x <= 0` lets NaN/Inf through), and bounds max_iter (<=100_000), n_restarts (<=1_000), and m_steps (<=1_000) so oversized loops/allocations and NaN-poisoned fits are rejected up front. - VULN-0005: plausible_values bounds n_draws (1..100_000) before forwarding to the core; serving_prior bounds n_dims (1..64) for direct callers. VULN-0001/0002 (non-finite params, malformed bundle) were already closed by the prior commit's strict JSON parse + _validate_bundle; VULN-0003 (missing-module import failure) is a false positive from the scanner's PR-diff-only checkout (backend.py/diagnostics.py/report.py all exist and the package imports). Regression tests extended to 32. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ning
The security-hardening bounds changed several FitConfig.validate messages,
breaking test_config.py's message-substring assertions (the values were
still correctly rejected). Reword the new messages as supersets of the old
text ("... >= 1 and <= N", "... > 0 and finite") so existing tests pass and
the stricter semantics remain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head77946ddc1af0a3b91dd4612980d485bf3ffe0454. -
Head SHA:
77946ddc1af0a3b91dd4612980d485bf3ffe0454 -
Workflow run: 29316547720
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (27 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (27 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (10 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (10 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (8 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (8 files)"]
R3 --> V3["targeted test run"]
Offload Bock-Mislevy (1982) EAP scoring to the wgpu path — the 31k-person serving hot path — per the all-math-in-Rust / maximize-GPU policy. - WGSL score_pass: one thread per person, race-free (each person owns its theta_eap/theta_sd/xi_eap/loglik slots; no atomics/slot-ownership unlike the E-step). Reuses the cell_l binary-sparsity decomposition and the same logp0/logp1/c0 tables the CPU scoring builds. f32, so ~1e-4 vs the f64 CPU reduction. Separate 19-binding bind-group layout + pipeline. - score_eap now delegates to a new score_eap_device(..., device): Cpu keeps the exact f64 reduction (the default -- all precision-sensitive callers and serving parity unchanged); Gpu/Auto try score_eap_gpu and fall back to CPU when no adapter or n_dims/latent_dim > 8. The CPU reduction was extracted to score_eap_cpu_reduce (the parity reference). - PyO3 score_bank_eap gains device="cpu"; serving.score_respondents threads device through so serving can request the GPU. On-device parity test gpu_eap_matches_cpu_reduction (<=2e-3) and a PyO3 device=gpu-vs-cpu smoke both pass on the local RTX; lib suite 67 green, serving/scoring pytest green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headc2048efe9d6bb600709aabfc8343927325405acf. -
Head SHA:
c2048efe9d6bb600709aabfc8343927325405acf -
Workflow run: 29318123391
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (27 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (27 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (10 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (10 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (8 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (8 files)"]
R3 --> V3["targeted test run"]
…re surface Extends the input-validation / DoS guards to the code added in this PR: - preprocessing.irtree_expand: bound dense expansion (persons*items*nodes <= 50M) before allocating; validate node_dims (finite, non-negative, integer) before int64 cast (VULN-0001/0002) - validation: reject labels above uint32 max before the narrowing cast; require human_human baseline to match the paired sample size (VULN-0003/0011) - inference.observed_information: cap finite-difference Hessian at 5000 params (O(n^2) memory and objective calls); oakes_standard_errors validates factor_id (1-D, per-item, finite, non-negative, integer) before deriving n_dims (VULN-0004/0005) - serving._validate_bundle + estimators.marginal._xi_grid: reject tensor grids with q_xi ** latent_dim > 1M points (VULN-0006) - linking.link_fixed_item_parameters: reject duplicate/fractional/ negative/non-finite anchors, non-2-D theta, non-finite item params, and non-finite computed linking coefficients (VULN-0007..0010) Regression tests in tests/test_security_hardening.py (53 pass); full suite 325 pass. All 11 Strix PoCs verified blocked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headb5d9d90c0a397c5bbeb4aa864f4fc6b291cd8936. -
Head SHA:
b5d9d90c0a397c5bbeb4aa864f4fc6b291cd8936 -
Workflow run: 29318985676
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (27 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (27 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Docs (10 files)"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs (10 files)"]
R2 --> V2["docs review"]
Evidence --> S3["Test (8 files)"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test (8 files)"]
R3 --> V3["targeted test run"]
…t (6 more) Strix re-scan of b5d9d90 reported 12 findings; #1 ("incomplete package release") is a scanner artifact of its PR-scope-only checkout (every named module exists; `import fast_mlsirm` succeeds). The 11 real findings: - serving.score_respondents/plausible_values: bound the dense respondent matrix (rows x n_items) before np.full (VULN-0002) - linking: range-check anchor indices on the float BEFORE the int64 cast (uint64 max wrapped to -1 -> last-item index) + same for factor_id (0003) - validation.validate_judge: bound category count k (dense k x k core matrix) (0004) - preprocessing.irtree_expand: 50M-element ceiling (400 MB, inclusive) -> 64 MiB byte budget (0005) - config.MLS2PLMConfig.validate: bound sim dims + n_persons*n_items cells (0006); FitConfig.validate: bound aggregate max_iter*n_restarts (0008) - estimators.marginal.fit_marginal_numpy: bound population counts (n_groups/n_clusters <= n_persons) (0007) + EM working set (0012) - inference.observed_information: reject non-finite step (0009); oakes_standard_errors: validate factor_id (0010, extended below) - fitstats: shared _validate_factor_id bounds n_dims for s_x2/person_fit/ infit_outfit and all public entries (0010) Proactive boundary-audit workflow found 6 more Strix had not surfaced: - serving._validate_bundle: bound scoring-table product (max(items,dims) x q_theta x q_xi**latent_dim; 55+ GB otherwise) and validate the population block (serving_prior read an unvalidated, attacker-controlled sigma_u -> TypeError/OverflowError crash or silent Inf/NaN score poisoning) - linking.irt_link: validate slope/intercept finiteness + slope positivity before the Nelder-Mead core (NaN would panic it); link_fixed_item_parameters requires a positive linking scale - validation.validate_judge: compact sparse subgroup labels (core loops 0..max(label)+1 -> O(4e9) CPU-DoS); oakes: reject n_dims > n_items Regression tests in tests/test_security_hardening.py (all 17+ new cases, suite 76 file-local); full suite 348 pass; every PoC verified blocked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Problem Untrusted factor_id arrays were converted to int64 before their domain was validated. A uint64 maximum label could wrap to -1 and select the last trait in predict_proba, while fit-statistics accepted it as d=[-1] with n_dims=0. Reproduction/Evidence On PR head 5b97972, the fixed-seed Python probe returned predict_uint64_max=[[0.9820137900379085]] and fitstats_d=[-1], n_dims=0. The new focused regression selection initially failed all 3 selected cases. Root cause Both diagnostic prediction and fit-statistics performed a lossy NumPy int64 cast before checking the original dtype and bounds. predict_proba also bypassed the objective module's factor-id validator. Change Validate the original array shape, integer dtype, sign, and upper bound before conversion. Reuse the hardened objective validator in predict_proba and add uint64-wraparound and fractional-label regressions. Validation uv run ruff check python/fast_mlsirm/objective.py python/fast_mlsirm/diagnostics.py python/fast_mlsirm/fitstats.py tests/test_security_hardening.py (passed) uv run pytest -ra tests/test_diagnostics.py tests/test_fitstats.py tests/test_security_hardening.py (201 passed) uv run pytest -ra tests/test_objective.py tests/test_irt_stability.py (22 passed) uv run pytest --collect-only -q (515 collected) uv run pytest -ra (515 passed in 89.07s; 0 skipped/xfail/xpass/deselected) Sources Strix current-head finding: https://github.com/ContextualWisdomLab/fast-mlsirm/actions/runs/29479224139/job/87558924195 This is an input-contract correction; no statistical-source citation is applicable.
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head4a80c136124843ad219556a9a1bd7c8a661f8bbf. -
Head SHA:
4a80c136124843ad219556a9a1bd7c8a661f8bbf -
Workflow run: 29481001652
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
Problem Public Rust and PyO3 scoring paths could construct Halton or Monte Carlo grids without the point and dimension limits enforced by Python FitConfig. Huge inputs could overflow n * latent_dim and panic, or attempt unbounded allocations. Reproduction/Evidence Before the change, cargo test -p mlsirm-core node_rules_reject -- --nocapture failed 3/3. usize::MAX panicked at nodes.rs:84 with 'attempt to multiply with overflow'; 1,000,001 points and unsafe latent dimensions were accepted. Root cause build_xi_nodes validated only n != 0 for stochastic rules and used unchecked multiplication. The Python-owned 1,000,000-point and 8-dimension resource limits were not mirrored at the native boundary. Change Mirror the repository limits in the Rust node builder, reject invalid dimensions and oversized stochastic point counts, use checked arithmetic for all grid lengths, and add panic-free boundary regressions. Validation - cargo test -p mlsirm-core node_rules_reject -- --nocapture: 3 passed - cargo test --workspace: 235 unit passed, 31 ignored; 16 integration passed; 1 property passed; 0 failed - cargo test --workspace -- --list: 266 unit, 16 integration, 1 property - cargo test --workspace -- --ignored --list: 31 explicitly ignored tests - cargo clippy -p mlsirm-core --lib --all-features: exit 0 (pre-existing warnings) - git diff --check: passed Sources This is a repository resource-safety and API-contract correction. The caps mirror existing Python configuration policy; no statistical claim or new academic citation is introduced.
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head3f809578e422b5a02f8c9e32d30b59866cfab2e4. -
Head SHA:
3f809578e422b5a02f8c9e32d30b59866cfab2e4 -
Workflow run: 29482946408
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
Problem: The public testlet wrapper coerced iteration and quadrature controls before validation, treated infinities as missing responses, and allowed unbounded response matrices and iteration counts to reach the native estimator. Reproduction/Evidence: .venv/bin/python -m pytest -q tests/test_security_hardening.py -k "fit_testlet_rejects_unsafe_controls or fit_testlet_rejects_unsafe_responses or fit_testlet_rejects_oversized_response_matrix" -ra failed 16/16 on 3f80957 because the recording core was invoked. Cases included bool/fractional/100001 max_iter, nonfinite tolerance and variance, unsupported/fractional q_gamma, finite response 2, infinity, zero persons, and a response-cell budget probe. Root cause: fit_testlet converted controls with int/float/bool and built its observed mask with isfinite before enforcing the documented binary/NaN response contract or repository resource policy. Change: Validate nonempty binary-or-NaN matrices before native dispatch, reject infinities, cap response matrices at 20000000 cells, require supported quadrature, require finite nonnegative controls, and reuse MAX_MAX_ITER=100000. Document that the caps are repository resource guards rather than testlet-model claims. Validation: - focused testlet security regressions: 22 passed, 170 deselected - existing public testlet feature: 1 passed, 61 deselected - full Python suite: 531 passed in 381.38s; no skips/xfails/xpasses/deselections - full collection: 531 tests - uv run --frozen ruff check: passed - git diff --check: passed - CodeGraph resynced and verified all public callers cross the new validation boundary Sources: No statistical formula changed. The iteration and cell limits are existing repository resource policies, so no new literature claim or citation is introduced.
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head8a20ac8903485bb6e31906827a75c9f66a8ece7f. -
Head SHA:
8a20ac8903485bb6e31906827a75c9f66a8ece7f -
Workflow run: 29484307786
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
Lift the D<=3 cap on the confirmatory compensatory MIRT (imposed by its q^D
Gauss-Hermite product grid) by swapping the E-step integration nodes for a
Halton quasi-Monte-Carlo (or seeded Monte-Carlo) rule, reaching D = 4, 5, 6.
fit_compensatory_mirt gains a node_rule ("gh" default, or "qmc"/"mc") plus
xi_points/xi_seed. This is Jank's (2005) QMC-EM: the E-step integral is evaluated
at xi_points prior draws (Halton radical inverse through the inverse-normal CDF,
equal weights 1/xi_points) instead of the product grid, and the node set is built
ONCE before the EM loop, so the per-item Newton M-step and the correlated-Sigma
ECM step are byte-for-byte the same code on the swapped nodes. The reused,
parity-tested node generator (nodes::build_xi_nodes, shared with the marginal
QMC-EM family) has a Gauss-Hermite arm bit-identical to the existing grid, so the
"gh" path is unchanged bit-for-bit and every prior MIRT test passes verbatim.
Both the orthogonal and the correlated-Sigma (Cholesky node-map theta_g = L z_g)
paths carry over. With Sigma = I the nodes never move, so the orthogonal fit is
monotone in the QMC-approximated marginal likelihood; the correlated Sigma M-step
reparametrizes the node cloud, so that fit is monotone only up to QMC quadrature
error (overall ascent with ~1e-5-relative per-step wobble that shrinks as
xi_points grows) -- documented, with the orthogonal path recommended when strict
monotonicity matters.
Validation is rule-dependent: "gh" keeps D<=3 + the q^D node cap; "qmc"/"mc" cap
D<=6 (the MonteCarlo builder has no internal cap, so this bound is its sole guard)
and bound xi_points (1..=200_000, with checked xi_points*n_items and
xi_points*n_dims). q applies only to "gh"; xi_points/xi_seed only to "qmc"/"mc".
Guards beyond the reused-grid regression: a deterministic layout pin recomputes
build_xi_nodes(Halton).grid[j*D+k] from the radical inverse and prime-per-axis (a
value-recovery/FD test is node-source-agnostic and cannot see a layout bug); the
QMC weights are pinned to -ln(n) (they cancel in the self-normalized posterior and
are otherwise invisible); an FD anchor pins the gradient and cross-Hessian on a
fixed Halton grid at D=4 with a non-identity dims map; the reduction anchor is
two-sided (QMC agrees with GH within error AND differs bit-wise, catching a silent
GH fallback); and the reflection anchor is driven by a reverse-keyed largest anchor.
Monte-Carlo (D in {4,5}, pure anchors + alternating-sign cross-loaders, Halton
xi_points 4000/6000, N 2000/1500): 100% convergence; normal trait near-unbiased
(loading RMSE ~0.13/0.17, bias ~0.01); per-dim-standardized right-skew trait shows
the expected mild attenuation (RMSE ~0.16/0.21, bias ~-0.07/-0.09); per-dimension
trait EAP correlation ~0.58-0.64 (pilot figures; the #[ignore] test runs 500 reps).
Exposed to Python as fit_compensatory_mirt(..., node_rule=, xi_points=, xi_seed=);
xi_seed is validated as an exact u64 (no float64 round-trip, which would corrupt
seeds > 2^53 and overflow near u64::MAX).
Jank, W. (2005). Quasi-Monte Carlo sampling to improve the efficiency of Monte
Carlo EM. Computational Statistics & Data Analysis, 48(4), 685-701.
https://doi.org/10.1016/j.csda.2004.03.019
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head6c19e350e516b18a25a2a5e542a6e88eaa5dfdb0. -
Head SHA:
6c19e350e516b18a25a2a5e542a6e88eaa5dfdb0 -
Workflow run: 29485339567
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
…Bock, 1972)
Add fit_nominal_mirt: a confirmatory MULTIDIMENSIONAL nominal response model
(Bock, 1972; Thissen, Cai & Bock, 2010) generalizing the unidimensional
poly::fit_nominal to D latent dimensions.
Each item's n_cat UNORDERED categories get a free multidimensional discrimination
a_ik (free on the confirmatory loading pattern, items x D) and intercept c_ik:
P(Y=k|theta) = softmax_k(sum_{d in S_i} a_ikd theta_d + c_ik), baseline category 0
pinned a_i0 = c_i0 = 0, theta ~ MVN(0, I_D). It reduces to fit_nominal EXACTLY at
D = 1 (the same general free-a_k parametrization).
Estimated by Bock-Aitkin marginal MLE over the D-dimensional latent grid, reusing
the compensatory-MIRT node machinery (nodes::build_xi_nodes): node_rule "gh" uses
the q^D Gauss-Hermite grid (D <= 3), "qmc"/"mc" use xi_points Halton / Monte-Carlo
draws (D <= 6, Jank 2005 QMC-EM). The per-item M-step is a Newton on the concave
multinomial-logit objective, byte-for-byte the FD-Hessian ascent of
poly::nominal_m_step (ridge is Hessian conditioning only, NOT a parameter prior,
so the fit is genuine MML and the D=1 reduction is bit-exact), with the softmax
residual resid_k = r_k - n P_k driving d/dc_ik = sum resid_k and
d/da_ikd = sum resid_k theta_d. EM uses fit_nominal's relative-tolerance stopping
with a SIGNED monotonic-decrease guard (a decrease errors, unlike the compensatory
MIRT's .abs() check).
Identification: baseline category + unit trait variances + a PURE single-dimension
anchor item per dimension pin the rotation to the coordinate axes (a pure anchor
forces every category slope onto its axis, so an orthogonal rotation must send that
axis to +-e_d; the confirmatory labels forbid axis permutation) -- leaving only a
per-dimension reflection, which (as in fit_nominal) is not canonicalized. validate
rejects a rotationally-degenerate pattern, an out-of-range category, and -- a guard
fit_nominal lacks -- ANY unobserved category for an item (its intercept would
diverge and its D slopes be unidentified), plus a nodes*items*n_cat count-table cap
and the rule-dependent D / q / xi_points bounds.
Guards: the D=1 anchor reproduces fit_nominal's parameters and whole loglik trace
bit-exactly (< 1e-9); a deterministic finite-difference anchor pins EVERY
per-(category, dimension) gradient component on a fixed node set at D=2 (GH) AND
D=4 (Halton) with a NON-IDENTITY dims map and distinct random per-category counts
(catching a category<->dimension transposition the D=1 reduction cannot see); a D=2
recovery carries a genuinely NEGATIVE cross-loader slope AND two OPPOSITE-sign
sibling categories on the same dimension (catching a collapse of the free
per-category slopes to a shared scalar discrimination); baseline / off-pattern
entries are asserted EXACTLY 0.0 with a free-parameter-count invariant.
Monte-Carlo (D in {2,3}, pure anchors + sign-varied cross-loaders, n_cat 3, GH
q 15/11, N 2500/2000, up to per-dim reflection): 100% convergence; normal trait
near-unbiased (slope RMSE ~0.12/0.13, bias ~0.00-0.01); per-dim-standardized
right-skew trait shows the expected mild attenuation (RMSE ~0.21/0.22, bias ~-0.09);
per-dim trait EAP correlation ~0.61-0.67 (40-rep pilot; the #[ignore] test runs 500).
Exposed to Python as fit_nominal_mirt / NominalMirtFit.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses
are scored in two or more nominal categories. Psychometrika, 37(1), 29-51.
https://doi.org/10.1007/BF02291411
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head3b1aa41ccfa79090f16ab059696e5209b26869a1. -
Head SHA:
3b1aa41ccfa79090f16ab059696e5209b26869a1 -
Workflow run: 29490621344
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
Problem: fit_nominal_mirt accepted finite non-negative fractional responses even though the public contract requires integer categories. Reproduction/Evidence: On PR head 3b1aa41, the same-head Strix mocked-core reproduction showed 0.9 and 1.9 reaching the native boundary as 0 and 1 after astype(np.int64). Root cause: The wrapper checked only the observed maximum before converting the full response array to int64, so NumPy silently truncated fractional values. Change: Require every observed response to equal its floor before conversion, and add a regression test proving fractional categories fail before the Rust core is called. Validation: - Regression target: tests/test_security_hardening.py:: test_nominal_mirt_rejects_fractional_categories_before_native - The pre-change current-head Python and Rust CI checks passed; the same-head Strix reproduction supplied the failing boundary evidence. - Local execution was unavailable because the mandatory checkout is read-only, 51 commits behind the PR head, and the system Python lacks NumPy. New-head GitHub checks are the authoritative validation after this fast-forward. Sources: Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29-51. https://doi.org/10.1007/BF02291411 The integer-category rejection is the repository's documented API contract; it is not attributed as a claim of the cited paper.
Problem: fit() silently truncated fractional factor_id labels before validating the factor mapping, which could change the fitted dimensional structure. Reproduction/Evidence: On PR head e239904, the same-head Strix mocked-boundary reproduction showed fractional factor_id values reaching validation only after np.asarray(..., dtype=np.int64) had truncated them. Root cause: fit() coerced factor_id to int64 before calling validate_factor_id(), so the validator could no longer distinguish caller-provided fractions from genuine integer labels. Change: Preserve the original factor_id dtype through shape and integer-kind checks, then compute n_dims and run the existing range validator. Add a regression test for fractional and NaN factor labels. Validation: - Both changed current-head source strings pass Python AST parsing. - An in-memory execution of the patched current-head fit.py rejected both factor_id=[0.5] and factor_id=[NaN] before fitting. - Regression target: tests/test_security_hardening.py:: test_fit_rejects_fractional_factor_id_before_integer_cast - New-head GitHub checks remain the authoritative full-suite validation. Sources: This is enforcement of the repository's existing public API contract in objective.validate_factor_id; no external statistical claim is introduced.
…ima, 1969)
Add fit_grm_mirt: a confirmatory MULTIDIMENSIONAL graded response model (Samejima,
1969; Muraki & Carlson, 1995) — the ORDERED-category counterpart of the
multidimensional nominal model and the polytomous generalization of the
compensatory MIRT.
Each item has a SINGLE multidimensional discrimination vector a_i (free on the
confirmatory loading pattern, items x D) and n_cat-1 ORDERED boundary intercepts
beta_i: P(Y>=k|theta) = sigmoid(sum_{d in S_i} a_id theta_d + beta_i,{k-1}),
theta ~ MVN(0, I). Reduces to poly::fit_poly_unidim(GRM) at D=1 within optimizer
tolerance and up to reflection (NOT bit-exact: fit_poly_unidim forces a>0 via a
log_a parametrization, while the confirmatory model uses an UNCONSTRAINED slope so
reverse-keyed / negative cross-loadings are representable — the MIRT choice).
Estimated by Bock-Aitkin marginal MLE over the D-dim latent grid, reusing the
compensatory-MIRT node machinery (nodes::build_xi_nodes): node_rule "gh" uses the
q^D Gauss-Hermite grid (D<=3), "qmc"/"mc" use xi_points Halton / Monte-Carlo draws
(D<=6, Jank 2005 QMC-EM), and the GRM cumulative-logit cell of poly::grm_logprobs /
grm_node_gradient. The per-item M-step is an FD-Hessian Newton over
[a_{d0}..a_{d,L-1}, beta_1..beta_{M-1}], byte-for-byte the ascent of
poly::m_step_item (ridge = Hessian conditioning only, not a prior), with the GRM
node gradient chained to the multidimensional slope (d/da_id = sum g_base theta_d,
d/dbeta_j = sum g_thr[j]).
The ORDERED-threshold constraint is maintained WITHOUT an explicit reparametrization:
every adjacent boundary pair is a middle category whose log-probability goes
non-finite the instant the pair inverts (0*NaN=NaN, so a zero expected count cannot
mask it), so the backtracking line search — which rejects any non-finite step —
keeps beta fully ordered by adjacency + transitivity. EM uses the SIGNED
monotonic-decrease stopping guard (a likelihood decrease errors, not the
compensatory MIRT's .abs() check).
Identification: unit trait variances + ordered thresholds + a PURE single-dimension
anchor item per dimension pin the rotation; the per-dimension reflection leaves base
(hence every threshold) invariant, so it is CANONICALIZED (flip the dimension so its
largest pure anchor loads positive, negating that dimension's slopes AND theta_d but
NOT the thresholds). validate rejects a rotationally-degenerate pattern, an
out-of-range category, and ANY unobserved category for an item (a GRM boundary would
diverge), with a nodes*items*n_cat count-table cap and the rule-dependent D/q/xi_points
bounds.
Guards: D=1 within-tol reduction to fit_poly_unidim(Grm) (all-positive DGP); a
deterministic FD gradient anchor pinning every per-(dimension, threshold) slot on a
fixed node set at D=2 (GH) AND D=4 (Halton) with a non-identity dims map, M>=4
categories, strictly-decreasing thresholds (gaps >> the FD step, since the GRM cell
NaNs on an inverted boundary) and distinct random counts; a SEPARATE deterministic
objective-value assertion at D=4 (dims [0,2,3]) that pins the node-column dims map by
computing base + GRM log-probs BY HAND and matching the estimator to < 1e-9 (the FD
anchor is map-invariant, and no D>=4 fit is exercised by the D<=3 recovery/MC); a
reflection-FIRES test (reverse-keyed largest anchor -> recovered positive, co-loader
negative, thresholds unchanged/ordered); and a D=2 recovery with a genuinely NEGATIVE
cross-loader on a positively-anchored dimension and strictly-ordered recovered
thresholds.
Monte-Carlo (D in {2,3}, pure anchors + sign-varied cross-loaders, n_cat 3, GH
q 15/11, N 2500/2000): 100% convergence; normal trait near-unbiased (loading RMSE
~0.10, bias ~0.00-0.01; threshold RMSE ~0.05-0.06); per-dim-standardized right-skew
trait shows the expected mild attenuation (loading RMSE ~0.17/0.18, bias ~-0.12/-0.13);
per-dim trait EAP correlation ~0.63-0.70; EM monotone and thresholds ordered every
replication (40-rep pilot; the #[ignore] test runs 500).
Exposed to Python as fit_grm_mirt / GrmMirtFit.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded
scores. Psychometrika Monograph Supplement, 34(4, Pt. 2).
https://doi.org/10.1007/BF03372160
Muraki, E., & Carlson, J. E. (1995). Full-information factor analysis for polytomous
item responses. Applied Psychological Measurement, 19(1), 73-90.
https://doi.org/10.1177/014662169501900109
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head7fefd6ab3c5bebf1f2668cfa1f2104337b7edb6e. -
Head SHA:
7fefd6ab3c5bebf1f2668cfa1f2104337b7edb6e -
Workflow run: 29495496646
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
Problem The new exploratory/confirmatory item-response APIs encoded dimensionality in names such as fit_grm_mirt and exposed a separate n_dims input. That split the model family contract and could mislabel a confirmatory pure-anchor estimator as unrestricted exploratory MIRT. Reproduction/Evidence R mirt uses a numeric model to request an exploratory factor count and a model specification for confirmatory structure. The current Rust 2PL, GRM, and nominal cores require a fixed binary loading pattern with pure anchors when D > 1; they do not estimate an unrestricted loading matrix or implement exploratory rotation/identification. Root cause The initial API mirrored implementation module names instead of separating the item response function from its latent model specification. Change - Rename the public families to fit_2pl, fit_grm, and fit_nominal, with matching dimension-agnostic result and Rust core names. - Add fast_mlsirm.models exploratory/confirmatory specifications and one model= argument. model=1 derives a one-column free-loading pattern; models.confirmatory(pattern) carries the loading structure and dimension count. - Reject numeric multidimensional exploratory requests until unrestricted loading estimation, rotation, and identification exist. - Keep n_dims only as a derived read-only result property. - Remove the brand-new *_mirt modules and aliases, update tests and changelog, and correct Samejima's publisher-verified volume/supplement citation. - Remove an unrelated CRM rustdoc block that had been attached to the 2PL binding. Validation - python -m ruff check python/fast_mlsirm/models.py python/fast_mlsirm/twopl.py python/fast_mlsirm/nominal.py python/fast_mlsirm/grm.py python/fast_mlsirm/__init__.py tests/test_security_hardening.py passed. - AST parsing passed for all changed Python modules and tests. - Model-spec runtime checks reported MODEL_CONTRACT_OK. - Mocked Rust-core wrapper checks reported WRAPPER_MODEL_CONTRACT_OK. - rustfmt --edition 2021 was applied to all changed Rust sources and a second formatting pass was idempotent. - Full native cargo/pytest execution is left to current-head CI because the available checkout is read-only, stale, and contains unrelated user changes. Sources - Chalmers, R. P. (2012). Journal of Statistical Software, 48(6), 1-29. https://doi.org/10.18637/jss.v048.i06 - Samejima, F. (1969). Psychometrika, 34(S1), 1-97. https://doi.org/10.1007/BF03372160 - Bock, R. D. (1972). Psychometrika, 37(1), 29-51. https://doi.org/10.1007/BF02291411 - Bock, R. D., Gibbons, R., & Muraki, E. (1988). Applied Psychological Measurement, 12(3), 261-280. https://doi.org/10.1177/014662168801200305 - Muraki, E., & Carlson, J. E. (1995). Applied Psychological Measurement, 19(1), 73-90. https://doi.org/10.1177/014662169501900109
Problem The Rust current-head CI job failed to compile the nominal module tests after the multidimensional estimator was renamed to fit_nominal. Reproduction/Evidence CI run 29499100684, job 87623208360, failed cargo test --workspace with 22 E0308/E0609 errors in crates/mlsirm-core/src/nominal.rs. Calls with the multidimensional signature resolved to crate::poly::fit_nominal and therefore returned NominalFit, which has no slope, theta, or n_parameters fields. Root cause The test module explicitly imported crate::poly::fit_nominal. That explicit import shadowed use super::* once the multidimensional function acquired the same item-family name. Change Alias the unidimensional parity helper as fit_nominal_unidim and use that alias only for the one parity call. All other test calls now resolve to the multidimensional function under test. Validation - The 22 compiler diagnostics share this single shadowed-import root cause. - The one unidimensional call is the only call using the poly signature. - Current-head CI will rerun cargo test --workspace on this commit. Sources - No external source is needed; this is a Rust name-resolution correction evidenced by the current-head compiler diagnostics.
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headc54c79b26d75261809b54d71d10aaf87f6fd5177. -
Head SHA:
c54c79b26d75261809b54d71d10aaf87f6fd5177 -
Workflow run: 29499407108
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
…model (Muraki, 1992)
Add fit_gpcm: a confirmatory MULTIDIMENSIONAL generalized partial credit model
(Muraki, 1992), completing the polytomous-MIRT trio (fit_nominal / fit_grm /
fit_gpcm) under the dimension-agnostic model= API. It is the ADJACENT-category-logit
counterpart of the cumulative fit_grm and the INTEGER-scoring restriction of the
free-slope nominal model.
Each item has a SINGLE multidimensional discrimination vector a_i (free on the
confirmatory loading pattern from model=models.confirmatory(...), items x D) and
n_cat-1 category step intercepts gamma_i, with INTEGER category scores k=0..n_cat-1:
psi_k = k*(sum_{d in S_i} a_id theta_d) + gamma_i,k, gamma_i,0 = 0 pinned, and
P(Y=k|theta) = softmax_k(psi_k), theta ~ MVN(0, I). Unlike the GRM's thresholds the
GPCM steps are UNORDERED (the softmax is finite for any real gamma, so no ordering
constraint exists or is imposed). This is the a_ikd = k a_id integer-scoring
restriction of the multidimensional nominal in a distinct single-slope
parametrization — NOT a mode of fit_nominal (which optimizes free per-category
slopes) — so it warrants its own estimator. Reduces to
poly::fit_poly_unidim(PolyModel::Gpcm) at D=1 within optimizer tolerance and up to
reflection (NOT bit-exact: fit_poly_unidim forces a>0 via a log_a parametrization,
while the confirmatory model uses an UNCONSTRAINED slope so reverse-keyed / negative
cross-loadings are representable).
Estimated by Bock-Aitkin marginal MLE over the D-dim latent grid, reusing the
compensatory-MIRT node machinery (nodes::build_xi_nodes): node_rule "gh" uses the
q^D Gauss-Hermite grid (D<=3), "qmc"/"mc" use xi_points Halton / Monte-Carlo draws
(D<=6, Jank 2005 QMC-EM), and the GPCM softmax cell of poly::gpcm_logprobs /
gpcm_node_gradient. The per-item M-step is an FD-Hessian Newton over
[a_{d0}..a_{d,L-1}, gamma_1..gamma_{M-1}], byte-for-byte the ascent of
poly::m_step_item (ridge = Hessian conditioning only, not a prior), with the GPCM
node gradient chained to the multidimensional slope (d/da_id = sum g_base theta_d,
d/dgamma_j = sum g_intercepts[j]). Category scores are FIXED integers 0..n_cat-1
(that fixity is what makes the model GPCM rather than nominal), so the free
per-category slope gradient returned by the shared cell (g_scores) is DROPPED. Init
is gamma_k = ln(freq_k/freq_0) (a plain marginal log-odds, NOT a cumulative
GRM-style boundary). EM uses the SIGNED monotonic-decrease stopping guard.
Identification: unit trait variances + a PURE single-dimension anchor item per
dimension pin the rotation; the per-dimension reflection leaves base (hence every
step and category probability) invariant, so it is CANONICALIZED (flip the dimension
so its largest pure anchor loads positive, negating that dimension's slopes AND
theta_d but NOT the steps). validate rejects a rotationally-degenerate pattern, an
out-of-range category, and ANY unobserved category for an item, with a
nodes*items*n_cat count-table cap and the rule-dependent D/q/xi_points bounds.
Guards (adversarial-review-hardened): D=1 reduction to fit_poly_unidim(Gpcm); a
deterministic FD-gradient anchor at D=2 (GH) and D=4 (Halton) with a NON-IDENTITY
dims map, M>=4 categories, deliberately NON-MONOTONE step values, and distinct
random per-category counts; a separate deterministic objective-value pin at D=4
(dims [0,2,3]) computing base and the GPCM log-probabilities BY HAND with LITERAL
integer scores; a reflection-FIRES test constructed so the RAW EM mode lands the
pure anchor NEGATIVE (a WEAK reverse-keyed pure anchor plus a STRONG positively-keyed
cross-loader dominating the axis) so canonicalization MUST fire — asserting the
anchor ends positive, the co-loader negative, theta_0 sign-flipped, and the steps
unchanged (mutation-verified: disabling the flip fails all three sign checks); and a
D=2 recovery with a genuinely NEGATIVE cross-loader recovering the unordered steps by
RMSE. A #[ignore] Monte-Carlo (D in {2,3}, n_cat=4, GH q=15/11, N=2500/2000, 500
replications) recovers the loadings near-unbiased under a normal trait (loading RMSE
~0.08-0.09, step RMSE ~0.06-0.07) with the expected mild attenuation under a
right-skew trait, per-dimension trait EAP correlation ~0.74-0.77 and 100% convergence.
Compute lives in mlsirm_core::gpcm::fit_gpcm; exposed to Python as fit_gpcm / GpcmFit
via the models= specification API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headcc821ec5f98902716ec3d8ab4f35afced1d3584f. -
Head SHA:
cc821ec5f98902716ec3d8ab4f35afced1d3584f -
Workflow run: 29510810080
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 2
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
…Robbins-Monro (Cai, 2010)
Add fit_mhrm: the general compensatory multidimensional 2PL
(P(X_ij=1|theta_j) = sigmoid(sum_{d in S_i} a_id theta_jd + b_i), theta ~ MVN(0, I_D))
— the same model as fit_2pl — estimated by a STOCHASTIC-approximation EM that scales to
a latent dimensionality (n_dims up to 64) where the deterministic q^D Gauss-Hermite grid
and the QMC E-step of fit_2pl are infeasible.
Each cycle:
1. I-STEP (imputation): a short PERSISTENT (warm-started) symmetric random-walk Metropolis
chain draws each person's theta from its posterior pi(theta) prop phi(theta;0,I) prod_i
P_i(y|theta). The symmetric proposal cancels the Hastings ratio (accept = min(1, exp(sum_i
loglik-diff - 0.5(||theta*||^2 - ||theta||^2)))); the proposal SD is auto-tuned toward a
target acceptance during burn-in; the chain carries across cycles so no per-cycle burn-in
is needed.
2. RM STEP (per item, BLOCK-DIAGONAL): items are conditionally independent given theta, so the
complete-data score s = sum_p X_p (y_p - P_p) and information H = sum_p w_p X_p X_p' are the
CLOSED-FORM logistic gradient/information (no quadrature, D-independent per-node cost; X =
[theta_d for d in S_i, 1]). RM info recursion Gamma += gain (H - Gamma); Newton ascent
xi += gain * (Gamma+ridge)^{-1} s. Gain is constant during burn-in then 1/(k-k0)^alpha
(sum gain=inf, sum gain^2<inf; Robbins & Monro, 1951), converging a.s. to a marginal-score
root. Convergence is WINDOWED (running mean of ||xi delta|| over the last `window` cycles <
tol) — MH-RM iterates are non-monotone, so no likelihood-decrease guard.
Reuses mmle::{log_sigmoid, sigmoid_stable} and poly::solve_small; a deterministic inline
LCG + Box-Muller for the proposals.
Identification: unit trait variances fix the loading scale, a PURE single-dimension anchor
item per dimension pins the rotation, and the per-dimension reflection is CANONICALIZED
(largest pure anchor positive), enforced IN-LOOP every cycle — flipping the loading column,
the persistent theta chain, the averaged trait, AND the RM information/SE accumulators
together — so the stochastic running average stays in one mirror mode. Loadings are
UNCONSTRAINED (reverse-keyed / negative cross-loadings representable). This first release
fits the ORTHOGONAL 2PL (Sigma = I); a free latent correlation matrix and the polytomous
families are natural extensions of the same loop.
Standard errors: the Louis (1982) observed information, approximated by a parallel RM filter
that subtracts the UNCENTERED per-person score cross-product sum_p (y-P)^2 X X' from the
complete-data information (the standard single-imputation m=1 form) — conservative where the
block is PD, falling back to the complete-data Fisher information where the subtraction would
leave it non-PD. Documented as such (exact per-person centering needs m>1).
Guards (adversarial-review-hardened): a deterministic finite-difference anchor pins the
per-item score and information against numerical derivatives of the complete-data logistic
log-likelihood on an ASYMMETRIC D=2 cross-loader with a negative loading, and additionally
pins the Louis missing-information SIGN (hobs = H - sum_p r^2 X X'); the D=1 fit agrees with
the deterministic unidimensional MMLE (mmle::fit_mmle_2pl) within Monte-Carlo tolerance; a
D=6 recovery (3 pure anchors per dimension + a negative cross-loader, GH/QMC infeasible)
recovers the loadings and per-dimension traits; a mutation-verified reflection-fires test
drives a weak reverse-keyed pure anchor against a strong positive cross-loader so raw MH-RM
lands the anchor negative and canonicalization must fire; a white-box gain-schedule anchor
pins the Robbins-Monro gain at the burn-in boundary; and validate rejects rotationally-
degenerate patterns, non-binary responses, n_dims > 64, length mismatches, and
burn_in >= max_cycles. A #[ignore] Monte-Carlo (D in {2, 6}, normal + skew, 500 reps)
exercises the high-dimensional regime.
Compute lives in mlsirm_core::mhrm::fit_mhrm; exposed to Python as fit_mhrm / MhrmFit via
the models= specification API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete the Metropolis-Hastings Robbins-Monro fit to correlated latent factors via a new
estimate_corr option (Cai, 2010b confirmatory item factor analysis): theta ~ MVN(0, Phi) with
Phi a free CORRELATION matrix (unit diagonal) instead of orthogonal factors.
The MH acceptance prior becomes -0.5(theta*' Phi^{-1} theta* - theta' Phi^{-1} theta) (the
symmetric proposal cancels; Phi^{-1} is recomputed by Cholesky each cycle), and the D(D-1)/2 free
off-diagonal correlations ascend the Gaussian-prior objective
Q(Phi) = -0.5[log|Phi| + tr(Phi^{-1} C)] (C = the imputed second moment, RAW/uncentered because
E[theta]=0 is fixed by identification) by a per-cycle Robbins-Monro GRADIENT step, kept
positive-definite by BACKTRACKING (halve the step until the rebuilt Phi is PD, preferred over a
full reject to avoid frozen cycles near the PD boundary). This reuses the twopl.rs correlation
machinery VERBATIM (build_corr, sigma_grad, chol_lower, sym_inv_logdet, flip_corr_dim, now
pub(crate)) -- the same helpers fit_2pl's deterministic ECM correlation step uses, so Phi
estimation is shared, not duplicated.
The per-dimension reflection flips the correlation off-diagonals for the flipped dimension
(corr(theta_d, theta_k) -> -corr) together with the loading column and trait chain, keeping the
reported Phi consistent with the canonicalized signs. estimate_corr=false (default) keeps Phi = I
and is BIT-IDENTICAL to the previous orthogonal fit -- the acceptance prior branches to the
original per-dimension ||theta*||^2 - ||theta||^2 on the same RNG stream. It is a gradient-RM (not
Cai's Newton-preconditioned) covariance update, documented as such: it still converges almost
surely to the same Phi root, only the un-curvature-adapted rate differs.
Guards (spec-verified GO-WITH-MUST-FIXES applied; adversarial impl-review 0 confirmed defects): a
recovery test recovers an exchangeable Phi off-diagonal at a POSITIVE (rho=0.4), a
near-PD-boundary (D=3, rho=0.5), and a NEGATIVE (rho=-0.5) correlation within Monte-Carlo tolerance
and confirms the recovered matrix stays a valid PD correlation matrix; estimate_corr=false yields
exactly the identity; and a #[ignore] 500-rep Monte-Carlo at the near-boundary D=3, rho=0.5 reports
the correlation RMSE/bias (a persistent PD-backtracking stall would surface there). The score/info
FD anchor, D=1 MMLE reduction, D=6 recovery, mutation-verified reflection, and gain-schedule anchor
are unchanged. Exposed to Python as the estimate_corr argument and the corr field of MhrmFit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the Metropolis-Hastings Robbins-Monro estimator (Cai, 2010) from binary 2PL items to the
ordered polytomous generalized partial credit model (Muraki, 1992), selected by a new
MhrmModel::Gpcm { n_cat } on MhrmConfig (Python: fit_mhrm(..., family="gpcm", n_cat=K)). This
scales high-dimensional POLYTOMOUS confirmatory item factor analysis to a latent dimensionality where
the deterministic gpcm.rs (Gauss-Hermite / QMC EM) is infeasible, reusing the same MH imputation +
block-diagonal Robbins-Monro stochastic-Newton machinery as the binary fit.
Each item keeps a SINGLE multidimensional discrimination a_i (free on the confirmatory loading
pattern) and gains K-1 free UNORDERED step intercepts: base_i = sum_{d in S_i} a_id theta_d (NO
intercept), P(Y=k) = softmax_k(k*base_i + step_ik) with step_i0 = 0 pinned. The MH I-step likelihood
is the inline log-softmax of the observed category (no per-node Vec allocation). The per-item RM step
uses the CLOSED-FORM multinomial complete-data Hessian H = sum_p J_p'(diag(P) - P P')J_p as BOTH the
Robbins-Monro preconditioner AND the Louis positive term, where the design row J_p[k] = d psi_k/d param
is k*theta_pd for slope a_id and [k==j] for step_j. This is deliberately NOT the BHHH score
cross-product: BHHH is exactly the term the Louis identity subtracts, so H_BHHH - sum_p s_p s_p' = 0
would give a degenerate SE. The complete-data score sum_p J_p'([k==y_p] - P) equals gpcm.rs's
[g_base*theta_d; g_intercepts] with the integer scores fixed (g_scores dropped -- what makes it GPCM,
not nominal). The per-dimension reflection flips ONLY the slope column and the trait chain; the
UNORDERED steps are INVARIANT (base = k*sum a_d theta_d is invariant under the joint (a, theta) sign
flip), exactly as gpcm.rs.
family="2pl" (default) keeps the binary path BIT-IDENTICAL -- the closed-form log_sigmoid score and
sum w X X' information are byte-for-byte unchanged on the same RNG stream (confirmed: the 8 existing
2PL mhrm tests pass with zero regression). MhrmResult gains step (J*(n_cat-1)), n_cat, and se_step
(2PL keeps intercept/se_intercept, empty step/se_step, n_cat=2); n_parameters = n_free_loadings +
n_items*(n_cat-1) (+ D(D-1)/2 correlations when estimate_corr). Validation adds the GPCM guards:
n_cat >= 2, responses in 0..n_cat, and every declared category observed per item (else the step is
unidentified, mirroring gpcm.rs). GRM (Samejima cumulative-logit, ordered thresholds) is DEFERRED: a
single stochastic RM Newton step cannot run the backtracking line search grm.rs uses to keep the
thresholds strictly decreasing; the standard route is a softplus threshold-gap reparametrization
(future work).
An adversarial impl-review found and fixed two defects the initial tests missed: the output/SE routing
keyed on n_free_cat==1 as an "is 2PL" proxy, mis-collapsing a legal Gpcm{n_cat=2} fit's single step
into the 2PL intercept/se_intercept fields (now keyed on the model family via matches!(cfg.model,
TwoPl)); and the declared MHRM_MAX_CAT cap was never enforced (unbounded n_cat allocation -- now
validated). Both are covered by new regressions.
Guards (spec-verified GO-WITH-MUST-FIXES applied -- the exact-Hessian information replaced the
originally-proposed BHHH): a deterministic finite-difference anchor pins
the GPCM score AND the exact-multinomial information against the complete-data GPCM log-likelihood on an
asymmetric cross-loader with a NEGATIVE loading and NON-MONOTONE steps (a sign flip, a
transposed/dropped design slot, an over-collapsed step block, or BHHH-as-information all fail it), with
an independent per-person score outer-product re-sum pinning the sign of the Louis missing-information
subtraction; a D=1 reduction agrees with poly::fit_poly_unidim(Gpcm) within Monte-Carlo tolerance; a
D=5 recovery (GH/QMC infeasible) recovers loadings, steps, and the negative cross-loader with correct
sign; a reflection-FIRES test witnesses the canonicalization flipping a negative anchor while leaving
the steps un-swept; validation rejects out-of-range responses and a never-observed category; and a
#[ignore] 500-rep Monte-Carlo (normal + right-skew, D=2 and D=5, K=3) reports loading/step RMSE and
bias. Exposed to Python as the family/n_cat arguments and the step/se_step/n_cat fields of MhrmFit,
with a D=3 GPCM recovery + validation pytest. Core 272 tests and 542 pytest green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Problem Untrusted factor CSV and JSON inputs were fully materialized before validation. The NumPy GPCM reference accepted an unbounded category count and allocated a dense K-by-K Hessian. Direct EAP scoring cast arbitrary factor identifiers to int64 and derived an unbounded latent dimension before allocating its context. Reproduction/Evidence Strix run 29499407084 job 87624290522 reproduced VULN-0001..0004 on c54c79b: a 60 MB factor CSV reached about 206 MB RSS; n_cat=100000 implied a roughly 74.5 GiB Hessian; unbounded CLI JSON was deserialized before cell limits; factor_id=[100000000] attempted a roughly 763 MiB context allocation and uint64 2**63 wrapped during int64 coercion. The affected files were unchanged through parent 13873c9. Root cause Byte and dimension limits existed for NumPy and serving matrix inputs but were not applied at the text-deserialization and direct estimator API boundaries. Change Add bounded UTF-8/JSON loaders, route factor CSV, bundle, fit-summary, and CLI response JSON through them, cap the NumPy GPCM category block at 256, and validate factor identifiers and explicit dimensions before quadrature or allocation. Add regressions that assert rejection occurs before NumPy parsing or quadrature. Validation - ast.parse passed for all five changed Python files. - ruff check --stdin-filename FILE - passed for io.py, serving.py, cli.py, and test_security_hardening.py. - marginal.py passed ruff with --ignore E741,F841; the unignored run reported only seven pre-existing E741/F841 findings outside the changed lines. - An in-memory fixed-input harness rejected both malicious factor identifiers, the unbounded explicit dimension, and n_cat=100000 before quadrature; normal MIRT EAP remained finite with shape (2, 1). - An in-memory text-loader harness rejected oversized CSV/JSON before parsing and accepted normal CSV/JSON inputs. - Full Python collection and tests are delegated to current-head CI because the retained local checkout contains unrelated user changes. Sources - https://github.com/ContextualWisdomLab/fast-mlsirm/actions/runs/29499407084/job/87624290522
Problem: The Python MH-RM references listed two distinct Cai 2010 articles without APA year suffixes while the implementation prose cited 2010b. Page ranges also used hyphens and the Louis journal title omitted its methodological series qualifier. Reproduction/Evidence: Inspect the References sections in python/fast_mlsirm/mhrm.py and the corresponding Rust rustdoc. The two same-author same-year works were both rendered as Cai 2010. Root cause: The new confirmatory MH-RM documentation copied verified metadata without applying APA 7 same-author same-year disambiguation consistently across Python and Rust. Change: Label the exploratory paper 2010a and the confirmatory paper 2010b, bind confirmatory claims to 2010b, restore the full Louis journal title, and normalize verified page ranges to en dashes in MH-RM and GPCM references. Validation: Python AST parsing passed for both edited modules. ruff check passed for both edited modules. ruff format --check passed for gpcm.py; mhrm.py has the same pre-existing formatter result before and after this comments-only change. Rust changes are rustdoc-only; a structural diff confirmed no non-doc Rust lines changed. Sources: https://doi.org/10.1007/s11336-009-9136-x https://doi.org/10.3102/1076998609353115 https://doi.org/10.1111/j.2517-6161.1982.tb01203.x https://doi.org/10.1177/014662169201600206
Problem: The compensatory 2PL QMC validation bounded xi_points and checked multiplication overflow, but did not bound the resulting node-by-item table size. Small one-person inputs could therefore request hundreds of gigabytes before statistical work began. Reproduction/Evidence: With n_persons=1, n_items=301, and xi_points=200000, all individual limits pass and the inputs contain only 301 responses. The E-step then allocates four dense 200000 by 301 f64 tables, about 1.9 GB. At n_items=20000 the same path requests at least 128 GB. The existing checked_mul only detects usize overflow, not resource exhaustion. Root cause: MIRT_MAX_NODES constrained one axis but no aggregate node times item budget was applied before log-probability and expected-count allocations. Change: Derive the active node count for GH, Halton, and Monte Carlo rules, cap node times item cells at 60000000 consistently with the polytomous QMC estimators, and validate the node times dimension buffer. Add a regression case that remains tiny but crosses the aggregate table budget. Validation: rustfmt --edition 2021 --check passed on the edited current-head source. The regression constructs 301 response cells and asserts rejection before any node table is allocated. Full current-head Rust CI is required because the provided checkout is stale and contains unrelated user modifications that cannot be reset safely. Sources: Jank 2005 motivates the QMC E-step; the aggregate memory cap is a repository-specific safety contract. https://doi.org/10.1016/j.csda.2004.03.019
…1988)
Add a new mlsirm_core::dif module implementing the Mantel-Haenszel differential item functioning
procedure -- the observed-score, calibration-free complement to the parametric IRT-LR DIF
(poly::poly_dif_sweep / dif_polytomous). No item response model is fitted: examinees are matched on the
number-correct total (thin matching, studied item INCLUDED by default per Donoghue, Holland & Thayer,
1993; exclude_studied_item uses the rest score, recomputing the strata per item), and for each item a
2x2 group-by-response table is formed at every matching level.
Over the DIF-informative strata (all four marginal totals positive, so the hypergeometric variance is
positive):
- Common odds ratio alpha_MH = (sum_m A_m D_m/T_m)/(sum_m B_m C_m/T_m).
- Continuity-corrected MH chi-square max(0, |sum_m A_m - sum_m E(A_m)| - 0.5)^2 / sum_m Var(A_m), with
E(A_m)=n_Rm m1_m/T_m and Var(A_m)=n_Rm n_Fm m1_m m0_m/(T_m^2 (T_m-1)), referred to chi^2(1) via the
existing fitstats::chi2_sf (no new tail function).
- ETS delta metric MH_D-DIF = -2.35 ln(alpha_MH) (negative = harder for the focal group) with the
Robins-Breslow-Greenland (1986) standard error.
- ETS A/B/C classification (Zieky, 1993): A if not significant at .05 or |D-DIF|<1.0; C if |D-DIF|>=1.5
and |D-DIF|-1.645*SE>1.0; B otherwise. A degenerate common odds ratio (0/inf) or the absence of any
informative stratum yields NaN statistics and an Undefined ("U") class -- deliberately NOT the
affirmative "A" (negligible), which would claim a result the data cannot support.
- Standardized P-DIF companion (Dorans & Kulick, 1986) sum_m n_Fm (P_Fm - P_Rm)/sum_m n_Fm, focal minus
reference so its sign agrees with MH_D-DIF, gated on both groups present.
Benjamini-Hochberg controls the across-item FDR (fitstats::benjamini_hochberg, NaN p-values skipped).
Distinct from the parametric IRT-LR DIF (which fits a two-group marginal EM twice per item); this shares
no estimation machinery and lives in its own module. SIBTEST (Shealy & Stout, 1993) and item
purification are noted as future work.
Spec-verified (GO-WITH-MUST-FIXES, applied): STD-P-DIF focal-minus-reference sign; the Var_m>0 stratum
gate (all four marginals positive, not merely "a correct and an incorrect present", which admits
single-group strata); degenerate-odds guards returning NaN + Undefined rather than dividing by zero or
taking ln(0)/ln(inf); and the continuity numerator clamped at zero so |D|<0.5 contributes no spurious
statistic.
Guards. A two-stratum hand-computed anchor pins alpha_MH=4.5, the continuity-corrected chi2=48.586,
MH_D-DIF=-2.35 ln(4.5), SE=0.5129, STD-P-DIF=-0.35, and the class-C label (a dropped -0.5, a wrong
variance denominator, a sign flip, or a reference-minus-focal STD-P-DIF all fail it). A no-DIF symmetry
anchor gives alpha=1, zero delta, class A; a single-group and a perfectly-separated table give NaN and
Undefined (never A); and a planted uniform (b-shift) DIF simulation with no impact flags the DIF item
(class B/C, correct delta sign) while classifying the clean items A and agreeing with the parametric
IRT-LR DIF on the flagged item. Because the MH chi-square is over-powered at large N and the studied
item mildly contaminates the matching total, the ETS A/B/C classification (not the raw significance) is
the practical-significance guard against spuriously flagging clean items -- documented as such.
Exposed to Python as fast_mlsirm.mantel_haenszel_dif (per-item dict of alpha_mh/chi2_mh/p_value/
mh_d_dif/se_d_dif/std_p_dif/ets_class/flagged_bh) with a planted-DIF integration test. Core tests and
pytest green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headd2a2d2950c91a7a86de5f443ba4764020a7375a0. -
Head SHA:
d2a2d2950c91a7a86de5f443ba4764020a7375a0 -
Workflow run: 29539250771
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
…m, 1989)
Add scoring::score_wle, the bias-reduced maximum-likelihood ability estimator for a unidimensional
dichotomous test (2PL/3PL/4PL). The MLE of theta carries an O(1/n) bias; Warm removes its leading term
by weighting the likelihood by a function w with w'/w = J(theta)/(2 I(theta)), giving the estimating
equation
dlnL/dtheta + J(theta)/(2 I(theta)) = 0,
with, for P_i = c_i + (d_i - c_i) sigmoid(a_i(theta - b_i)):
- score dlnL/dtheta = sum_i (y_i - P_i) P_i'/(P_i Q_i),
- information I(theta) = sum_i P_i'^2/(P_i Q_i) [item_information_4pl per item, reused],
- Warm correction J(theta) = sum_i P_i' P_i''/(P_i Q_i), computed DIRECTLY from P' P''.
J is deliberately NOT I'(theta)/2: they coincide only for the 2PL/Rasch (c=0,d=1), where the weight is
sqrt(I) (the Jeffreys prior). For the 3PL/4PL the second derivative carries (1-2s) while the information
derivative carries (1-2P), so a sqrt(I)-weighted estimator would apply the wrong correction. Two
properties Warm establishes and the tests verify: the leading MLE bias is removed, and -- unlike the
MLE, which is +/-infinity for the all-correct / all-incorrect pattern -- the estimate is FINITE for
every response pattern. The estimate is the GLOBAL maximizer of the weighted log-likelihood Phi
(Phi' = g = score + J/(2I)), located by a grid scan of g whose trapezoidal cumulative integral recovers
Phi, plus a local root refinement around the global-max node. This is robust to the 3PL/4PL case where
the weighted likelihood can be multimodal (Samejima, 1973; Yen, Burket & Sykes, 1991) and a single
bracketed root can select a non-dominant mode; when the finite Warm root falls beyond theta_bound (very
easy/hard items for the pattern) theta is clamped to the boundary and flagged, and a person with no
observed items returns NaN. The reported SE is 1/sqrt(I(theta_wle)).
Spec-verified (GO-WITH-MUST-FIXES, applied): compute J directly (not I'/2); guard the J/(2I) division
against I ~ 0 (item_information_4pl saturates to 0 as P rounds to 1 for large a) via an information
floor and a clamped sigmoid; take plain natural-scale a/b/c/d arrays (the LSIRM ItemBank has no c/d and
stores alpha on the log scale). An adversarial implementation review then caught and fixed two defects
the initial tests missed: (1) the original single expanding-bracket bisection assumed g is globally
monotone, which holds only for the 2PL/Rasch -- for the multimodal 3PL/4PL it could return a
non-dominant root (a decisive case returned theta ~ +1.70 when the dominant weighted-likelihood mode was
theta ~ -4.13); replaced by the global-mode grid search above (the 2PL results are unchanged -- Phi is
unimodal there). (2) A person with all items missing silently returned theta = 0 (indistinguishable
from a genuine estimate); now returns NaN with the boundary flag set. Reuses the private sigmoid,
validate_dichotomous_responses, and pub item_information_4pl; skips unobserved items in all three sums.
Guards. An estimating-equation ROOT anchor verifies the returned theta_hat against g recomputed from
INDEPENDENT finite-difference derivatives of P (so a sign error in the analytic P' P'' J term is not
shared), across the 2PL, 3PL, and Rasch; a 2PL finiteness anchor confirms the perfect/zero patterns give
finite, interior estimates with correct > incorrect; a monotonicity anchor confirms the estimate is
nondecreasing in the number-correct score; a global-mode anchor pins the multimodal-3PL worst case to
the dominant mode (theta < -3, not the +1.70 minor root); an all-missing person returns NaN; validation
guards trip non-vacuously; and a #[ignore] 500-rep Monte-Carlo confirms Warm's headline result --
aggregate |bias| WLE 0.0375 vs the boundary-clamped MLE 0.5006 (a ~13x reduction), the gap concentrated
at extreme abilities (theta=+/-2: WLE ~0.01 vs MLE ~0.2).
Exposed to Python as fast_mlsirm.score_wle (theta/se/boundary; c=0,d=1 default 2PL, NaN=missing). Core
tests and pytest green. Polytomous WLE (Penfield & Bergeron, 2005) deferred.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…970/1972/1973)
Add a new mlsirm_core::rasch_cml module: conditional maximum likelihood (CML) estimation of the
dichotomous Rasch model, plus Andersen's (1973) conditional likelihood-ratio test of fit. Conditioning
each response pattern on its raw score -- the sufficient statistic for the person parameter -- ELIMINATES
the person parameters, so the item difficulties are estimated without any assumption on the ability
distribution (Rasch's specific objectivity) and consistently at fixed test length, unlike the marginal-
ML path (which must posit a theta distribution) or joint ML (inconsistent as N grows).
With eps_i = exp(-beta_i), s_i = sum_v x_vi, n_r = persons with raw score r, and gamma_r the elementary
symmetric function of order r of {eps}:
ln L_c(beta) = -sum_i s_i beta_i - sum_{r=1}^{k-1} n_r ln gamma_r(eps).
Persons scoring 0 or k carry no conditional information and are dropped. The score equation is observed
s_i = expected, with E[s_i|r] = eps_i gamma_{r-1}^{(i)}/gamma_r.
The ESF and its per-item / per-pair derivatives use the SUMMATION algorithm (a fresh forward recursion
gamma_r += eps_j gamma_{r-1} over the relevant item subset), which is numerically stable; the
subtractive difference recursion gamma_r^{(i)} = gamma_r - eps_i gamma_{r-1}^{(i)} is deliberately
avoided because it cancels catastrophically for very easy items (large eps_i) (Verhelst, Glas & van der
Sluis, 1984). Newton on beta with a backtracking line search, sum-zero identification via a
reduced-coordinate solve (reuse poly::solve_small) and per-iteration re-centering; the standard errors
come from the pseudoinverse of the conditional information I_c = -H, I_c^+ = (I_c + (1/k)J)^{-1} -
(1/k)J (reuse twopl::sym_inv_logdet), since I_c is rank k-1 with null space span(ones).
Andersen's (1973) conditional LR test partitions the persons into G subgroups, fits CML within each and
over the pooled sample, and refers LR = 2[sum_g llc_g(beta_g) - llc(beta_pooled)] to chi2((G-1)(k-1))
(reuse fitstats::chi2_sf); a significant LR rejects the invariance of the item difficulties across the
split (Rasch misfit / DIF).
Spec-verified (GO-WITH-MUST-FIXES, applied): the summation ESF over the cancellation-prone difference
recursion for gamma^{(i)}/gamma^{(ij)}; drop r=0/k persons; sum-zero centering each Newton iteration; the
reduced (k-1) system for the SE (do not invert the singular full information); reuse solve_small,
sym_inv_logdet, chi2_sf; a documented item-count cap (CML_MAX_ITEMS=100). Complete data only; the
missing-data and polytomous CML extensions are deferred.
Guards. The summation-algorithm ESF (and the leave-one-out / leave-two-out passes) match brute-force
subset sums exactly; a deterministic finite-difference anchor pins the CML gradient AND the full Hessian
against ln L_c (catching the d eps/d beta = -eps sign and the ESF-derivative recursions -- a plain
value-recovery test cannot); the DEFINING person-distribution-free property is the primary anchor -- the
same beta_hat (up to the sum-zero constant) is recovered whether the simulating theta is N(0,1) or
strongly right-skewed, which a distribution-DEPENDENT estimator (JML/marginal-ML) would fail; and the
Andersen LR does not over-reject an arbitrary split of true Rasch data (statistic near its df) but
rejects a planted group-specific difficulty shift, with the df=(G-1)(k-1) and the upper tail pinned.
An adversarial implementation review (the faithfulness lens was clean -- the CML math is correct) found
and fixed two defects the initial tests missed: andersen_lr_test now surfaces a `converged` flag, since
a stalled per-group fit can drive the pre-clamp statistic negative and the .max(0.0) clamp would
otherwise report lr=0 / p=1 as a clean non-rejection (a starved-max_iter regression pins this); and the
PyO3 binding caps n_groups at 256 to prevent a lossy u8 group-label truncation.
Exposed to Python as fast_mlsirm.fit_rasch_cml (beta/se/loglik/n_iter/converged/n_used) and
andersen_lr_test (lr/df/p_value/n_used/converged). Core tests and pytest green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head999a33df19e39dd7cc88fd0dcdb86f4465fd7956. -
Head SHA:
999a33df19e39dd7cc88fd0dcdb86f4465fd7956 -
Workflow run: 29546763168
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
Evidence["OpenCode evidence"] --> Review["Current PR review path"]
Review --> Verify["Required checks"]
1 participant
Summary
Model-design PR implementing marginal maximum likelihood (Bock-Aitkin EM) for the full latent-space model family, estimation-level population structures, a wgpu GPU E-step, all-Rust scoring/fit-statistic compute, and the implementable core of four supplied literature batches. Design doc:
docs/mmle_marginal_lsirm_design.md; per-equation paper basis:docs/papers/mmle-lsirm-formula-compilation.md(+group_{a,b,c}_specs.md,implemented-literature-map.md,corpus-triage-batch{3,4}.md).Estimator (
mlsirm_core::marginal)MIRT/MLS2PLM/MLSRM/ULS2PLM/ULSRMunder the simple-structure contract; deterministic Gauss-Hermite E-step made tractable by the conditional factorization (Q_xi^K * sum_d Q_theta, notQ^(1+D+K)); Fisher-preconditioned GEM M-step; Jeon et al. (2021) LSIRM priors as MAP penalties (PenaltyConfig::lsirm_prior).sigma_u/ICC) populations; FIPC anchors (Kim 2006, MWU-MEM-style) withPopulationSpec::SingleFree; concurrent calibration = multigroup + structural missingness (Hanson-Beguin), with a recovery test.xi_rule = gh | qmc | mc): Halton (+ Cranley-Patterson shift) and seeded-MC node sets (Jank 2005; Wei-Tanner 1990; Meng-Schilling 1996), liftinglatent_dim <= 3for the grid rule.zero_inflation=True; Perumean-Chaney 2013 guidance) and a context-varying item covariate with estimated coefficient (Debeer-Janssen 2013 position effects).gpu_marginal.rs): ~110 s -> ~5 s per multilevel E-step iteration on a 31k x 57 dataset (RTX 3050 Ti); race-free slot-ownership reductions; M-step and final EAP stay CPU f64; 1 GiB buffer guard with CPU fallback.Scoring & serving (
scoring.rs,serving.py)fast-mlsirm scoreCLI.Fit statistics & validation (
fitstats.rs,agreement.rs,oakes.rs)rms_residualpractical-significance guard (Sinharay-Haberman 2014),l_z/l_z*(Snijders, prior-mean-centeredr_0), infit/outfit, BH-FDR, information criteria (Kang-Cohen-Sung 2009), Vuong tests (Schneider et al. 2019), Q3/GDDM (Svetina-Levy 2014), residual item fit (Haberman et al. 2013), adjusted pairwise chi2/df (Tay-Drasgow 2012), Chen-Thissen (1997) LD indices, resampling person fit (Sinharay 2016), TCC drift (Guo et al. 2015).validate_judge): QWK/r/SMD/degradation/subgroup-SMD with the paper's thresholds.select_items()screening pipeline,dif_analysis()(virtual-item LR; Jeon-Rijmen-Rabe-Hesketh 2013 + Makransky-Glas 2013),irtree_expand()(Jeon-De Boeck 2016).Contracts
tests/test_marginal_parity.py).mlsirm-core; Python is orchestration + parity references.cargo test --workspace+ binding-crate manifest run).estimator="mmle"fast path for plainULS2PLM/ULSRMunchanged; spatial models now fit instead of raising.Real-data validation (31,356 x 57, 307 teams)
Zero-inflated multilevel MLS2PLM selected by BIC (Δ=310 vs plain multilevel; Δ=2,336 vs single); ICC = 0.80; π = 0.21; corrected screening retains 21/57 items with a full audit; serving bundle + EAPsum tables exported and smoke-tested end to end (EAP/MAP/EAPsum).
Explicitly-scoped follow-ups (docs/papers/corpus-triage-batch4.md, priority order)
BIFAC2PLMbifactor family (Gibbons-Hedeker reduction slots into the existing conditional-factorization E-step)🤖 Generated with Claude Code