Repository navigation
fix(sensitivity): refit at fixed multipliers of the slab scale, classify on the fits' own statistics - #221
Open
MaartenMarsman wants to merge 3 commits into
Open
fix(sensitivity): refit at fixed multipliers of the slab scale, classify on the fits' own statistics#221MaartenMarsman wants to merge 3 commits into
MaartenMarsman wants to merge 3 commits into
Conversation
…assify on the fits' own statistics prior_sensitivity_check() refits the model at scale_multipliers (default c(0.5, 2)) times the chosen scale, each refit with the original fit's iter and warmup, warm-started for ordinal fits. Each fit contributes its own Rao-Blackwellized inclusion probability, log Bayes factor, verdict and the MCSE of the log Bayes factor. An edge is stable when every refit's verdict equals the original fit's; otherwise it is beyond Monte Carlo error when the log-BF change to a refit with a different verdict exceeds twice the two fits' combined MCSE, within it otherwise. insufficient at the chosen scale is the only not-certifiable criterion. Each fit's diagnostics are in $grid; no fit is excluded. The reweighted curve, its plot method, the replicate refit and noise band, the short refit schedule and the convergence gate are removed; the colour palette the other plots use moves to plot_style.R.
…int, and the check on small fits The curve tests and the weekly reweighting block go with the curve. The plot-method tests that read the slab frame through the removed draw extractor compare the extractor with the raw draws directly.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #221 +/- ##
===========================================
+ Coverage 92.84% 93.14% +0.29%
===========================================
Files 115 114 -1
Lines 18697 18495 -202
===========================================
- Hits 17360 17227 -133
+ Misses 1337 1268 -69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
prior_sensitivity_check()becomes a fixed-scale sensitivity analysis: full-length refits at a few multipliers of the chosen slab scale, classified on the fits' own statistics. The importance-reweighted curve between anchors, its plot method, the replicate refit with its noise band, and the convergence gate are removed. Replaces the closed #220.What the check does now
scale_multipliers(defaultc(0.5, 2); the argument wasanchors) with the original fit'siterandwarmup. The original fit is the 1x fit and is not refit. Ordinal refits start from the original fit's final state and NUTS metric; GGM and mixed refits start cold (warm starts for those: backlog Item 48).stablewhen every refit's verdict equals it; otherwisebeyond-monte-carlo-errorwhen, at some refit whose verdict differs, the log-BF change from the original fit exceeds twice the two fits' combined MCSE, andwithin-monte-carlo-errorwhen it does not at any.moved_atis the differing multiplier nearest 1.stability_lowerandstability_upperare the ends of the contiguous run of multipliers around 1 holding the original verdict.insufficientat the chosen scale is the only not-certifiable criterion, computed as before.$grid; the print names fits above R-hat 1.01. All fits enter the classification.prior_sensitivity_engine()runs the refits and takes the per-fit tables;sensitivity_classification()classifies from the tables. The refit settings are stored per fit in$grid(backlog Item 49, rerunning one fit longer, builds on this).Removed:
R/anchor_curve.Rand its tests,plot.bgms_prior_sensitivity(), the argumentsanchors,ess_floorandtolerance, the short refit schedule, the replicate refit and$wobble,refit_convergence_gate(), the weekly reweighting block, the mover levelsindistinguishable-from-wobbleandmoved-beyond-wobble, the curve matrices and$curve.$anchor_verdictis$verdict_by_scale.Commits
Wenchuan (tutorial script R/16: 15 variables, 136 edges, 4 x 10000 draws, 5000 warmup)
Fit 137 s, check 245 s (two refits at the fit's length). Printed report:
CRAN build on the same script: robust 91, changed within noise 6, changed beyond noise 31, not certifiable 8, from five short refits and a reweighted curve.
Verification
tests/testthat/test-prior-sensitivity.R: 79 pass at tier T0; the slow and certification blocks (bgmCompare, GGM and mixed, warm-versus-cold refit) were not run.dev/review-2026-08/repro/run-anchor.shon the six-variable Wenchuan fit (2 chains, 2000 draws): develop fits five short refits plus a replicate, reports a 43-point curve and names one edge as scale-dependent whose verdict is undecided at every anchor; the branch fits two refits at the fit's length, has no curve or plot method, and reports that edge as held, with the one not-certifiable edge the only exception. Comparison in~/bgms-review/anchor-comparison.txt.