Switch from n(z) A/B/C to Smokescreen blinding: per-part-at-birth, hash-commitment custody (PR 6) - #253
Open
cailmdaley wants to merge 32 commits into
Open
Switch from n(z) A/B/C to Smokescreen blinding: per-part-at-birth, hash-commitment custody (PR 6)#253cailmdaley wants to merge 32 commits into
cailmdaley wants to merge 32 commits into
Conversation
This was referenced Jul 10, 2026
cailmdaley
changed the title
cailmdaley
force-pushed
the
feat/sacc-6-blinding
branch
from
July 10, 2026 22:52
971767b to
11b8895
Compare
cailmdaley
changed the base branch from
feat/sacc-5-firecrown-likelihood
to
feat/sacc-2-sacc-io
July 10, 2026 22:52
cailmdaley
changed the title
cailmdaley
added a commit
that referenced
this pull request
Jul 11, 2026
…ll image from lock (#266) * felt: fork-implementation — #243 green, #253 rework under fix; fork PRs reviewed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KpaRHk3QwN13myduQ3hJyf * felt: fork-implementation — constitution absorbs #241 re-rulings (one-file layout, per-part blinding at birth) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KpaRHk3QwN13myduQ3hJyf * deps: declare cosmo_numba + numba, adopt committed uv.lock, install image from lock Make sp_validation's environment reproducible so an image build can never re-resolve numpy past numba's ceiling — the drift that silently upgraded numpy to 2.5 and broke numba/ngmix. - Declare cosmo-numba (aguinot/cosmo-numba@main; not on PyPI) — the numba B-mode kernels b_modes.py imports. main carries numpy-2 FFT support via its rocket-fft dep and declares its own deps, so numba's numpy window reaches the resolver. Also pin numba directly: the one load-bearing constraint, made visible and resilient to cosmo-numba's dep metadata (which has emptied out between refs). - Declare the other imported-but-undeclared deps the audit found: matplotlib, pandas, pyyaml (core, src/); fitsio (glass extra); a new `workflow` extra for snakemake + mpi4py. cv_runner and unions_wl left undeclared (no resolvable source) with a NOTE. - requires-python and ruff target -> 3.12 (the container's Python; cosmo-numba's floor). - Commit uv.lock (un-ignored) as the SSOT; scope it to Linux via [tool.uv]. numpy resolves to 2.4.6, inside numba 0.66's <2.5 window. - Dockerfile installs from the lock: `uv sync --frozen --inexact` into the base image's /app/.venv (--inexact keeps the ShapePipe stack). Drops the ad-hoc snakemake layer and the cs_util `--upgrade` workaround — the lock pins cs_util 0.2.2 (with cs_util.size), decoupling us from the base image's cs_util. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vtw1qcgTrQ6YuMvxwYzNup * felt: uv-lock-cosmo-numba — @main resolution, install model, cosmo-numba gotcha finding Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vtw1qcgTrQ6YuMvxwYzNup * test: don't load base image's stale pytest-pydocstyle/pycodestyle plugins uv sync installs a newer pytest than the base ShapePipe image's pytest-pydocstyle/pytest-pycodestyle (>=2.4) expect; their pytest_collect_file hooks use the removed `path` arg and crash collection. sp_validation lints with ruff, so disable both via addopts (`-p no:`). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vtw1qcgTrQ6YuMvxwYzNup --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
cailmdaley
force-pushed
the
feat/sacc-2-sacc-io
branch
from
July 16, 2026 00:39
55cb97d to
4489dd4
Compare
…abulary
Rebuild the per-part-at-birth blinding stack on top of feat/sacc-2-sacc-io.
sacc_io.py is now PR-2's canonical module verbatim + a single appended
gather(): the terminal assembly delegates its tracer/point/covariance
assembly to PR-2's merge() and adds only the blind-custody call
(assert_consistent_blind) and shared-stamp write.
The fail-closed load gate lives in sacc_io.load() per the PRD ("sacc_io fails
closed on load"); the blinding tooling uses allow_unblinded=True explicitly
where it must read a not-yet-blinded real vector (blind_part). save() now
carries the required type= (inherited from each part's provenance). Grid
vocabulary swept coarse->reporting, fine->integration across blinding.py,
blinding_theory.py, b_modes.py, blind_data_vector.py, and the blinding tests.
_extract_block/_set_values stay index-based (a code comment says why):
Smokescreen's ConcealDataVector aligns theory_fn output to the sub-SACC mean
by row position, so the block must be carved/written by contiguous integer
index, not by PR-2's tag-matching extract()/update_statistic().
Escrow/custody/CAMB<->CCL machinery is preserved exactly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WzUt7VbtXwr2SCHUdiQTyt
…nfig Three integration-drift fixes surfaced by the reconciled base: - test_ac6_ac8 / test_ac8_dotted: the end-to-end fixtures now pass type="mock" to sio.save (PR-2 requires the provenance stamp); the parts are mocks, blind_part re-saves inheriting that type. - test_ac13: the CosmoSIS halofit config moved to cosmo_inference/cosmosis_config/templates/ on develop; point the AC13 assertion at the new path (token unchanged: mead2020_feedback). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WzUt7VbtXwr2SCHUdiQTyt
cailmdaley
force-pushed
the
feat/sacc-6-blinding
branch
from
July 16, 2026 09:55
4b058f7 to
c09b063
Compare
cailmdaley
added a commit
that referenced
this pull request
Jul 21, 2026
Fold the integration-grid ξ± into the single terminal {version}.sacc, and
make part loading fail closed on unblinded real data.
Integration ξ± (grid='integration') is no longer its own terminal product.
The xi_highres part is now gathered by rule assemble_sacc into {version}.sacc
as tagged points, next to the reporting ξ± block. It is fiducial-only (the
10k-bin MPI run emits only the fiducial part), so it joins the fiducial
version's terminal file alone. assemble_sacc.py adds xi_integration to
CANONICAL; its own DiagonalCovariance passes straight through.
assemble_sacc.py no longer loads every part with allow_unblinded=True. The
run type (data|mock, from config, default data) gates it: mock runs load
freely, data runs fail closed unless a part carries the concealed=True stamp.
This is the seam for PR #253's blind-at-birth — a concealed data part then
assembles with allow_unblinded=False untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
… feat/sacc-6-blinding
Author the Snakemake rules that realise sp_validation.blinding's three-verb custody surface on the migrated cosmo_val workflow (issues #247/#252). - rule blind_init (once per catalogue version): draws the seed, publishes commitment.json + the encrypted seed bundle. Refuses to overwrite state. - rule blind_part (generic over the three blindable stems — reporting ξ±, integration ξ±, analysis pseudo-Cℓ): conceals the part at birth, escrows the true vector beside the blinded output. The plaintext part is a temp() output of its producer on a data run, so only its blinded sibling persists. - common.py: run-type-aware path helpers (blindable_part / maybe_temp / blind_state_paths / version_of) so a data run binds ξ-derived consumers to the blinded siblings and a mock run bypasses blinding entirely — the whole blinding subgraph appears or vanishes with RUN_TYPE. - assemble_sacc: assert_consistent_blind across parts (assembly-time commitment check of #252), stamping the shared blind on the terminal file. - Constrain the npatch wildcard to \d+ so a producer's ξ± output cannot absorb the _blinded suffix (which made rule xi ambiguous with blind_part). The cosmo_val assemble DAG dry-runs: blind_init x2, blind_part x5, assemble consuming the blinded ξ± + pseudo-Cℓ parts. Born-blinded COSEBIs/pure-E/B and the ρ/τ concealed pass-through follow in the next commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
- common.py path helpers mirror sp_validation.blinding init_paths/part_paths (drift guard), version_of on all three blindable stems, run-type switch. - Data-run fail-closed assembly: assemble_sacc refuses an unblinded type=data part, passes when every part is concealed under one commitment, and refuses a blinded/plaintext mix and divergent commitments. - Candide-only: the blinding subgraph (blind_init/blind_part) resolves in the cosmo_val assemble dry-run and binds the blinded ξ± + pseudo-Cℓ siblings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
cailmdaley
added a commit
that referenced
this pull request
Jul 21, 2026
Keep the integration-grid ξ± as its own per-part intermediate
({version}_xi_integration.sacc) rather than folding it into the terminal
{version}.sacc. Per the #247 ruling (comment 5033716753), the terminal file
carries the analysis vector only; COSEBIs/pure-E/B consume the integration
part directly, and Snakemake provenance covers its traceability. This keeps
the terminal file at tens-of-MB scale.
Removes xi_integration from CANONICAL, the cv_xi_integration_sacc helper, and
the fiducial-gated assemble input. Keeps the fail-closed allow_unblinded gate,
glass A/B/C comments, and dependency restorations from the prior rework. The
integration part stays blinded at birth on data runs (per #253).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
Merge the updated feat/sacc-4-cosmo-val-sacc, which removes the integration-grid
ξ± from the terminal {version}.sacc gather (per the #247 ruling: it persists as
its own per-part intermediate, consumed by COSEBIs/pure-E/B, with Snakemake
provenance covering traceability).
Blinding wiring: the blind_part rule and its BLINDABLE_STEM regex still cover the
integration ξ± (it remains blindable at birth as a per-part file). Only the
assemble subgraph changes — cv_assemble_inputs no longer requests the blinded
integration part, so the dry-run wiring test asserts _xi_integration_blinded.sacc
is NOT pulled into the assemble subgraph.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
cailmdaley
added a commit
that referenced
this pull request
Jul 21, 2026
…-version xi_highres COSEBIs and pure-E/B now derive their E-mode outputs from the born-as-SACC ξ± parts on disk instead of only the raw catalogue recompute. This makes born blinding automatic downstream: whatever the part carries (blinded on data runs in PR #253) flows into the E-modes with no blind-aware consumer code. b_modes: add the values-only seams cosebis_from_xi and pure_eb_from_xi (copied from the sacc-6b spur, de-blinded) — same cosmo_numba kernels as the raw path. cosebis_to_sacc_part / pure_eb_to_sacc_part: add en_override / eb_override so the E-mode En (COSEBIs) and the six pure-mode arrays (pure-E/B) written to the SACC part come from the consumed part; Bn and the covariance stay blind-invariant from the raw estimator run. No concealment machinery (that is PR #253's). cv_cosebis.py / cv_pure_eb.py: keep the raw plot_* run (Bn + jackknife covariance need the catalogue and are blind-invariant), then load the integration part (COSEBIs) / reporting + integration parts (pure-E/B), re-derive the E-modes through the seams, and override both the SACC part and the diagnostic npz. pure-E/B reads the reporting-grid bin edges from the raw reporting gg (SACC stores centers only). Unconditional and version-agnostic. xi_highres: per version (was fiducial-hardcoded); in-container single-process at the config-driven 1000-bin grid (the 10k-bin bare-host MPI path is unnecessary). Grid comes from a dedicated cosmo_val.integration block ([0.08, 300] @ 1000) so the one part serves both consumers (pure-E/B full range, COSEBIs scale-cuts to 0.9); decoupled from covariance.smk's FIDUCIAL grid. Shared twopoint.smk falls back to the fiducial integration grid for configs without a cosmo_val section (e.g. papers/bmodes). The raw .txt byproduct is left undeclared to avoid an AmbiguousRuleException with rule xi; nothing in the DAG consumes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
…g-aware
Merge the updated feat/sacc-4-cosmo-val-sacc (cv_cosebis/cv_pure_eb consume the
ξ± SACC parts via the b_modes seams; per-version in-container xi_highres on the
dedicated cosmo_val.integration grid; the cosmo_inference zombie purge; the
1000-bin default).
On this blinding branch the consumers bind the *_blinded* parts on data runs and
stamp their derived output concealed, so born blinding flows end to end with no
blind-aware consumer code:
- cv_cosebis/cv_pure_eb inputs wrap the ξ± parts in blindable_part (→ _blinded
sibling on a data run, plaintext on a mock run) and bind commitment.json on a
data run — per version, not fiducial-gated.
- cv_cosebis.py/cv_pure_eb.py thread commitment_path to the *_to_sacc_part
writers; cosebis_to_sacc_part/pure_eb_to_sacc_part re-derive En / the six
pure-mode arrays from the (blinded) part and stamp the emitted part concealed.
- xi_highres keeps its maybe_temp blindable wrapper on {version}_xi_integration.sacc,
so blind_part produces the _blinded sibling the consumers now request.
The dry-run wiring test asserts _xi_integration_blinded.sacc IS in the cosmo_val
subgraph again (the consumers are its explicit consumers on a data run).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EaL7prmKUHwJQcDyW3LoxD
…ustody Two changes to blinding.py, both about the seam with Smokescreen. **The concealing factor is a vector, so ask for a vector.** `_concealing_factor` carved a sub-SACC per block (`s.copy()` + `keep_indices`, covariance and all), handed it to `ConcealDataVector`, applied the blind, and subtracted the result back off to recover the shift it wanted in the first place. The fork now exposes `concealing_factor(fiducial_params, shifts_dict, *, seed, theory_fn)` — the pure theory difference, no SACC, no data vector — so call that. The theory factories already read their layout off whatever SACC they are handed, so they run on the part directly and return a full-length vector, NaN outside their own block; the factor is that vector at the block's rows. `unblind_sacc` differenced the two theory vectors itself. It now goes through the same `_concealing_factor` call, so the shift added and the shift subtracted cannot drift apart. `_extract_block` is gone with its last caller. A backend that leaves a row of its *own* block unfilled would have shifted that point by NaN, silently. The slice is now checked finite and refuses instead. **A blind is (seed, config, draw scheme) — not (seed, config).** The fork versions its shift-draw semantics as `smokescreen.DRAW_SCHEME`: upstream's one global RNG stream over sorted keys is scheme 1, the fork's per-key `(seed, key)` RNG is scheme 2. Blind under one and unblind under the other and the same seed draws a *different* hidden cosmology — so the unblind subtracts a shift that was never added, leaving a smooth residual in the "revealed" vector while the seed hash, the config digest and the escrow equality check all still pass. It is the one blinding failure with no symptom. The scheme is now custody state. `blind_init` writes it into commitment.json; `_stamp_provenance` stamps `blind_draw_scheme` on every blinded file; and it is re-checked against the installed fork wherever a shift is drawn or subtracted — `_read_seed` (so a scheme change between blinding part 1 and part 2 is caught), `unblind_sacc` before any subtraction, `assert_consistent_blind` at the terminal, `stamp_concealed_passthrough`, and the CLI's seedless `verify`. A missing record fails closed: a blind whose scheme is unknown cannot be shown reproducible. Docstrings that promised same-(seed, config) reproducibility "forever", or per-key draw independence, now say what guarantees it. Closes review findings 1, 2 and 4 on PR #253. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRYg9fjMevgcsvgjh3KN6x
`sacc_io.gather` was the documented custody terminal — it asserts one blind across all blindable parts, then assembles — but nothing in production called it. `workflow/scripts/assemble_sacc.py` inlined its own copy of the same wrapper (assert, assemble, stamp) around a *different* assembler, and blinding.py's docstrings claimed gather guarded the terminal. Two custody implementations, one of them dead, and a docstring true of neither. The difference between them is only the assembly: `merge` unions parts that already carry covariance; `assemble_analysis_sacc` rebuilds from an n(z) and requires one covariance block per part, which is what the production terminal needs (its ξ± and pseudo-Cℓ parts are born cov-less and have blocks injected first). That is a real difference and both are worth keeping — so the assembler becomes an argument, `gather(parts, metadata=None, assemble=None)`, and assemble_sacc.py passes its own in rather than wrapping its own guard around it. The custody wrapper now exists once and cannot be routed around. Tests: the production path gets the case the fail-closed load gate cannot catch — every part concealed, so every part loads, and only `assert_consistent_blind` can see that the install's draw scheme is not the blind's. Plus the terminal file's draw-scheme stamp. Closes review finding 3 on PR #253. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRYg9fjMevgcsvgjh3KN6x
The local sha256(seed) commitment leaked the blind. Smokescreen derives its per-key RNG base seed from the first 8 bytes of the *same* undomained digest, so the published commitment carried that base seed verbatim in its first 16 hex characters. With the commitment and the (public) fiducial config, anyone could redraw the hidden cosmology and subtract the shift — from the artifact introduced to protect it. seed_commitment now delegates to smokescreen.seed_commitment, which hashes COMMITMENT_DOMAIN + str(seed). One definition of the commitment across the stack, in a different hash domain from the seed normalizer. A regression test asserts the commitment never embeds _normalize_seed(seed). The commitment.json key is renamed seed_sha256 -> seed_commitment: it is no longer sha256 of the seed, and the old name claimed otherwise. No real blind exists, so nothing migrates. Also, four honesty fixes the same review found: - The NaN refusal fires on both blind and unblind (they share one call), so its message no longer says "refusing to blind". - The CLI module docstring describes the three-way commitment (seed, config, draw scheme), and _verify's docstring says that its draw-scheme check makes the result environment-dependent by design. - _verify loads with allow_unblinded=True, so its "file is not marked concealed" diagnostic is reachable instead of an uncaught load exception. - The module docstring states what a pre-draw_scheme blind costs: recoverable only by hand from escrow. None exists; the CLI offers no override. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRYg9fjMevgcsvgjh3KN6x
Two claims the code could not cash, both at the terminal assembly. The mock branch of assert_consistent_blind was unreachable. Every part writer hardcoded type="data", so a mock campaign's parts declared themselves data and a mock terminal assembly failed closed — only test fixtures ever built a type="mock" part. The run type is now plumbed to all six writers: CosmologyValidation carries run_type (set from config cosmo_val.type via cv_init_params) and its four writers stamp it; run_2pcf takes run_type= and a --run-type flag; run_2pcf_highres takes --run-type. papers/cosmo_val declares `type: data` explicitly, so the switch is visible where it is made. rho/tau was never stamped. The docstring said it passed through stamp_concealed_passthrough, but nothing called it for rho/tau — psf_systematics wrote the part with no commitment, so a data run's assemble_sacc died at sacc_io.load(allow_unblinded=False) on that part, before custody was ever checked. The rho_tau_stats rule now binds commitment.json on a data run and run_rho_tau passes it through calculate_rho_tau_stats, exactly as cv_cosebis and cv_pure_eb do. rho/tau carries no cosmological vector; the stamp shifts nothing and only clears the load gate. Tests: a mock campaign's parts assemble unconcealed, and the real rho/tau writer emits a part that loads without the escape hatch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRYg9fjMevgcsvgjh3KN6x
sacc_io.save grows a commitment= kwarg and owns the passthrough stamp (three copy-pasted writer tails collapse); commitment_input() in workflow/common.py is the one place a data run binds commitment.json into rule inputs; the CLI's verify delegates draw-scheme checks to _assert_draw_scheme instead of re-wording them; _stamp_provenance loses its derivable scheme parameter. Draw-scheme rationale stated once (draw_scheme), mock-assembly rule once (assert_consistent_blind); dead escrow-recovery prose cut; doubled test assertions deduped; is_data_run reads run_type(). Container suite: 70/70 blinding+wiring+dry-run, 282 passed overall; 3 failures pre-existing and environmental (path checks on other users' scratch, stale-container glass import, a pure-eb value pin that fails identically on the unedited base). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G9MahwJEQ1t9EuvUXijmy3
# Conflicts: # src/sp_validation/sacc_io.py
# Conflicts: # workflow/rules/cosmo_val.smk # workflow/rules/twopoint.smk # workflow/scripts/run_2pcf.py # workflow/scripts/run_2pcf_highres.py
# Conflicts: # src/sp_validation/cosmo_val/cosebis.py # src/sp_validation/cosmo_val/pure_eb.py # src/sp_validation/cosmo_val/sacc_writers.py # workflow/common.py # workflow/rules/cosmo_val.smk # workflow/rules/inference.smk # workflow/rules/twopoint.smk # workflow/scripts/assemble_sacc.py # workflow/scripts/cv_cosebis.py # workflow/scripts/cv_pure_eb.py # workflow/scripts/run_2pcf.py
Comments/docstrings: state each load-bearing invariant once at its home (custody triple in blinding's module docstring, scheme-mismatch in draw_scheme, assembly gate in assert_consistent_blind, NaN guard in _concealing_factor) and point at it from elsewhere. Code: - blinding_paths: new dependency-free module owning init_paths/part_paths; blinding re-exports it and workflow/common imports it instead of hand-mirroring the layout. - coerce_fields: one unknown-key-check + float-coercion helper behind both from_overrides and config_digest (digests unchanged). - _pairs, _apply_blocks, functools.partial: collapse the duplicated pair-discovery and blind/unblind block loops. - blinding.verify() is public and drives the CLI, which drops its duplicate overwrite pre-check in favour of the library's FileExistsError. - Memoize WeakLensingTracer per (cosmology, bin), mirroring _COSMO_CACHE; share the CAMB matter-power z-grid between make_camb_params and xi_camb. - CosmologyValidation takes blind_root at init and resolves each version's commitment itself, instead of threading commitment_path through the three part writers (fixes the latent one-commitment-for-every-version stamp in the psf_systematics per-version loop). - blinding.smk derives BLINDABLE_STEM from the name-builders, and its two rules shell out to scripts/blind_data_vector.py; the workflow/scripts wrappers are gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014hSQTC6WwTH1p9w4FuKJGz
…e temp()
The generic `pseudo_cl` rule serves every variant (bmodes claims, mocks, the
fine COSEBIs binning), so its output could not be temp()'d and the unblinded
analysis part persisted on data runs. The analysis variant now has its own
rule and its own name (pseudo_cl_analysis_{version}_{tag}.sacc, built by
common.pseudo_cl_analysis_stem — one authority for producer, assembler and the
blindable-stem regex), whose plaintext is maybe_temp'd: on a data run only the
blinded sibling persists. The generic rule is unchanged for the other workflows.
The A/B/C blind is n(z) vocabulary, not this part's concealment, so the
analysis part is now pseudo_cl_analysis_{version}_{binning}_nbins={n}.sacc.
pseudo_cl_binning_tag() is the shared half; pseudo_cl_tag() (blind= included)
still names the generic pseudo_cl / pseudo_cl_cov products.
Conflicts resolved keeping this branch's blinding facts with #251's trimmed comment phrasing (no removed comment regrown).
cv_cosebis now consumes the COSEBIs grid's blinded ξ± part (one input, through blindable_part) and stamps its own output from the version's commitment, so the COSEBIs part stays born-blinded under the new single-rule shape. The blindable stem list is built from the grid table, so the new grid joined it unchanged.
cv_pure_eb now binds the two blinded ξ± parts plus the integration covariance and stamps its own output from the version's commitment, so the pure-E/B part stays born-blinded with no catalogue access anywhere in the rule.
The pseudo-Cℓ figures and the B-mode summary now read the analysis part, so on a data run they bind its blinded sibling like every other consumer — the plaintext part stays temp() and unread. The blinding dry-run test asserted the un-normalised ξ± name (maxsep=300), which is exactly the producer/consumer mismatch the grid-table canonicalisation fixes; it now asserts the canonical one. The assembly tests supply the analytic ξ± covariance the terminal file takes.
cailmdaley
marked this pull request as ready for review
August 31, 2026 16:13
cailmdaley
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #244
Blinding is per intermediate product, at birth.
blind-initdraws the seed once per catalogue version and publishes only its commitment; each blindable part (coarse ξ±, fine ξ±, pseudo-Cℓ) is shifted the moment it is computed; only blinded intermediates persist on disk. The analysis pseudo-Cℓ variant has its own rule (pseudo_cl_analysis, stempseudo_cl_analysis_{version}_powspace_nbins=32.sacc— no legacyblind=field, see #312), so its plaintext intermediate ismaybe_temp'd like the rest. Two disclosed residuals: the raw ξ±.txtbyproducts feeding the CosmoCov covariance seam (closes when the covariance consumer migrates), and the operator-onlypseudo_cl_all/pseudo_cl_fine_allconvenience targets, which can still materialize plaintext spectra through the generic glass-mock rule — outside the analysis DAG, untouched by the assemble path. COSEBIs and pure-E/B are derived from the already-blinded fine ξ± and stamped as concealed passthroughs. Terminal assembly (sacc_io.gather(assemble=...), the single custody seam) asserts every part carries the same commitment.Legacy A/B/C blinding: replaced, then scrubbed. This PR introduces the sole blinding mechanism going forward; the pre-Smokescreen blind strata (catalogue A/B/C path,
blind=filename tags) are deliberately untouched here to keep the diff reviewable, and are removed in #312 — which gates the first real assembly run, so no real product is ever made while two blinding vocabularies coexist.Mechanics
concealing_factor/factor_from_params): blind and unblind are one code path operating on vector slices — no sub-SACC copies. Theory is the fork protocol'stheory_fn; the CAMB↔CCL cross-check runs as tests (σ8-matched, halofit strings matched).commitment.jsonholdsseed_commitment(the fork's domain-separated digest — the seed itself is never written) anddraw_scheme. Every shift/subtract gate asserts the installed Smokescreen'sDRAW_SCHEMEagainst the committed one, so an install with different draw semantics refuses to touch the data rather than silently applying the wrong shift.data/mock): mock parts are writable end-to-end and declare themselves; an unconcealed blindable part only assembles when it declares itself a mock (seeblinding.assert_consistent_blind). rho/tau statistics ride the same passthrough stamping.blind-initis a DAG node; data runs bindcommitment.jsoninto each blindable rule's inputs (common.commitment_input), so the blind exists before any part is computed.Surfaces
src/sp_validation/blinding.py— seed custody, commitment, shift application, consistency gates (publicverify);blinding_paths.pyholds the dependency-free path layout the DAG shares.src/sp_validation/blinding_theory.py— CCL fiducial theory (theory_fnfactories, per-bin tracer cache) + the independent CAMB cross-check path.src/sp_validation/sacc_io.py—save(..., commitment=)stamps concealed passthroughs;gather(assemble=)is the terminal seam.scripts/blind_data_vector.py— CLI: init / status / verify / unblind (unblind is deliberately manual).workflow/rules/blinding.smk(rules shell out to the CLI; blindable stems derived from the part name-builders) + rule-input wiring incosmo_val.smk/twopoint.smk;CosmologyValidationresolves each version's commitment itself (blind_root).Verified
Blinding + wiring + migration suites green on this head (allocation-run), including the blinding-subgraph Snakemake dry-run. The blinding tests pin: commitment delegation to the fork (domain-separated, seed not embedded), draw-scheme fail-closed at every gate, blind→unblind round-trip bit-exactness, mock-vs-data assembly gating, CAMB↔CCL agreement.
— Claude (Fable + Opus) on behalf of Cail.
🤖 Generated with Claude Code
https://claude.ai/code/session_01G9MahwJEQ1t9EuvUXijmy3