Data-quality findings from a full-coverage conversion of the CR26 consolidated rules (v2026.07.14.01)

[christoph-puppe]

Hi all

sharing findings from an experiment I ran in personal capacity. I wrote a converter that maps the complete consolidated rules JSON into an experimental semantic model, with a computed coverage report: every source field path inventoried, every path assigned a declared destination. Result: 7,294 leaf values, 100 % declared coverage, zero unexplained fields. The process surfaced a few things that may be useful to the FedRAMP data team:

1. Nine undeclared subsets. CSX (in 20x data blocks) and CSF (in rev5 data blocks) appear under FRR.*.data.* for CDS, CPO, FRC, IVV and SDR, but are not declared in the corresponding info.subsets. Tooling that builds applicability scopes from info.subsets will silently miss these rules.
2. Layered rule counts. The corpus contains 246 rules = 225 track-independent (data.all) + 12 rev5-only + 9 20x-only. Worth documenting, since naive parsers that read only data.all will report 225.
3. 29 variant-only rules exist solely inside varies_by_class with no base statement/force — consumers need a defined interpretation for the base case.
4. Consistency compliment: across all 111 class-variant force changes, there is not a single easing — every published class delta tightens or specifies. That's a remarkably clean dataset property.
5. FRD quality: all 264 term references from rules and KSIs resolve against the FRD via term + alts matching — the alias lists demonstrably earn their keep. The 51 variant timeframes without a base timeframe and the 79 CTL overlay entries are documented in the report as well.
6. The 373 KSI→800-53 control links import cleanly as typed, provenance-carrying mapping objects.

The converter script and the full coverage report (JSON: every path → destination, with counts) are available. The wider experiment — a semantic-core architecture hypothesis derived from measuring three national corpora (ISM, BSI IT-Grundschutz++, CR26), with a handbook and worked examples packaged as an AI skill — is here: Semantic OSCAL Ideas. Criticism of the architecture itself is explicitly welcome; happy to file any of the data findings above as separate issues if useful.

A summary can be found on LinkedIn

— Christoph Puppe (personal capacity)

[pete-gov]

1. Nine undeclared subsets. CSX (in 20x data blocks) and CSF (in rev5 data blocks) appear under FRR.*.data.* for CDS, CPO, FRC, IVV and SDR, but are not declared in the corresponding info.subsets. Tooling that builds applicability scopes from info.subsets will silently miss these rules.

For this one I'm hoping it's a structural misunderstanding - subsets are declared in info.subsets if they apply to both Rev5 and 20x. If a subset applies only to one of those, then it is declared in info.20x.subsets or info.rev5.subsets instead.

For example, CSF is a Rev5-only subset so it is declared in info.rev5.subsets. CSX is a 20x-only subset so it is declared in info.20x.subsets.

3. 29 variant-only rules exist solely inside varies_by_class ...

Yep, this is explicit in the schema, there either needs to be a statement/etc. OR there needs to be a varies_by_class with the statement/etc.

Similar to the structure for (1) above - things apply globally by default, but in some cases there are specific applications and those are then defined on their own.

Cheers!

[christoph-puppe]

Thanks @pete-gov — confirmed, structural misunderstanding on my side, and finding 1 is withdrawn.

My check only looked at the common info.subsets block. Re-running it including info.20x.subsets / info.rev5.subsets: zero undeclared subsets — all nine are declared exactly as you describe, with full name, description, and applicability. The global-by-default / specific-when-needed layering is clean design; my checker just wasn't layered enough. The embarrassing part: AGENTS.md states this resolution rule explicitly, and I cited that file in my post without following its clearest instruction. Fair cop.

I'll adapt my converter code to resolve exactly this misunderstanding: it will read the framework-specific info.20x.subsets / info.rev5.subsets declarations alongside the common block — which actually improves the conversion, since the nine framework-specific subsets then carry their real declared titles and applicability instead of synthesized ones. Updated converter and coverage report will land in my repo, and I'll edit the original post above with strikethrough to match.

On (3): agreed — the either/or is explicit in the schema, so the pattern is intentional and finding 3 narrows to a small question rather than an issue: is there (or would you want) presentation guidance for class-agnostic consumers — search indexes, generic exports — on how to display a variant-only rule outside any class context? Entirely optional; happy to drop it if the answer is "class context is always required."

I've also downgraded my own finding 2 to plain statistics — the all/20x/rev5 structure is documented; the counts (246 = 225 + 12 rev5-only + 9 20x-only) just stand as descriptive data. What survives review: zero easings across the 111 class-variant force deltas, 264/264 term references resolving via FRD term+alts, and the 373 KSI→control links importing cleanly.

Exactly the kind of review I was hoping for. Cheers!

[christoph-puppe]

Following up as promised: the converter fix landed, and the whole experiment has now shipped as JASCON v1.0.0.

For CR26 specifically: the corpus converts with UNMAPPED = 0 — the layered info.20x.subsets / info.rev5.subsets resolution from your correction is now the reference behavior — exports back to schema-valid OSCAL 1.2.2, and the 373 KSI→800-53 links round-trip as typed, provenance-carrying mapping objects. The converted corpus and computed coverage report are in converted_examples/.

Verify it yourself — Linux/macOS:

uv run --with jsonschema python semantic-oscal/scripts/validate_core.py

powershell -ExecutionPolicy Bypass -File semantic-oscal/scripts/validate_core.ps1

Thanks again for the review — it's baked into the release. Cheers!