diff --git a/docs/adr/0062-normalise-wire-to-core-at-both-ingress-and-persistence.md b/docs/adr/0062-normalise-wire-to-core-at-both-ingress-and-persistence.md new file mode 100644 index 000000000..141ffaf6b --- /dev/null +++ b/docs/adr/0062-normalise-wire-to-core-at-both-ingress-and-persistence.md @@ -0,0 +1,164 @@ +--- +status: accepted +date: 2026-08-13 +deciders: Vultron maintainers +consulted: Vultron maintainers +informed: Vultron contributors +--- + +# Normalise Wire → Core at Ingress, and Enforce It Again at the Persistence Boundary + +## Context and Problem Statement + +`ParticipantStatus` and `CaseParticipant` each exist in two structurally +incompatible shapes. The core types nest their dimensions +(`rm: RmDimension`, `vfd: VfdDimension` — SDO-03-002, ADR-0036); the wire +projections carry them flat (`rm_state`, `vfd_state`). Reading a nested +dimension off a wire-shaped object yields `None`, and every core reader +substituted an initial state for that `None` — silently resetting a +participant's RM ladder (#2232, with #2264 as the symptom). + +`Record.from_obj()` was supposed to keep wire objects out of the store, but its +guard was `type_.startswith("as_")` and wire vocabulary `type_` values are +**bare** (`"CaseParticipant"`). Fifteen wire classes therefore shadowed a +`CORE_VOCABULARY` entry and were written into core-typed rows unchallenged. + +So the question is not only *how* to normalise, but **where**: a wire-shaped +object can enter the system at an HTTP ingress boundary, and it can reach the +persistence boundary from several call paths. Enforcing in the wrong place +either misses paths or breaks legitimate inbound traffic. + +## Decision Drivers + +- No wire-shaped row may exist in the DataLayer — that is the invariant #2232 + asks for, and rows outlive whichever code wrote them. +- A wire-shaped object arriving over HTTP is **legitimate inbound data**, not + corruption. Treating it as an error is a denial of service against the + protocol. +- A shape mismatch discovered while reading a *stored* row **is** corruption and + must fail loudly (ARCH-15-001, ARCH-15-002) — the silent degrade is the whole + defect. +- The received-side behavior tree must not abort because one embedded + participant is malformed; the HTTP inbox re-queues on exception, so an + escaping raise becomes an undrainable poison message. +- Enforcement must be verifiable by a test, not by reviewer vigilance: 15 types + shadow a core type and the count will change. + +## Considered Options + +- Normalise at wire→core ingress only +- Normalise at the persistence boundary only +- Normalise at ingress, and enforce again at the persistence boundary +- Unify the two shapes into one class + +## Decision Outcome + +Chosen option: **"Normalise at ingress, and enforce again at the persistence +boundary"**, because the two placements answer different questions and neither +subsumes the other. Ingress projection is what makes the *behaviour* correct — +inbound data is converted where it arrives, so no core reader ever sees a wire +shape and no reader has to degrade. Persistence-boundary normalisation is what +makes the *invariant* hold — it is the single choke point every write passes +through, so it can guarantee the stored row is canonical no matter which ingress +path missed. + +Concretely: + +- **Ingress (primary).** `_project_to_core_participant()` in + `vultron/core/use_cases/received/case/_helpers.py` projects each embedded + participant of a received case snapshot via `to_core()` before anything reads + it. An unprojectable participant is logged at ERROR and **skipped**, not + raised: losing one malformed participant is strictly better than losing the + case. +- **Persistence (backstop).** `_normalize_to_core()` in + `vultron/adapters/driven/db_record.py` projects the object *and its direct + children* for every `type_` in `_NORMALIZE_WIRE_TO_CORE`. Children matter + because a `VulnerabilityCase` row stores `case_participants` inline; one level + suffices because `to_core()` recurses. +- **Readers stay strict.** `participant_status_rm_state()` and + `participant_status_vfd_state()` raise `VultronValidationError` on a non-core + shape. Absence (an empty status list) remains a legitimate `None`. + +Rejected for now: unifying the two shapes. It is the right end state — one class +per concept, wire as a pure projection (ADR-0017) — but it is a breaking change +across the AS2 vocabulary and the persisted-row format, and #2232 is a live data +corruption bug. This ADR deliberately buys correctness now without foreclosing +unification later; both enforcement points become redundant, and removable, once +the shapes converge. + +### Consequences + +- Good, because the DataLayer invariant ("no wire-shaped row") holds regardless + of which write path is used, including paths not yet written. +- Good, because inbound wire data still works: projection at ingress means + making readers strict does not break the protocol. +- Good, because a projection failure now raises `VultronValidationError` rather + than a bare `ValueError`, so it cannot be absorbed by handlers written for + `crud.create()`'s duplicate-row `ValueError`. +- Bad, because the same projection is expressed in two places, and a reader can + reasonably wonder which one is authoritative. Mitigated by + `notes/datalayer-design.md`, which names the persistence boundary as the + backstop. +- Bad, because `_NORMALIZE_WIRE_TO_CORE` covers 2 of the 15 shadowing types. The + other 13 differ only by key spelling today, so they are misspelled rather than + unreadable — tracked in #2268. +- Neutral, because per-write child projection costs one `model_fields` scan on + objects that are already being serialised. + +## Validation + +- `test/architecture/test_normalize_wire_to_core_ratchet.py` — grow-only ratchet + on `_NORMALIZE_WIRE_TO_CORE`, plus an exact enumeration of the un-normalised + shadowing types so a newly added one must be triaged. +- `test/adapters/driven/test_db_record.py` — top-level and nested-child + normalisation; projection failure raises a non-`ValueError`. +- `test/core/use_cases/received/case/test_helpers.py` — a wire-shaped incoming + participant against a core-shaped stored one: no raise escapes, the RM + regression guard still fires, and the persisted row is core-shaped. +- `test/core/models/test_participant_status_shape.py` — both canonical readers + raise on a wire shape and on a present-but-unusable dimension. + +## Pros and Cons of the Options + +### Normalise at wire→core ingress only + +- Good, because it converts data where it arrives, which is where the type + information about "this came from the wire" actually exists. +- Good, because it keeps the adapter free of shape-specific knowledge. +- Bad, because it is an open set of call sites. #2232's first fix took this + reading of the issue, missed the received-case path, and turned every inbound + `Announce(VulnerabilityCase)` into an aborted behavior tree. +- Bad, because it cannot state an invariant about stored rows. + +### Normalise at the persistence boundary only + +- Good, because it is one choke point and yields a checkable invariant. +- Bad, because core readers still meet wire-shaped objects *before* the write — + which is exactly where the RM ladder was being reset. +- Bad, because by the time an error surfaces the useful context (which activity, + which sender) is gone. + +### Normalise at ingress, and enforce again at the persistence boundary + +- Good, because behaviour and invariant are both covered, and each placement + fails safe for the failure mode it owns. +- Neutral, because the redundancy is real but cheap and testable. +- Bad, because two enforcement points must be kept in agreement. + +### Unify the two shapes into one class + +- Good, because it removes the defect class rather than guarding against it. +- Bad, because it breaks the AS2 wire contract and the persisted-row format at + once, with no incremental path — not an acceptable shape for a bug fix. + +## More Information + +Related: issue #2232 (the shape duality), issue #2264 (initial-state +substitution sites), issue #2268 (migrating the remaining 13 shadowing types). +Related ADRs: ADR-0017 (wire is a projection of core), ADR-0034 (`dl.read()` +returns core objects — this is its write-side counterpart), ADR-0036 +(dimension objects on `ParticipantStatus`). + +Generated spec requirements: none new — this decision implements existing +`specs/architecture.yaml` ARCH-15-001, ARCH-15-002 and is the write-side +counterpart to `specs/datalayer.yaml` DL-05-001 through DL-05-004. diff --git a/docs/adr/index.md b/docs/adr/index.md index 944f0e169..7e9627ebb 100644 --- a/docs/adr/index.md +++ b/docs/adr/index.md @@ -129,6 +129,7 @@ General information about architectural decision records is available at &1 | tail -5`. That +command cannot show this happening, for two compounding reasons: + +1. The pipeline's exit status is `tail`'s, not pytest's, so a killed run still + reports `0`. +2. The faulthandler stack dump is long and lands at the end of the combined + stream, so `tail -5` shows dump frames where the + `N passed, M skipped in Xs` summary line would normally be. + +The result is a validation cycle that produced no summary line and exited `0`. +Both of my first two unit-suite runs on this branch were killed this way; I +only noticed because the last visible progress marker read `[ 10%]`. Redirecting +to a file and checking pytest's own exit code showed `UNIT_EXIT=1` and the kill +inside `test/metadata/specs/test_real_specs.py::test_real_specs_lint_no_hard_errors`. + +The test itself is fine in isolation — 3.05s against the 5s budget, 61% of it — +which is exactly the problem: it is close enough to the ceiling that ambient +load decides the outcome, and the whole suite dies with it. A third run passed +cleanly (6592 passed in 115s), confirming a load-sensitive flake rather than a +branch-owned failure. + +**Suggestions:** + +- Have the run-tests skill capture full output and assert on pytest's own exit + code, e.g. `uv run pytest --tb=short > /tmp/unit.log 2>&1; echo $?` followed + by a grep for the summary line — so an absent summary is loud rather than + silent. `tail -5` on a pipe is not a safe success signal. +- Either raise the global `timeout` or mark the spec-lint tests with a larger + per-test budget. A single slow test taking the entire suite down is a + disproportionate failure mode, and 61%-of-budget is not headroom. diff --git a/plan/incoming/learnings/20260812-normalize-at-persistence-not-wire-ingress.md b/plan/incoming/learnings/20260812-normalize-at-persistence-not-wire-ingress.md new file mode 100644 index 000000000..4e07ced83 --- /dev/null +++ b/plan/incoming/learnings/20260812-normalize-at-persistence-not-wire-ingress.md @@ -0,0 +1,41 @@ +--- +title: Design choice — normalise wire→core at the persistence boundary, not at wire ingress +type: learning +timestamp: 2026-08-12 +source: ISSUE-2232 +signal: design-question +--- + +Issue #2232 prescribed: *"Normalize at the boundary (wire -> core on ingress) so no +wire-shaped row is ever persisted."* The fix normalises at the **persistence** +boundary (`Record.from_obj`) rather than at **wire ingress**, and scopes it to +2 of the 15 shadowing types. Both departures were deliberate. + +**Why persistence rather than ingress.** Wire ingress is not a single chokepoint +— wire objects are constructed in-process by BT nodes, trigger factories, and +test fixtures, not only parsed from inbound AS2. Measured during analysis: +~63 files import `as_CaseParticipant`/`as_ParticipantStatus` *and* call +`.save(`/`.create(`. `Record.from_obj` is the one place every persisted object +passes through, so a guard there is total; a guard at ingress would have been +partial while looking complete. The issue's "Done when" clause — *"a +wire-shaped ParticipantStatus cannot be persisted"* — is a statement about +persistence, and that is where it is now enforced. + +**Why normalise rather than reject.** The first attempt rejected all 15 +shadowing types outright. That is the stronger invariant, but it turned ~63 +test files into `size:L` churn unrelated to the defect. Normalising via the +existing `to_core()` projections satisfies both "Done when" clauses (no +wire-shaped row is stored; a shape mismatch raises) without that churn. + +**Why 2 types and not 15.** `CaseParticipant` and `ParticipantStatus` differ +*structurally* between the two shapes — core nests `rm: RmDimension`, wire +carries a flat `rm_state` — so a wire row silently yields `None`. The other 13 +currently differ only by key spelling, which is a coincidence rather than an +invariant. `_NORMALIZE_WIRE_TO_CORE` is documented as shrink-only (may grow, +never shrink) and the remaining 13 are tracked in #2268. + +**Cost paid for the narrow scope:** the write path now has a *second* +exemption set to keep honest, alongside read-side `KNOWN_WIRE_ESCAPES`. Two +ratchets for one underlying problem is a smell; #2268 records the structural +alternative (disjoint `type_` namespaces between the two vocabularies), which +would delete both. diff --git a/plan/incoming/learnings/20260812-test-asserted-the-degraded-fallback.md b/plan/incoming/learnings/20260812-test-asserted-the-degraded-fallback.md new file mode 100644 index 000000000..c7fb41ab4 --- /dev/null +++ b/plan/incoming/learnings/20260812-test-asserted-the-degraded-fallback.md @@ -0,0 +1,41 @@ +--- +title: Process — a test asserted the ARCH-15 violation as intended behaviour +type: learning +timestamp: 2026-08-12 +source: ISSUE-2232 +signal: process-issue +--- + +`test_resolve_participant_state_defaults_when_invalid_rm_type` asserted +`rm == RM.START` for a status whose `rm.state` was the string `"not-an-rm"`, +under the docstring *"Falls back to RM.START when rm_state is not an RM enum +value."* That is precisely the defect #2264 describes: substituting `RM.START` +for an unreadable status silently resets the participant's RM ladder to its +initial state, after which the next legitimate transition is rejected as +backwards. + +The test did not merely fail to catch the bug — it **locked it in**. Fixing +the code turned a green test red, so the regression suite argued for the +defect. ARCH-15-001..004 ("Silent `None` Returns and Fake `SUCCESS` Are the +Same Bug") already forbade the behaviour when the test was written. + +Two things made it look reasonable: + +1. The word *"defaults"* in the test name conflates **absence** with + **unreadability**. `RM.START` is the right answer for a participant with no + recorded status; it is never the right answer for a status that exists but + cannot be read. The sibling + `test_resolve_participant_state_defaults_when_no_statuses` covers the + legitimate case and still passes unchanged. +2. Asserting the observed behaviour of a defensive `isinstance` fallback feels + like coverage. It is really a snapshot of an unreviewed default. + +**Signal to watch for in review:** a test whose docstring says "falls back +to", "defaults to", or "tolerates" for *malformed* input, rather than for +*absent* input. Ask which of the two the fallback is actually serving; if it +is malformed input, ARCH-15 requires a raise or `Status.FAILURE` and the test +is asserting a bug. + +Note also that `test_resolve_participant_state_defaults_when_invalid_vfd_type` +was left passing on purpose: the vfd dimension remains lenient because RM is +read first, so a wire-shaped status raises before vfd is reached. diff --git a/test/adapters/driven/datalayer_sqlite/test_participant_status_summary.py b/test/adapters/driven/datalayer_sqlite/test_participant_status_summary.py new file mode 100644 index 000000000..c6ad09710 --- /dev/null +++ b/test/adapters/driven/datalayer_sqlite/test_participant_status_summary.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python + +# Copyright (c) 2026 Carnegie Mellon University and Contributors. +# - see Contributors.md for a full list of Contributors +# - see ContributionInstructions.md for information on how you can Contribute to this project +# Vultron Multiparty Coordinated Vulnerability Disclosure Protocol Prototype is +# licensed under a MIT (SEI)-style license, please see LICENSE.md distributed +# with this Software or contact permission@sei.cmu.edu for full terms. +# Created, in part, with funding and support from the United States Government +# (see Acknowledgments file). This program may include and/or can make use of +# certain third party source code, object code, documentation and other files +# ("Third Party Software"). See LICENSE.md for more details. +# Carnegie Mellon®, CERT® and CERT Coordination Center® are registered in the +# U.S. Patent and Trademark Office by Carnegie Mellon University + +"""Tests for the participant-status save/read log summary (issue #2232). + +``participant_status_summary`` is the adapter's read/save observability hook — +the thing that makes a read-after-write shape problem diagnosable from container +logs without dumping full JSON. It read only the flat wire spellings +(``rm_state``/``rmState``), so once rows became canonically core-shaped every +line reported ``vfd=None,rm=None``: the diagnostic went blank at exactly the +moment a shape migration made it most useful. +""" + +from vultron.adapters.driven.datalayer_sqlite.schema import ( + _dimension_state, + participant_status_summary, +) + + +class TestDimensionState: + """``_dimension_state`` reads either persisted shape.""" + + def test_reads_the_canonical_nested_shape(self): + """Core shape (ADR-0036): ``{"rm": {"state": ...}}``.""" + status = {"rm": {"state": "RECEIVED"}} + assert _dimension_state(status, "rm") == "RECEIVED" + + def test_reads_the_flat_wire_shape(self): + """Legacy/wire rows must remain readable while they still exist.""" + assert _dimension_state({"rm_state": "VALID"}, "rm") == "VALID" + + def test_reads_the_camel_cased_wire_shape(self): + assert _dimension_state({"vfdState": "Vfd"}, "vfd") == "Vfd" + + def test_prefers_the_nested_shape_when_both_are_present(self): + """A mixed row is exactly the bug; report the canonical side.""" + status = {"rm": {"state": "ACCEPTED"}, "rm_state": "START"} + assert _dimension_state(status, "rm") == "ACCEPTED" + + def test_falls_back_when_the_nested_dimension_has_no_state(self): + status = {"rm": {}, "rm_state": "START"} + assert _dimension_state(status, "rm") == "START" + + def test_returns_none_when_the_dimension_is_absent(self): + assert _dimension_state({}, "rm") is None + + def test_non_dict_nested_value_does_not_raise(self): + """A malformed row must degrade to the flat lookup, not explode. + + This helper runs inside a logging call; raising here would turn a + diagnostic into an outage. + """ + assert _dimension_state({"rm": "RECEIVED"}, "rm") is None + + +class TestParticipantStatusSummary: + """The summary line must report real states for core-shaped rows.""" + + def test_reports_states_for_a_core_shaped_row(self): + data = { + "participant_statuses": [ + { + "rm": {"state": "RECEIVED"}, + "vfd": {"state": "vfd"}, + "published": "2026-01-01T00:00:00Z", + "updated": None, + } + ] + } + summary = participant_status_summary(data) + assert "n_statuses=1" in summary + assert "rm='RECEIVED'" in summary + assert "vfd='vfd'" in summary + + def test_reports_states_for_a_flat_wire_row(self): + data = { + "participant_statuses": [{"rm_state": "VALID", "vfd_state": "Vfd"}] + } + summary = participant_status_summary(data) + assert "rm='VALID'" in summary + assert "vfd='Vfd'" in summary + + def test_reports_every_entry_in_the_ladder(self): + data = { + "participant_statuses": [ + {"rm": {"state": "START"}}, + {"rm": {"state": "RECEIVED"}}, + ] + } + summary = participant_status_summary(data) + assert "n_statuses=2" in summary + assert "[0]" in summary and "[1]" in summary + assert "rm='START'" in summary + assert "rm='RECEIVED'" in summary + + def test_empty_ladder_is_reported_as_zero(self): + assert participant_status_summary({"participant_statuses": []}) == ( + "n_statuses=0" + ) + + def test_camel_cased_status_list_key_is_accepted(self): + data = {"participantStatuses": [{"rm": {"state": "START"}}]} + assert "rm='START'" in participant_status_summary(data) + + def test_non_participant_row_returns_empty_string(self): + """Callers branch on ``""`` to skip the log line cheaply.""" + assert participant_status_summary({"id_": "urn:uuid:x"}) == "" + assert participant_status_summary(None) == "" + assert participant_status_summary("not-a-row") == "" + + def test_non_dict_status_entry_is_reported_by_type(self): + summary = participant_status_summary({"participant_statuses": ["x"]}) + assert "[0]" in summary diff --git a/test/adapters/driven/test_db_record.py b/test/adapters/driven/test_db_record.py index d0dfab752..9871a1ae5 100644 --- a/test/adapters/driven/test_db_record.py +++ b/test/adapters/driven/test_db_record.py @@ -14,6 +14,7 @@ from typing import Any, cast import pytest +from pydantic import BaseModel from vultron.adapters.driven.db_record import ( Record, @@ -21,6 +22,7 @@ object_to_record, record_to_object, ) +from vultron.errors import VultronValidationError from vultron.wire.as2.factories import rm_submit_report_activity @@ -237,3 +239,175 @@ def test_object_to_record_nested_report_not_duplicated_in_offer_data(): serialised = json.dumps(record.data_) assert report.content is not None assert report.content not in serialised + + +# --------------------------------------------------------------------------- +# Wire/core shape guard on the write path (issue #2232) +# --------------------------------------------------------------------------- + + +def test_object_to_record_normalizes_wire_class_shadowing_a_core_type(): + """A wire vocab class whose ``type_`` has a core counterpart is normalised. + + Regression for #2232: the only shape guard was ``type_.startswith("as_")``, + but wire vocabulary ``type_`` values are bare ("CaseParticipant"), so a + wire-shaped object was happily written into a core-typed DataLayer row. + Core readers then saw a flat ``rm_state`` where they expected a nested + ``rm`` dimension. + + The row must now carry the canonical core shape — nested + ``rm: {"state": ...}`` — so no wire-shaped ``CaseParticipant`` row exists to + be misread. + """ + from vultron.core.models.registry import CORE_VOCABULARY + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + + wire_participant = as_CaseParticipant( + attributed_to="https://example.org/actors/vendor", + context="https://example.org/cases/case-2232", + ) + # The pre-existing guard cannot catch this: type_ is bare, not "as_"-prefixed. + assert not str(wire_participant.type_).startswith("as_") + assert str(wire_participant.type_) in CORE_VOCABULARY + + record = object_to_record(cast(Any, wire_participant)) + + assert record.type_ == "CaseParticipant" + statuses = record.data_["participant_statuses"] + assert statuses, "normalised participant must retain its RM ladder" + for status in statuses: + # Canonical core shape: nested rm dimension, no flat rm_state. + assert "rm_state" not in status + assert status["rm"]["state"] == "START" + + +def test_object_to_record_normalizes_wire_participant_status(): + """A wire ``ParticipantStatus`` persists in the nested core ``rm`` shape.""" + from vultron.core.states.rm import RM + from vultron.wire.as2.vocab.objects.case_status import ( + as_ParticipantStatus, + ) + + wire_status = as_ParticipantStatus( + rm_state=RM.VALID, + context="https://example.org/cases/case-2232", + attributed_to="https://example.org/actors/vendor", + ) + + record = object_to_record(cast(Any, wire_status)) + + assert record.type_ == "ParticipantStatus" + assert "rm_state" not in record.data_ + assert record.data_["rm"]["state"] == "VALID" + + +def test_object_to_record_normalizes_wire_participant_nested_in_core_case(): + """A wire participant nested inside a core case is normalised too. + + Regression for the first fix of #2232, which inspected only the top-level + object. A ``VulnerabilityCase`` row stores its ``case_participants`` + inline, so a wire-shaped participant nested in a core-shaped case still + persisted a flat ``rm_state`` — the row shape the issue's "Done when" + forbids. + """ + from vultron.core.models.case import VulnerabilityCase + from vultron.core.states.rm import RM + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + from vultron.wire.as2.vocab.objects.case_status import ( + as_ParticipantStatus, + ) + + case_id = "urn:uuid:3f1b8d0e-1111-4111-8111-000000002232" + wire_participant = as_CaseParticipant( + attributed_to="https://example.org/actors/vendor", + context=case_id, + participant_statuses=[ + as_ParticipantStatus(context=case_id, rm_state=RM.RECEIVED) + ], + ) + case = VulnerabilityCase(id_=case_id, name="case-2232").model_copy( + update={"case_participants": [wire_participant]} + ) + + record = object_to_record(cast(Any, case)) + + stored_status = record.data_["case_participants"][0][ + "participant_statuses" + ][0] + assert "rm_state" not in stored_status + assert stored_status["rm"]["state"] == "RECEIVED" + + +def test_object_to_record_raises_when_wire_class_has_no_to_core(): + """A shadowing wire class without ``to_core()`` cannot be persisted. + + Covers the ``to_core is None`` branch: the object shadows a core type, so + storing it as-is would produce a row nothing can read back reliably, and + there is no projection available to fix it. + """ + from vultron.core.models.protocols import PersistableModel + + class _ShadowingWireClass(BaseModel): + """Stands in for a wire class that never grew a ``to_core()``.""" + + id_: str = "urn:uuid:00000000-0000-4000-8000-000000002232" + type_: str = "ParticipantStatus" + + # Impersonate the wire package so the module-prefix check matches. + _ShadowingWireClass.__module__ = "vultron.wire.as2.vocab.objects.fake" + + with pytest.raises(VultronValidationError, match="no to_core"): + object_to_record(cast(PersistableModel, _ShadowingWireClass())) + + +def test_normalization_failure_is_distinguishable_from_duplicate_row(): + """A projection failure must not look like an "already exists" ValueError. + + ``crud.create`` raises ``ValueError`` for a genuine duplicate and callers + legitimately swallow that. When normalisation failure raised ``ValueError`` + too, an unprojectable object was silently never stored and never logged + (the ingress pre-store in ``routers/actors/_inbox.py`` did exactly this). + A distinct, non-``ValueError`` type keeps the two causes separable. + """ + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + + # NonEmptyString rejects "" on the core class but not on the wire class, + # so this object is constructible yet unprojectable. + unprojectable = as_CaseParticipant( + attributed_to="https://example.org/actors/vendor", + context="https://example.org/cases/case-2232", + accepted_embargo_ids=[""], + ) + + with pytest.raises(VultronValidationError) as exc_info: + object_to_record(cast(Any, unprojectable)) + + assert not isinstance(exc_info.value, ValueError) + assert "2232" in str(exc_info.value) + + +def test_object_to_record_still_accepts_wire_activities(): + """Activities have no core counterpart, so they must remain persistable.""" + from vultron.wire.as2.vocab.objects.vulnerability_report import ( + as_VulnerabilityReport, + ) + + report = as_VulnerabilityReport( + name="CVE-2232", + content="details", + attributed_to="https://example.org/finder", + ) + offer = rm_submit_report_activity( + report, + "https://example.org/finder", + actor="https://example.org/finder", + ) + + record = object_to_record(offer) + assert record.type_ == "Offer" diff --git a/test/adapters/driven/test_sqlite_core_roundtrip.py b/test/adapters/driven/test_sqlite_core_roundtrip.py index 85c367f06..e6835c010 100644 --- a/test/adapters/driven/test_sqlite_core_roundtrip.py +++ b/test/adapters/driven/test_sqlite_core_roundtrip.py @@ -26,16 +26,30 @@ import pytest from datetime import datetime, timedelta, timezone +from sqlmodel import Session + from vultron.adapters.driven.datalayer_sqlite import SqliteDataLayer +from vultron.adapters.driven.datalayer_sqlite.schema import VultronObjectRecord from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_participant import CaseParticipant from vultron.core.models.case_status import CaseStatus from vultron.core.models.embargo_event import EmbargoEvent from vultron.core.models.embargo_policy import EmbargoPolicy -from vultron.core.models.participant_status import ParticipantStatus +from vultron.core.models.participant_status import ( + ParticipantStatus, + participant_status_rm_state, +) from vultron.core.models.report import VulnerabilityReport from vultron.core.models.vulnerability_record import VulnerabilityRecord +from vultron.core.states import CS_vfd, RM +from vultron.enums.roles import CVDRole +from vultron.errors import VultronValidationError from vultron.wire.as2.vocab.base.objects.object_types import as_Note +from vultron.wire.as2.vocab.objects.case_participant import as_CaseParticipant +from vultron.wire.as2.vocab.objects.case_status import as_ParticipantStatus +from vultron.wire.as2.vocab.objects.vulnerability_case import ( + as_VulnerabilityCase, +) _CASE_CONTEXT = "urn:uuid:case-context-fixture" _NOW = datetime(2026, 1, 1, tzinfo=timezone.utc) @@ -184,3 +198,108 @@ def test_core_entity_type_string_matches_class_name(dl): result = dl.read(case.id_) assert result is not None assert result.type_ == VulnerabilityCase.__name__ + + +# --------------------------------------------------------------------------- +# DL-05-002: a row that fails core validation still reads back as core (#2232) +# --------------------------------------------------------------------------- + + +def _insert_raw_row(dl, id_, type_, data): + """Insert *data* verbatim, bypassing object_to_record normalisation. + + Mirrors the ``crud.create`` + ``StorableRecord`` write path, which stores + ``record.data_`` as given, skipping ``Record.from_obj``'s wire→core + normalisation (issue #2283). + """ + with Session(dl._engine) as session: + session.add( + VultronObjectRecord(id_=id_, type_=type_, actor_id=None, data=data) + ) + session.commit() + + +def _mixed_spelling_case_row(case_id): + """A stored case row whose nested participant uses wire (camelCase) keys. + + Snake_case at the case level — so ``case_participants`` is populated — but + each entry is dumped ``by_alias``, which is what makes core validation of + the *participant* fail while the case itself looks well formed. + """ + status = as_ParticipantStatus( + context=case_id, rm_state=RM.ACCEPTED, vfd_state=CS_vfd.vfd + ) + participant = as_CaseParticipant( + id_="urn:uuid:participant-2232", + attributed_to="https://example.org/actors/finder", + context=case_id, + case_roles=[CVDRole.FINDER], + participant_statuses=[status], + ) + case = as_VulnerabilityCase( + id_=case_id, name="mixed", case_participants=[participant] + ) + data = case.model_dump(mode="json") + data["case_participants"] = [ + participant.model_dump(mode="json", by_alias=True, exclude_none=True) + ] + return data + + +def test_mixed_spelling_row_fails_core_validation(): + """Guard the premise: the fixture row really does fail core validation. + + Without this, the read-path tests below could pass for the wrong reason — + a row that validates cleanly never exercises the fallback at all. + """ + case_id = "urn:uuid:case-2232-premise" + with pytest.raises(VultronValidationError): + VulnerabilityCase.model_validate(_mixed_spelling_case_row(case_id)) + + +def test_read_projects_wire_fallback_back_to_core(dl): + """A row failing core validation reads back as the core type, not the wire one. + + Before #2232 the shape guard on ``CaseParticipant`` turned this row into an + ``as_VulnerabilityCase`` from ``dl.read()``, and ``resolve_case`` then raised + ``Expected VulnerabilityCase, got as_VulnerabilityCase`` — a 422 on every + subsequent case operation (fcv-reject demo, Phase 3 add-note-to-case). + The read path now projects the wire fallback through ``to_core()``. + """ + case_id = "urn:uuid:case-2232-read" + _insert_raw_row( + dl, case_id, "VulnerabilityCase", _mixed_spelling_case_row(case_id) + ) + + result = dl.read(case_id) + + assert result is not None + assert not isinstance(result, as_VulnerabilityCase) + assert isinstance(result, VulnerabilityCase), ( + f"Expected VulnerabilityCase, got {type(result).__name__} — the wire " + "fallback was returned un-projected (DL-05-002, issue #2232)." + ) + + +def test_read_projection_preserves_participant_rm_state(dl): + """The projected core object keeps the participant's RM ladder position. + + A projection that reset ``rm`` to ``RM.START`` would satisfy the type + assertion above while silently rewinding protocol state — that is #2264. + """ + case_id = "urn:uuid:case-2232-ladder" + _insert_raw_row( + dl, case_id, "VulnerabilityCase", _mixed_spelling_case_row(case_id) + ) + + result = dl.read(case_id) + + assert isinstance(result, VulnerabilityCase) + assert len(result.case_participants) == 1 + participant = result.case_participants[0] + assert isinstance(participant, CaseParticipant) + assert participant.participant_statuses + assert ( + participant_status_rm_state(participant.participant_statuses[0]) + is RM.ACCEPTED + ) diff --git a/test/adapters/driving/fastapi/routers/actors/test_inbox.py b/test/adapters/driving/fastapi/routers/actors/test_inbox.py index 0931c397d..5da883a8f 100644 --- a/test/adapters/driving/fastapi/routers/actors/test_inbox.py +++ b/test/adapters/driving/fastapi/routers/actors/test_inbox.py @@ -218,6 +218,57 @@ def test_store_nested_inbox_object_skips_when_no_body(datalayer): _store_nested_inbox_object(datalayer, activity, None) +def test_store_nested_inbox_object_logs_a_projection_failure( + datalayer, caplog +): + """An unpersistable inline object must be logged at ERROR (issue #2232). + + A projection failure and an "already exists" collision both used to surface + as ``ValueError`` and were swallowed together at DEBUG, so the row was + silently absent and downstream BT nodes reported a misleading "participant + not found". The distinct ``VultronValidationError`` is now logged loudly. + """ + import logging + + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + + # NonEmptyString rejects "" on the core class but not the wire class, so + # this participant is constructible yet cannot be projected to core. + unprojectable = as_CaseParticipant( + id_="urn:uuid:participant-2232-unprojectable", + attributed_to=_ACTOR_URI, + context="https://example.org/cases/case-2232", + accepted_embargo_ids=[""], + ) + activity = as_Announce(actor=_ACTOR_URI, object_=unprojectable) + + with caplog.at_level(logging.ERROR): + _store_nested_inbox_object(datalayer, activity, None) + + assert datalayer.read(unprojectable.id_) is None + assert "cannot be projected" in caplog.text + + +def test_store_nested_inbox_object_duplicate_stays_at_debug(datalayer, caplog): + """A genuine duplicate is not an error — it must not be logged as one.""" + import logging + + case = as_VulnerabilityCase( + id_="urn:uuid:case-dup-2232", + name="Duplicate Case", + ) + activity = as_Announce(actor=_ACTOR_URI, object_=case) + _store_nested_inbox_object(datalayer, activity, None) + + with caplog.at_level(logging.DEBUG): + _store_nested_inbox_object(datalayer, activity, None) + + assert "already exists" in caplog.text + assert not [r for r in caplog.records if r.levelno >= logging.ERROR] + + # --------------------------------------------------------------------------- # _record_inbox_receipt # --------------------------------------------------------------------------- diff --git a/test/architecture/test_normalize_wire_to_core_ratchet.py b/test/architecture/test_normalize_wire_to_core_ratchet.py new file mode 100644 index 000000000..6e085ce97 --- /dev/null +++ b/test/architecture/test_normalize_wire_to_core_ratchet.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python + +# Copyright (c) 2026 Carnegie Mellon University and Contributors. +# - see Contributors.md for a full list of Contributors +# - see ContributionInstructions.md for information on how you can Contribute to this project +# Vultron Multiparty Coordinated Vulnerability Disclosure Protocol Prototype is +# licensed under a MIT (SEI)-style license, please see LICENSE.md distributed +# with this Software or contact permission@sei.cmu.edu for full terms. +# Created, in part, with funding and support from the United States Government +# (see Acknowledgments file). This program may include and/or can make use of +# certain third party source code, object code, documentation and other files +# ("Third Party Software"). See LICENSE.md for more details. +# Carnegie Mellon®, CERT® and CERT Coordination Center® are registered in the +# U.S. Patent and Trademark Office by Carnegie Mellon University +"""Architecture ratchet: the write-side wire→core normalization set may only grow. + +``_NORMALIZE_WIRE_TO_CORE`` in ``vultron/adapters/driven/db_record.py`` lists the +``type_`` strings whose wire class is projected to its core counterpart before a +row is written. It is the write-side analogue of ``KNOWN_WIRE_ESCAPES`` in +``test_dl_read_returns_core_objects.py`` (DL-05-004) and needs the same +protection, for the mirror-image reason: + +- ``KNOWN_WIRE_ESCAPES`` is a list of *known bad* read paths, so it may only + **shrink**. +- ``_NORMALIZE_WIRE_TO_CORE`` is a list of *already migrated* write paths, so it + may only **grow**. Silently dropping an entry re-opens the shape duality that + issue #2232 closed, and it would do so without any test noticing: nothing else + asserts that a given type is normalised. + +Fifteen wire classes shadow a ``CORE_VOCABULARY`` entry (their ``type_`` is bare, +so the ``as_``-prefix guard in ``Record.from_obj`` never fires for them). Two are +normalised today; the remaining thirteen are enumerated below so that a *new* +shadowing type has to be triaged rather than joining the backlog unnoticed. + +Related: issue #2232 (the shape duality), issue #2268 (migrating the rest). +""" + +# Importing the SQLite adapter transitively imports the core and wire vocabulary +# modules, which is what populates both registries via ``__init_subclass__``. +# Without it the registries are nearly empty and this test would vacuously pass. +import vultron.adapters.driven.datalayer_sqlite # noqa: F401 +from vultron.adapters.driven.db_record import _NORMALIZE_WIRE_TO_CORE +from vultron.core.models.registry import CORE_VOCABULARY +from vultron.wire.as2.vocab.base.registry import VOCABULARY + +_WIRE_MODULE_PREFIX = "vultron.wire.as2" + +# --------------------------------------------------------------------------- +# Baseline: the types normalised as of issue #2232. This set may only GROW. +# Adding a type here is the second half of migrating it; removing one is a +# regression, not a refactor. +# --------------------------------------------------------------------------- +_NORMALIZED_AS_OF_2232: frozenset[str] = frozenset( + { + "CaseParticipant", + "ParticipantStatus", + } +) + +# --------------------------------------------------------------------------- +# Shadowing types NOT yet normalised (issue #2268). This set may only SHRINK: +# migrating a type moves it out of here and into ``_NORMALIZE_WIRE_TO_CORE``. +# +# The ten object types differ from their core counterpart only by key spelling +# today, so a wire-shaped row is misspelled rather than structurally unreadable. +# The five actor types have no ``to_core()`` projection at all, so they cannot be +# normalised until one exists. +# --------------------------------------------------------------------------- +_NOT_YET_NORMALIZED: frozenset[str] = frozenset( + { + "CaseLedgerEntry", + "CaseReference", + "CaseStatus", + "EmbargoEvent", + "EmbargoPolicy", + "VulnerabilityCase", + "VulnerabilityRecord", + "VulnerabilityReport", + # No to_core() projection exists for these yet. + "VultronApplication", + "VultronGroup", + "VultronOrganization", + "VultronPerson", + "VultronService", + } +) + + +def _shadowing_types() -> dict[str, type]: + """Return wire classes whose bare ``type_`` collides with a core type.""" + return { + type_: cls + for type_, cls in VOCABULARY.items() + if type_ in CORE_VOCABULARY + and cls.__module__.startswith(_WIRE_MODULE_PREFIX) + } + + +def test_registries_are_populated(): + """Guard the guard: an empty registry would make every assertion vacuous.""" + assert len(CORE_VOCABULARY) > 10 + assert len(VOCABULARY) > 50 + + +def test_normalize_set_may_only_grow(): + """Every type normalised as of #2232 must still be normalised.""" + missing = _NORMALIZED_AS_OF_2232 - _NORMALIZE_WIRE_TO_CORE + assert not missing, ( + "_NORMALIZE_WIRE_TO_CORE lost entries" + f" {sorted(missing)} — the write path would again persist a wire-shaped" + " row for those types (issue #2232). The set may only grow." + ) + + +def test_every_normalized_type_actually_shadows_a_core_type(): + """A stale entry is dead weight — it normalises nothing.""" + shadowing = set(_shadowing_types()) + stale = _NORMALIZE_WIRE_TO_CORE - shadowing + assert not stale, ( + f"_NORMALIZE_WIRE_TO_CORE entries {sorted(stale)} do not name a wire" + " class that shadows a CORE_VOCABULARY type; remove them or fix the" + " spelling." + ) + + +def test_every_normalized_type_has_a_to_core_projection(): + """Normalisation is implemented by ``to_core()``; without it the write raises.""" + shadowing = _shadowing_types() + for type_ in sorted(_NORMALIZE_WIRE_TO_CORE): + cls = shadowing[type_] + assert hasattr(cls, "to_core"), ( + f"{cls.__name__} is listed in _NORMALIZE_WIRE_TO_CORE but exposes no" + " to_core(), so every attempt to persist one raises instead of" + " normalising (issue #2232)." + ) + + +def test_unmigrated_shadowing_types_are_enumerated(): + """A newly added shadowing type must be triaged, not silently deferred. + + Fails in both directions on purpose: + + - a **new** shadowing wire class appears → decide whether it needs + normalising (issue #2232) and record the answer here; + - a type is **migrated** → move it into ``_NORMALIZE_WIRE_TO_CORE`` and drop + it from ``_NOT_YET_NORMALIZED`` so the backlog stays honest (issue #2268). + """ + unmigrated = set(_shadowing_types()) - set(_NORMALIZE_WIRE_TO_CORE) + assert unmigrated == set(_NOT_YET_NORMALIZED), ( + "the set of un-normalised shadowing types changed.\n" + f" newly un-normalised: {sorted(unmigrated - _NOT_YET_NORMALIZED)}\n" + f" no longer listed: {sorted(_NOT_YET_NORMALIZED - unmigrated)}" + ) diff --git a/test/core/behaviors/case/nodes/participant/test_helpers.py b/test/core/behaviors/case/nodes/participant/test_helpers.py index e4b842686..1d26487b7 100644 --- a/test/core/behaviors/case/nodes/participant/test_helpers.py +++ b/test/core/behaviors/case/nodes/participant/test_helpers.py @@ -153,3 +153,66 @@ def test_returns_none_when_case_not_found( logging.getLogger("test"), ) assert result is None + + +class TestResolveParticipantStateShapeGuard: + """``resolve_participant_state_from_dl`` must not substitute ``RM.START``. + + Regression for #2264 (a symptom of #2232): a wire-shaped participant has + no ``rm`` attribute, so ``latest.rm.state if hasattr(latest, "rm") else + RM.START`` silently reset the participant's RM ladder to its initial state + instead of failing. ARCH-15-001..004 requires the mismatch to raise. + """ + + _CONTEXT = "https://example.org/cases/case-2264" + + class _FakeDl: + def __init__(self, obj: Any) -> None: + self._obj = obj + + def read(self, _id: str) -> Any: + return self._obj + + def test_returns_state_for_core_shaped_participant(self) -> None: + import pytest # noqa: F401 (kept local; module has no pytest import) + + from vultron.core.behaviors.case.nodes.participant.common import ( + resolve_participant_state_from_dl, + ) + from vultron.core.models.case_participant import CaseParticipant + from vultron.core.states.rm import RM + + actor = "https://example.org/actors/alice" + participant = CaseParticipant( + attributed_to=actor, context=self._CONTEXT + ) + participant.append_rm_state( + RM.RECEIVED, actor=actor, context=self._CONTEXT + ) + + rm_state, _vfd = resolve_participant_state_from_dl( + cast(Any, self._FakeDl(participant)), participant.id_ + ) + assert rm_state is RM.RECEIVED + + def test_raises_on_wire_shaped_participant(self) -> None: + import pytest + + from vultron.core.behaviors.case.nodes.participant.common import ( + resolve_participant_state_from_dl, + ) + from vultron.errors import VultronValidationError + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + + wire_participant = as_CaseParticipant( + attributed_to="https://example.org/actors/vendor", + context=self._CONTEXT, + ) + + with pytest.raises(VultronValidationError): + resolve_participant_state_from_dl( + cast(Any, self._FakeDl(wire_participant)), + wire_participant.id_, + ) diff --git a/test/core/behaviors/status/test_add_participant_status_bt.py b/test/core/behaviors/status/test_add_participant_status_bt.py index 0b8f68d14..c5a2a8078 100644 --- a/test/core/behaviors/status/test_add_participant_status_bt.py +++ b/test/core/behaviors/status/test_add_participant_status_bt.py @@ -691,7 +691,9 @@ def test_skips_when_sender_is_not_case_owner( from vultron.core.states.cs import CS_pxa from vultron.wire.as2.vocab.objects.case_status import as_CaseStatus - cs = as_CaseStatus() + # ``context`` is required by core CaseStatus; omitting it made the + # nested status unprojectable to the core shape (#2232). + cs = as_CaseStatus(context=CASE_ID) cs.pxa_state = CS_pxa.Pxa # public-aware status_obj.case_status = cs populated_dl.save(status_obj) @@ -731,7 +733,9 @@ def test_triggers_teardown_on_public_aware_case_owner( populated_dl.create(embargo) populated_dl.save(case) - cs = as_CaseStatus() + # ``context`` is required by core CaseStatus; omitting it made the + # nested status unprojectable to the core shape (#2232). + cs = as_CaseStatus(context=CASE_ID) cs.pxa_state = CS_pxa.Pxa # public-aware status_obj.case_status = cs populated_dl.save(status_obj) diff --git a/test/core/models/test_participant_status_shape.py b/test/core/models/test_participant_status_shape.py new file mode 100644 index 000000000..4c0149f13 --- /dev/null +++ b/test/core/models/test_participant_status_shape.py @@ -0,0 +1,209 @@ +#!/usr/bin/env python + +# Copyright (c) 2026 Carnegie Mellon University and Contributors. +# - see Contributors.md for a full list of Contributors +# - see ContributionInstructions.md for information on how you can Contribute to this project +# Vultron Multiparty Coordinated Vulnerability Disclosure Protocol Prototype is +# licensed under a MIT (SEI)-style license, please see LICENSE.md distributed +# with this Software or contact permission@sei.cmu.edu for full terms. +# Created, in part, with funding and support from the United States Government +# (see Acknowledgments file). This program may include and/or can make use of +# certain third party source code, object code, documentation and other files +# ("Third Party Software"). See LICENSE.md for more details. +# Carnegie Mellon®, CERT® and CERT Coordination Center® are registered in the +# U.S. Patent and Trademark Office by Carnegie Mellon University + +"""Regression tests for the canonical ParticipantStatus shape (issue #2232). + +``ParticipantStatus`` exists in two incompatible shapes: + +- **core** (``vultron/core/models/participant_status.py``) — nested + ``rm: RmDimension``, read as ``status.rm.state``. +- **wire** (``vultron/wire/as2/vocab/objects/case_status.py``) — flat + ``rm_state: RM``, and no ``rm`` attribute at all. + +Two silent-failure modes followed from that, both reproduced here: + +1. Core ``CaseParticipant`` has no ``alias_generator``, so a wire-spelled + (camelCase) ``participantStatuses`` key was an unknown key, silently + dropped, and ``_init_participant_status_if_empty`` re-seeded a single + status at ``RM.START`` — losing the whole RM ladder. +2. Reading ``rm`` off a wire-shaped status yielded ``None``, so every core + reader degraded instead of failing (ARCH-15-001, ARCH-15-002). + +The fix keeps core snake_case-canonical (ARCH-12-003 forbids +``alias_generator=to_camel`` in core-branch types) and makes both failure +modes raise. Related: #2264 (RM.START substitution sites). +""" + +import pytest + +from vultron.core.models.case_participant import CaseParticipant +from vultron.core.models.dimensions import RmDimension, VfdDimension +from vultron.core.models.participant_status import ( + ParticipantStatus, + participant_status_rm_state, + participant_status_vfd_state, +) +from vultron.core.states.cs import CS_vfd +from vultron.core.states.rm import RM +from vultron.errors import VultronValidationError + +_ACTOR = "https://example.org/actors/alice" +_CONTEXT = "https://example.org/cases/case-2232" + + +def _core_participant_with_ladder() -> CaseParticipant: + """Return a core participant whose RM ladder is START → RECEIVED.""" + participant = CaseParticipant(attributed_to=_ACTOR, context=_CONTEXT) + participant.append_rm_state(RM.RECEIVED, actor=_ACTOR, context=_CONTEXT) + assert [s.rm.state.name for s in participant.participant_statuses] == [ + "START", + "RECEIVED", + ] + return participant + + +# --------------------------------------------------------------------------- +# Failure mode 1 — wire-spelled keys must not be silently dropped +# --------------------------------------------------------------------------- + + +class TestCaseParticipantRejectsWireSpelledKeys: + """Core ``CaseParticipant`` must raise, not silently drop, camelCase keys.""" + + def test_camel_case_participant_statuses_raises(self): + """``participantStatuses`` must raise instead of resetting the ladder. + + Before the fix this validated cleanly and returned a participant with + a single re-seeded ``RM.START`` status — a two-entry ladder silently + became one entry. + """ + data = _core_participant_with_ladder().model_dump(mode="json") + data["participantStatuses"] = data.pop("participant_statuses") + + with pytest.raises( + VultronValidationError, match="participantStatuses" + ): + CaseParticipant.model_validate(data) + + def test_camel_case_case_roles_raises(self): + """The same silent drop applied to every snake-only core field.""" + data = _core_participant_with_ladder().model_dump(mode="json") + data["caseRoles"] = data.pop("case_roles") + + with pytest.raises(VultronValidationError, match="caseRoles"): + CaseParticipant.model_validate(data) + + def test_snake_case_round_trip_is_unaffected(self): + """The canonical core shape must still round-trip losslessly.""" + original = _core_participant_with_ladder() + restored = CaseParticipant.model_validate( + original.model_dump(mode="json") + ) + assert [s.rm.state.name for s in restored.participant_statuses] == [ + "START", + "RECEIVED", + ] + + def test_sanctioned_camel_case_aliases_still_accepted(self): + """Fields with an explicit camelCase ``validation_alias`` stay valid. + + ``in_reply_to``/``inReplyTo`` and ``id``/``type`` are declared aliases, + not accidental wire spellings, so the guard must not reject them. + """ + participant = CaseParticipant.model_validate( + { + "id": "urn:uuid:2232-alias-check", + "type": "CaseParticipant", + "attributed_to": _ACTOR, + "context": _CONTEXT, + "inReplyTo": "urn:uuid:2232-parent", + } + ) + assert participant.in_reply_to == "urn:uuid:2232-parent" + + +# --------------------------------------------------------------------------- +# Failure mode 2 — a shape mismatch must raise, not degrade to None +# --------------------------------------------------------------------------- + + +class TestParticipantStatusRmStateHelper: + """``participant_status_rm_state`` is the canonical RM-dimension reader.""" + + def test_returns_state_for_core_shaped_status(self): + status = ParticipantStatus( + context=_CONTEXT, rm=RmDimension(state=RM.RECEIVED) + ) + assert participant_status_rm_state(status) is RM.RECEIVED + + def test_raises_on_wire_shaped_status(self): + """A flat ``rm_state`` status has no ``rm`` — that must raise.""" + from vultron.wire.as2.vocab.objects.case_status import ( + as_ParticipantStatus, + ) + + wire_status = as_ParticipantStatus( + context=_CONTEXT, rm_state=RM.RECEIVED + ) + assert getattr(wire_status, "rm", None) is None + + with pytest.raises(VultronValidationError, match="rm"): + participant_status_rm_state(wire_status) + + def test_raises_when_rm_carries_no_rm_state(self): + """A present-but-unusable ``rm`` must raise rather than return None. + + ``match=`` pins the *second* guard: without it this test also passes if + the ``rm is None`` branch fires, so it would not distinguish the two. + """ + + class _Bogus: + rm = object() + + with pytest.raises(VultronValidationError, match="no valid RM state"): + participant_status_rm_state(_Bogus()) + + +class TestParticipantStatusVfdStateHelper: + """``participant_status_vfd_state`` is the canonical VFD-dimension reader. + + The VFD dimension had the identical degrade (``getattr(status, "vfd", None)`` + → substitute ``CS_vfd.vfd``) sitting a few lines from the RM one, so fixing + only RM would have left the same defect alive one dimension over (#2232). + """ + + def test_returns_state_for_core_shaped_status(self): + status = ParticipantStatus( + context=_CONTEXT, vfd=VfdDimension(state=CS_vfd.Vfd) + ) + assert participant_status_vfd_state(status) is CS_vfd.Vfd + + def test_returns_initial_state_when_unset(self): + """A core status defaults its VFD dimension — that is not an error.""" + status = ParticipantStatus(context=_CONTEXT) + assert participant_status_vfd_state(status) is CS_vfd.vfd + + def test_raises_on_wire_shaped_status(self): + """A flat ``vfd_state`` status has no ``vfd`` — that must raise.""" + from vultron.wire.as2.vocab.objects.case_status import ( + as_ParticipantStatus, + ) + + wire_status = as_ParticipantStatus( + context=_CONTEXT, vfd_state=CS_vfd.Vfd + ) + assert getattr(wire_status, "vfd", None) is None + + with pytest.raises(VultronValidationError, match="'vfd' dimension"): + participant_status_vfd_state(wire_status) + + def test_raises_when_vfd_carries_no_vfd_state(self): + """A present-but-unusable ``vfd`` must raise rather than substitute.""" + + class _Bogus: + vfd = object() + + with pytest.raises(VultronValidationError, match="no valid VFD state"): + participant_status_vfd_state(_Bogus()) diff --git a/test/core/models/test_wire_spelling.py b/test/core/models/test_wire_spelling.py new file mode 100644 index 000000000..26cf0dfbb --- /dev/null +++ b/test/core/models/test_wire_spelling.py @@ -0,0 +1,206 @@ +#!/usr/bin/env python + +# Copyright (c) 2026 Carnegie Mellon University and Contributors. +# - see Contributors.md for a full list of Contributors +# - see ContributionInstructions.md for information on how you can Contribute to this project +# Vultron Multiparty Coordinated Vulnerability Disclosure Protocol Prototype is +# licensed under a MIT (SEI)-style license, please see LICENSE.md distributed +# with this Software or contact permission@sei.cmu.edu for full terms. +# Created, in part, with funding and support from the United States Government +# (see Acknowledgments file). This program may include and/or can make use of +# certain third party source code, object code, documentation and other files +# ("Third Party Software"). See LICENSE.md for more details. +# Carnegie Mellon®, CERT® and CERT Coordination Center® are registered in the +# U.S. Patent and Trademark Office by Carnegie Mellon University + +"""Tests for the wire-spelled-key guard (issue #2232, ARCH-15-001/002). + +The guard is computed *per exact class* rather than from a single module-level +map. That is the whole point: ``CaseParticipant`` has eight role subclasses, and +a subclass that adds a snake_case-only field would silently drop that field's +camelCase spelling if it inherited a map computed from its base. +""" + +import pytest +from pydantic import BaseModel, Field + +from vultron.core.models._wire_spelling import ( + clear_cache, + reject_wire_spelled_keys, + wire_spelled_keys, +) +from vultron.core.models.case_participant import ( + CaseActorParticipant, + CaseParticipant, + CoordinatorParticipant, + DeployerParticipant, + FinderParticipant, + FinderReporterParticipant, + OtherParticipant, + ReporterParticipant, + VendorParticipant, +) +from vultron.errors import VultronValidationError + +_ACTOR = "https://example.org/actors/alice" +_CONTEXT = "https://example.org/cases/case-2232" + +#: Every core participant class that can be validated from raw input. Listed +#: explicitly rather than via ``__subclasses__()`` so that adding a role class +#: without covering it here shows up as a missing entry, not a silently smaller +#: test matrix. +_PARTICIPANT_CLASSES = [ + CaseParticipant, + FinderParticipant, + ReporterParticipant, + FinderReporterParticipant, + VendorParticipant, + DeployerParticipant, + CoordinatorParticipant, + OtherParticipant, + CaseActorParticipant, +] + + +def test_every_case_participant_subclass_is_covered(): + """The matrix below must not fall behind the class hierarchy.""" + declared = set(_PARTICIPANT_CLASSES) + actual = {CaseParticipant, *CaseParticipant.__subclasses__()} + assert actual == declared, ( + "a new CaseParticipant role subclass was added without extending" + " _PARTICIPANT_CLASSES — its wire-shape guard would be untested" + ) + + +@pytest.mark.parametrize( + "model", _PARTICIPANT_CLASSES, ids=lambda c: c.__name__ +) +def test_wire_spelled_participant_statuses_raises(model): + """Each role subclass rejects ``participantStatuses`` in its own right.""" + data = { + "attributed_to": _ACTOR, + "context": _CONTEXT, + "participantStatuses": [], + } + with pytest.raises(VultronValidationError, match="participantStatuses"): + model.model_validate(data) + + +@pytest.mark.parametrize( + "model", _PARTICIPANT_CLASSES, ids=lambda c: c.__name__ +) +def test_canonical_snake_case_still_validates(model): + """The guard must not reject the canonical core shape.""" + participant = model.model_validate( + { + "attributed_to": _ACTOR, + "context": _CONTEXT, + "case_roles": [], + } + ) + assert participant.attributed_to == _ACTOR + + +class TestWireSpelledKeys: + """``wire_spelled_keys`` maps forbidden camelCase spellings per class.""" + + def test_snake_only_field_is_forbidden(self): + mapping = wire_spelled_keys(CaseParticipant) + assert mapping["participantStatuses"] == "participant_statuses" + + def test_sanctioned_alias_is_not_forbidden(self): + """``in_reply_to`` declares ``inReplyTo`` — a deliberate alias.""" + assert "inReplyTo" not in wire_spelled_keys(CaseParticipant) + + def test_trailing_underscore_fields_are_skipped(self): + """``id_``/``type_`` carry their own aliases and have no camel form.""" + mapping = wire_spelled_keys(CaseParticipant) + assert not any(key.startswith(("id", "type")) for key in mapping) + + def test_single_word_fields_are_skipped(self): + """``name``/``context`` camelCase to themselves, so cannot collide.""" + mapping = wire_spelled_keys(CaseParticipant) + assert "name" not in mapping + assert "context" not in mapping + + def test_subclass_gets_its_own_map_not_the_base_map(self): + """A subclass that adds a field must have that field guarded too. + + This is the hole a single shared module-level map would leave open: the + base's map knows nothing about ``extra_wire_field``, so a payload + spelling it ``extraWireField`` would be dropped in silence. + """ + + class _WithExtraField(CaseParticipant): + extra_wire_field: str | None = Field(default=None) + + try: + base_map = wire_spelled_keys(CaseParticipant) + sub_map = wire_spelled_keys(_WithExtraField) + assert "extraWireField" not in base_map + assert sub_map["extraWireField"] == "extra_wire_field" + + with pytest.raises(VultronValidationError, match="extraWireField"): + _WithExtraField.model_validate( + { + "attributed_to": _ACTOR, + "context": _CONTEXT, + "extraWireField": "dropped-in-silence", + } + ) + finally: + # The dynamic class would otherwise linger in the per-class cache. + clear_cache() + + def test_cache_returns_the_same_mapping_object(self): + assert wire_spelled_keys(CaseParticipant) is wire_spelled_keys( + CaseParticipant + ) + + +class TestRejectWireSpelledKeys: + """``reject_wire_spelled_keys`` is the validator-facing entry point.""" + + class _Model(BaseModel): + some_field: str | None = None + + def test_non_dict_input_passes_through(self): + """A ``mode="before"`` validator also sees non-dict input.""" + sentinel = object() + assert ( + reject_wire_spelled_keys(self._Model, sentinel, "hint") is sentinel + ) + + def test_clean_dict_is_returned_unchanged(self): + data = {"some_field": "ok"} + assert reject_wire_spelled_keys(self._Model, data, "hint") is data + + def test_error_names_the_boundary_the_caller_should_have_used(self): + """The message has to say what to do instead, not just what broke.""" + with pytest.raises(VultronValidationError) as exc_info: + reject_wire_spelled_keys( + self._Model, + {"someField": "wire-spelled"}, + "as_Thing.to_core()", + ) + message = str(exc_info.value) + assert "someField -> some_field" in message + assert "as_Thing.to_core()" in message + assert "#2232" in message + + def test_all_offenders_are_reported_not_just_the_first(self): + """Fixing one key at a time turns one bad payload into N round trips.""" + + class _TwoFields(BaseModel): + first_field: str | None = None + second_field: str | None = None + + with pytest.raises(VultronValidationError) as exc_info: + reject_wire_spelled_keys( + _TwoFields, + {"firstField": "a", "secondField": "b"}, + "hint", + ) + message = str(exc_info.value) + assert "firstField" in message + assert "secondField" in message diff --git a/test/core/use_cases/received/case/test_helpers.py b/test/core/use_cases/received/case/test_helpers.py index 538b8b66a..0e171d893 100644 --- a/test/core/use_cases/received/case/test_helpers.py +++ b/test/core/use_cases/received/case/test_helpers.py @@ -26,6 +26,8 @@ (BT-06-001, BT-15-001, #943). """ +from typing import Any, cast + import pytest from vultron.adapters.driven.datalayer_sqlite import SqliteDataLayer @@ -322,3 +324,230 @@ def test_reporter_participant_noop_if_already_closed( f"(it is already beyond ACCEPTED); got {len(statuses)} (#624)" ) assert statuses[0].rm.state == RM.CLOSED + + +# --------------------------------------------------------------------------- +# RM-regression guard must not go inert on a shape mismatch (issue #2232) +# --------------------------------------------------------------------------- + + +class TestParticipantRmStateShapeGuard: + """``_participant_rm_state`` must raise on a wire-shaped status. + + Regression for #2232: on a wire-shaped participant ``getattr(status, "rm")`` + was ``None``, so ``_participant_rm_state`` returned ``None`` and + ``_would_regress_participant`` returned ``False`` — the RM-rollback guard + shipped inert. A shape mismatch must raise (ARCH-15-001, ARCH-15-002); an + empty status list legitimately stays ``None``. + """ + + _CONTEXT = "https://example.org/cases/case-2232" + + def test_returns_latest_state_for_core_shaped_participant(self): + from vultron.core.models.case_participant import CaseParticipant + from vultron.core.use_cases.received.case._helpers import ( + _participant_rm_state, + ) + + actor = "https://example.org/actors/alice" + participant = CaseParticipant( + attributed_to=actor, context=self._CONTEXT + ) + participant.append_rm_state( + RM.RECEIVED, actor=actor, context=self._CONTEXT + ) + assert _participant_rm_state(participant) is RM.RECEIVED + + def test_returns_none_for_empty_status_list(self): + """Lenient where absence is legitimate (notes/domain-validation.md).""" + from vultron.core.use_cases.received.case._helpers import ( + _participant_rm_state, + ) + + class _NoStatuses: + participant_statuses: list = [] + + assert _participant_rm_state(_NoStatuses()) is None + + def test_raises_on_wire_shaped_participant(self): + from vultron.core.use_cases.received.case._helpers import ( + _participant_rm_state, + ) + from vultron.errors import VultronValidationError + from vultron.wire.as2.vocab.objects.case_participant import ( + as_CaseParticipant, + ) + + wire_participant = as_CaseParticipant( + attributed_to="https://example.org/actors/vendor", + context=self._CONTEXT, + ) + latest = wire_participant.participant_statuses[-1] + assert getattr(latest, "rm", None) is None + + with pytest.raises(VultronValidationError): + _participant_rm_state(wire_participant) + + +# --------------------------------------------------------------------------- +# Wire-shaped ingress must not abort the received-case path (issue #2232) +# --------------------------------------------------------------------------- + + +class TestStoreEmbeddedParticipantsProjectsWireIngress: + """``_store_embedded_participants`` must survive a wire-shaped snapshot. + + A received ``VulnerabilityCase`` is deserialised from AS2, so its embedded + participants are wire objects with a flat ``rm_state``. Making + ``_participant_rm_state`` raise on that shape (issue #2232) turned every + inbound ``Announce(VulnerabilityCase)`` into an aborted behavior tree unless + the participants are projected to core *at this ingress boundary* first. + + These tests pin the projection, not the raise: the raise is correct for a + corrupt stored row, and wrong as a response to legitimate inbound data. + """ + + _CASE_ID = "https://example.org/cases/case-2232-ingress" + _ACTOR_ID = "https://vendor.example.org/actors/vendor-2232" + _PARTICIPANT_ID = f"{_CASE_ID}/participants/vendor-2232" + + @pytest.fixture() + def dl(self): + return SqliteDataLayer("sqlite:///:memory:") + + def _wire_case(self, rm_state: RM) -> as_VulnerabilityCase: + """A received-shaped case carrying one wire participant at *rm_state*.""" + from vultron.wire.as2.vocab.objects.case_status import ( + as_ParticipantStatus, + ) + + wire_participant = as_CaseParticipant( + id_=self._PARTICIPANT_ID, + attributed_to=self._ACTOR_ID, + context=self._CASE_ID, + participant_statuses=[ + as_ParticipantStatus( + context=self._CASE_ID, + attributed_to=self._ACTOR_ID, + rm_state=rm_state, + ) + ], + ) + assert ( + getattr(wire_participant.participant_statuses[-1], "rm", None) + is None + ) + return as_VulnerabilityCase( + id_=self._CASE_ID, + name="Bug #2232 ingress case", + case_participants=[wire_participant], + ) + + def _seed_core_participant(self, dl, rm_state: RM) -> None: + """Store a core-shaped participant at *rm_state* before ingress.""" + status = ParticipantStatus( + rm=RmDimension(state=rm_state), + context=self._CASE_ID, + attributed_to=self._ACTOR_ID, + ) + dl.create( + VultronParticipant( + id_=self._PARTICIPANT_ID, + attributed_to=self._ACTOR_ID, + context=self._CASE_ID, + participant_statuses=[status], + ) + ) + + def test_wire_shaped_participant_is_stored_in_the_core_shape(self, dl): + """No raise escapes, and the persisted row is core-shaped.""" + from vultron.core.models.case_participant import CaseParticipant + from vultron.core.use_cases.received.case._helpers import ( + _store_embedded_participants, + ) + + case = self._wire_case(RM.RECEIVED) + + # The annotation says core ``VulnerabilityCase``, but the received + # path really hands it the deserialised wire case — that mismatch is + # exactly the shape duality under test (issue #2232). + _store_embedded_participants(cast(Any, case), dl, self._CASE_ID) + + stored = dl.read(self._PARTICIPANT_ID) + assert isinstance(stored, CaseParticipant), ( + "ingress must persist the canonical core type so dl.read() returns" + " a core object (DL-05-001)" + ) + latest = stored.participant_statuses[-1] + assert latest.rm.state == RM.RECEIVED + assert not hasattr(latest, "rm_state") + + def test_regression_guard_still_protects_a_local_core_participant( + self, dl + ): + """A behind-the-times wire snapshot must not roll local RM back. + + This is the case that exposed the ingress gap: the guard has to compare + a wire-shaped incoming against a core-shaped stored row, so it only + works once both sides go through the same projection. + """ + from vultron.core.use_cases.received.case._helpers import ( + _store_embedded_participants, + ) + + self._seed_core_participant(dl, RM.ACCEPTED) + case = self._wire_case(RM.RECEIVED) + + # The annotation says core ``VulnerabilityCase``, but the received + # path really hands it the deserialised wire case — that mismatch is + # exactly the shape duality under test (issue #2232). + _store_embedded_participants(cast(Any, case), dl, self._CASE_ID) + + stored = dl.read(self._PARTICIPANT_ID) + assert stored is not None + latest_rm = stored.participant_statuses[-1].rm.state + assert latest_rm == RM.ACCEPTED, ( + "local RM.ACCEPTED must survive an incoming RM.RECEIVED snapshot;" + f" got {latest_rm!r} (issue #2232)" + ) + + def test_forward_wire_snapshot_still_upgrades_local_participant(self, dl): + """A forward snapshot is applied — the guard is not blanket-inert.""" + from vultron.core.use_cases.received.case._helpers import ( + _store_embedded_participants, + ) + + self._seed_core_participant(dl, RM.RECEIVED) + case = self._wire_case(RM.VALID) + + # The annotation says core ``VulnerabilityCase``, but the received + # path really hands it the deserialised wire case — that mismatch is + # exactly the shape duality under test (issue #2232). + _store_embedded_participants(cast(Any, case), dl, self._CASE_ID) + + stored = dl.read(self._PARTICIPANT_ID) + assert stored is not None + assert stored.participant_statuses[-1].rm.state == RM.VALID + + def test_unprojectable_participant_is_skipped_not_fatal(self, caplog): + """One malformed participant must not cost the receiver the whole case. + + The HTTP inbox re-queues on exception, so letting a projection failure + propagate would turn the activity into an undrainable poison message. + """ + import logging + + from vultron.core.use_cases.received.case._helpers import ( + _project_to_core_participant, + ) + + class _NoToCore: + """Neither a core participant nor a wire projection.""" + + id_ = "https://example.org/cases/x/participants/bogus" + + with caplog.at_level(logging.ERROR): + result = _project_to_core_participant(_NoToCore(), _NoToCore.id_) + + assert result is None + assert "cannot be projected" in caplog.text diff --git a/test/core/use_cases/triggers/case/test_add_participant_status.py b/test/core/use_cases/triggers/case/test_add_participant_status.py index a3c5b67f2..a4206d652 100644 --- a/test/core/use_cases/triggers/case/test_add_participant_status.py +++ b/test/core/use_cases/triggers/case/test_add_participant_status.py @@ -30,6 +30,7 @@ from vultron.core.models.dimensions import RmDimension, VfdDimension from vultron.core.states.cs import CS_vfd from vultron.core.states.rm import RM +from vultron.errors import VultronValidationError # --------------------------------------------------------------------------- # Test stubs @@ -159,8 +160,17 @@ def test_resolve_participant_state_defaults_when_participant_not_found(): assert vfd == CS_vfd.vfd -def test_resolve_participant_state_defaults_when_invalid_rm_type(): - """Falls back to RM.START when rm_state is not an RM enum value.""" +def test_resolve_participant_state_raises_when_invalid_rm_type(): + """Raises when the latest status carries an unusable RM state. + + This previously fell back to ``RM.START``, which silently reset the + participant's RM ladder to its initial state and then rejected the next + legitimate transition as backwards (#2264, a symptom of #2232). A status + that exists but exposes no usable ``rm`` is a shape mismatch, not an + absence, so it must raise (ARCH-15-001..004). Absence — an empty + ``participant_statuses`` list — still returns ``RM.START``; see + ``test_resolve_participant_state_defaults_when_no_statuses``. + """ class _BadRmAttr: state = "not-an-rm" @@ -173,16 +183,23 @@ class _BadStatus: dl = _FakeDL(stored=participant) use_case = _make_use_case(dl) - rm, vfd = use_case._resolve_current_participant_state( - _as_persistence(dl), "any-id" - ) + with pytest.raises(VultronValidationError, match="no valid RM state"): + use_case._resolve_current_participant_state( + _as_persistence(dl), "any-id" + ) - assert rm == RM.START - assert isinstance(vfd, CS_vfd) +def test_resolve_participant_state_raises_when_invalid_vfd_type(): + """Raises when the latest status carries an unusable VFD state. -def test_resolve_participant_state_defaults_when_invalid_vfd_type(): - """Falls back to CS_vfd.vfd when vfd_state is not a CS_vfd enum value.""" + The VFD counterpart of + ``test_resolve_participant_state_raises_when_invalid_rm_type``: this + previously fell back to ``CS_vfd.vfd``, resetting the participant's + vendor-fix ladder to its initial state the same way ``RM.START`` reset the + RM ladder (#2264, a symptom of #2232). Absence — an empty + ``participant_statuses`` list — still returns ``CS_vfd.vfd``; see + ``test_resolve_participant_state_defaults_when_no_statuses``. + """ class _BadVfdAttr: state = "not-a-cs-vfd" @@ -195,12 +212,10 @@ class _BadStatus: dl = _FakeDL(stored=participant) use_case = _make_use_case(dl) - rm, vfd = use_case._resolve_current_participant_state( - _as_persistence(dl), "any-id" - ) - - assert isinstance(rm, RM) - assert vfd == CS_vfd.vfd + with pytest.raises(VultronValidationError, match="no valid VFD state"): + use_case._resolve_current_participant_state( + _as_persistence(dl), "any-id" + ) # --------------------------------------------------------------------------- diff --git a/vultron/adapters/driven/datalayer_sqlite/datalayer.py b/vultron/adapters/driven/datalayer_sqlite/datalayer.py index 73cbfde57..6cea368fc 100644 --- a/vultron/adapters/driven/datalayer_sqlite/datalayer.py +++ b/vultron/adapters/driven/datalayer_sqlite/datalayer.py @@ -31,6 +31,7 @@ from vultron.core.models.protocol_pair import ProtocolPair from vultron.core.models.protocols import PersistableModel from vultron.core.ports.datalayer import StorableRecord +from vultron.errors import VultronValidationError from vultron.semantic_registry import ( find_matching_semantics, semantics_to_activity_class as _semantics_to_activity_class, @@ -169,24 +170,130 @@ def _from_row(self, row: VultronObjectRecord) -> PersistableModel | None: ``as_Offer``), coerce via ``model_validate`` so that callers always receive the most precise type without manual coercion. """ + wire_obj: PersistableModel | None try: core_cls = find_in_core_vocabulary(row.type_) - obj = cast(PersistableModel, core_cls.model_validate(row.data)) - except (KeyError, ValidationError): - # KeyError: no core counterpart → fall back to wire vocabulary. - # ValidationError: stored data came from a wire object whose schema - # differs from the core class (e.g. as_EmbargoEvent lacks context). - # Fall back to the wire path in both cases. - rec = Record(id_=row.id_, type_=row.type_, data_=row.data) - try: - obj = cast(PersistableModel, record_to_object(rec)) - except (ValueError, ValidationError): + except KeyError: + # No core counterpart (AS2 Activity types) → wire vocabulary path. + wire_obj = self._wire_object_from_row(row) + if wire_obj is None: return None + obj = wire_obj + else: + try: + obj = cast(PersistableModel, core_cls.model_validate(row.data)) + except ValidationError: + # Stored data came from a wire object whose schema differs from + # the core class (e.g. as_EmbargoEvent lacks context). Return + # the wire object un-projected: that is the long-standing + # behaviour the KNOWN_WIRE_ESCAPES ratchet in + # test/architecture/test_dl_read_returns_core_objects.py + # measures, and projecting here would dehydrate inline nested + # objects that callers of these rows still expect inline. + wire_obj = self._wire_object_from_row(row) + if wire_obj is None: + return None + obj = wire_obj + except VultronValidationError as exc: + # A core type's own shape guard rejected the row — e.g. + # CaseParticipant's wire-spelled-key guard (#2232). It is not a + # ValueError subclass, so without naming it here it would escape + # this ladder entirely instead of falling back like every other + # shape mismatch (DL-05-002). + # + # Unlike the ValidationError case above, the row *is* a + # wire-spelled copy of a core type, so project it: handing back + # a wire object makes every core-typed caller fail (resolve_case + # raises "Expected VulnerabilityCase, got as_VulnerabilityCase"). + wire_obj = self._wire_object_from_row(row) + if wire_obj is None: + return None + obj = self._project_wire_row_to_core(row, wire_obj, exc) if obj is None: return None obj = self._rehydrate_fields(obj) return self._coerce_to_semantic_class(obj) + def _wire_object_from_row( + self, row: VultronObjectRecord + ) -> PersistableModel | None: + """Reconstruct *row* through the wire vocabulary, or ``None``.""" + rec = Record(id_=row.id_, type_=row.type_, data_=row.data) + try: + return cast(PersistableModel, record_to_object(rec)) + except (ValueError, ValidationError, VultronValidationError): + return None + + def _project_wire_row_to_core( + self, + row: VultronObjectRecord, + wire_obj: PersistableModel, + core_exc: Exception, + ) -> PersistableModel: + """Project a wire-vocabulary fallback back to its core counterpart. + + ``_from_row`` reaches this only when the row's ``type_`` *has* a core + counterpart but the stored data does not validate against it, so the + wire class was used instead. Handing that wire object to core callers + is what DL-05-001/DL-05-002 forbid: a wire ``as_VulnerabilityCase`` + reaching ``resolve_case`` raises "Expected VulnerabilityCase, got + as_VulnerabilityCase" rather than reading the case (issue #2232). + + ``to_core()`` is the same projection the write path applies in + ``_normalize_to_core`` — the persistence-boundary half of ADR-0062, + applied on the way out as well as on the way in. Wire types are looser + than core types, so a row that fails core validation directly can still + project cleanly: ``to_core()`` maps flat wire spellings onto the nested + core shape instead of dropping them. + + When the projection also fails, *wire_obj* is returned unchanged — that + is the pre-#2232 behaviour for these rows, and degrading it to ``None`` + would turn a wrongly-typed read into a missing-object read. Both + outcomes are logged: a silent fallback here is what made this class of + shape bug so hard to trace. + """ + to_core = getattr(wire_obj, "to_core", None) + if to_core is None: + logger.warning( + "Row %r (type %r) failed core validation (%s) and its wire" + " fallback %s has no to_core() projection; returning the wire" + " object (DL-05-002, issue #2232).", + row.id_, + row.type_, + core_exc, + type(wire_obj).__name__, + ) + return wire_obj + try: + projected = cast(PersistableModel, to_core()) + except ( + ValidationError, + VultronValidationError, + ValueError, + TypeError, + ) as exc: + logger.warning( + "Row %r (type %r) failed core validation (%s) and projecting" + " its wire fallback %s to core also failed (%s); returning the" + " wire object (issue #2232).", + row.id_, + row.type_, + core_exc, + type(wire_obj).__name__, + exc, + ) + return wire_obj + logger.debug( + "Row %r (type %r) failed core validation (%s); recovered the core" + " shape by projecting the wire fallback %s via to_core()" + " (issue #2232).", + row.id_, + row.type_, + core_exc, + type(wire_obj).__name__, + ) + return projected + def _rehydrate_fields(self, obj: PersistableModel) -> PersistableModel: """Expand dehydrated object-reference fields back to typed objects. @@ -322,7 +429,7 @@ def _coerce_to_semantic_class( obj.model_dump(by_alias=True, serialize_as_any=True) ), ) - except (ValidationError, TypeError) as exc: + except (ValidationError, VultronValidationError, TypeError) as exc: logger.warning( "Could not coerce %r to semantic class %r: %s", type(obj).__name__, @@ -338,7 +445,7 @@ def _object_from_storage( try: record = Record.model_validate(stored_record) return cast(PersistableModel, record_to_object(record)) - except (ValidationError, ValueError): + except (ValidationError, VultronValidationError, ValueError): pass raw_type = stored_record.get("type") @@ -348,7 +455,7 @@ def _object_from_storage( return cast( PersistableModel, vocab_cls.model_validate(stored_record) ) - except KeyError: + except (KeyError, ValidationError, VultronValidationError): pass raw_type = stored_record.get("type_") @@ -359,7 +466,7 @@ def _object_from_storage( return cast( PersistableModel, vocab_cls.model_validate(raw_data) ) - except KeyError: + except (KeyError, ValidationError, VultronValidationError): pass return None diff --git a/vultron/adapters/driven/datalayer_sqlite/schema.py b/vultron/adapters/driven/datalayer_sqlite/schema.py index 5eaa4b356..52beccd41 100644 --- a/vultron/adapters/driven/datalayer_sqlite/schema.py +++ b/vultron/adapters/driven/datalayer_sqlite/schema.py @@ -61,6 +61,23 @@ def matches_short_id(full_id: str, short_id: str) -> bool: return strip_id_prefix(full_id) == short_id +def _dimension_state(status: dict[str, Any], dimension: str) -> Any: + """Return a status dict's state for *dimension* in either persisted shape. + + The canonical core shape nests the state (``{"rm": {"state": "RECEIVED"}}``, + ADR-0036); the wire shape carries it flat (``{"rm_state": "RECEIVED"}``), + optionally camelCased. Reading only the flat spellings made this summary + report ``rm=None`` for every canonical row — removing the observability + that exists precisely to make shape migrations diagnosable (issue #2232). + """ + nested = status.get(dimension) + if isinstance(nested, dict): + state = nested.get("state") + if state is not None: + return state + return status.get(f"{dimension}_state") or status.get(f"{dimension}State") + + def participant_status_summary(data: Any) -> str: """Return a short debug summary of a CaseParticipant row's status list. @@ -71,9 +88,13 @@ def participant_status_summary(data: Any) -> str: """ if not isinstance(data, dict): return "" - statuses = data.get("participant_statuses") or data.get( - "participantStatuses" - ) + # Fall through on a *missing* key, not on a falsy one: an empty ladder is a + # participant row worth reporting as ``n_statuses=0`` (the state a re-seeded + # status list is about to be silently created from), and ``or`` made that + # branch unreachable by treating ``[]`` as "not a participant row". + statuses = data.get("participant_statuses") + if statuses is None: + statuses = data.get("participantStatuses") if not isinstance(statuses, list): return "" if not statuses: @@ -81,8 +102,8 @@ def participant_status_summary(data: Any) -> str: entries = [] for i, s in enumerate(statuses): if isinstance(s, dict): - vfd = s.get("vfd_state") or s.get("vfdState") - rm = s.get("rm_state") or s.get("rmState") + vfd = _dimension_state(s, "vfd") + rm = _dimension_state(s, "rm") pub = s.get("published") upd = s.get("updated") entries.append( diff --git a/vultron/adapters/driven/db_record.py b/vultron/adapters/driven/db_record.py index e2b6523d8..dc17c6543 100644 --- a/vultron/adapters/driven/db_record.py +++ b/vultron/adapters/driven/db_record.py @@ -17,14 +17,43 @@ """Provides a Record model for document database storage.""" -from typing import Any +from typing import Any, cast from pydantic import BaseModel, ValidationError from vultron.core.models.protocols import PersistableModel +from vultron.core.models.registry import CORE_VOCABULARY from vultron.core.ports.datalayer import StorableRecord +from vultron.errors import VultronValidationError from vultron.wire.as2.vocab.base.registry import find_in_vocabulary +_WIRE_MODULE_PREFIX = "vultron.wire.as2" + +# Wire vocabulary ``type_`` values are *bare* names ("CaseParticipant"), not +# ``as_``-prefixed, so the ``as_`` guard in ``Record.from_obj`` never fires for +# them. Fifteen wire classes therefore shadow a ``CORE_VOCABULARY`` entry and +# can be written into a core-typed row, producing a row whose field shape does +# not match the class that reads it back (issue #2232). +# +# Types listed here are normalised to their core counterpart via ``to_core()`` +# before serialisation, so the persisted row always carries the canonical core +# shape. The set may only GROW as the remaining shadowing types are migrated; +# it is the write-side analogue of ``KNOWN_WIRE_ESCAPES`` in +# ``test/architecture/test_dl_read_returns_core_objects.py`` (DL-05-004). +# +# ``ParticipantStatus`` and ``CaseParticipant`` are normalised because their +# two shapes are structurally incompatible: core nests ``rm: RmDimension`` +# while wire uses a flat ``rm_state``, so a wire-shaped row silently yields +# ``None`` for ``status.rm.state``. The other thirteen shadowing types +# (``VulnerabilityCase``, ``VulnerabilityReport``, the actor types, …) differ +# only by key spelling today and are not yet normalised — tracked in #2268. +_NORMALIZE_WIRE_TO_CORE: frozenset[str] = frozenset( + { + "CaseParticipant", + "ParticipantStatus", + } +) + # ActivityStreams fields typed as ``as_ObjectRef`` (accept URI string # references). Only these fields are candidates for dehydration. Fields # typed as concrete sub-objects (e.g. ``inbox``/``outbox`` on actors, @@ -195,6 +224,108 @@ def _retype_inline_object_refs( return obj +def _project_shadowing_wire_obj(obj: "BaseModel") -> "BaseModel": + """Project one object to its core counterpart when it shadows a core type. + + Returns *obj* unchanged unless it is a wire class whose bare ``type_`` + shadows a :data:`_NORMALIZE_WIRE_TO_CORE` entry. + + Raises: + VultronValidationError: when the wire object cannot be projected to its + core counterpart. Core types are stricter than wire types, so a + projection failure means the object was never valid domain data; + surfacing it beats persisting a row nothing can read (ARCH-15-002). + A dedicated error type — not a bare ``ValueError`` — because + ``crud.create`` raises ``ValueError`` for an already-existing row + and callers legitimately swallow *that*; the two must stay + distinguishable. + """ + if not type(obj).__module__.startswith(_WIRE_MODULE_PREFIX): + return obj + type_ = getattr(obj, "type_", None) + if not isinstance(type_, str): + return obj + if type_ not in _NORMALIZE_WIRE_TO_CORE or type_ not in CORE_VOCABULARY: + return obj + to_core = getattr(obj, "to_core", None) + if to_core is None: + raise VultronValidationError( + f"Wire class {type(obj).__name__} shadows core type '{type_}' but" + " has no to_core() projection, so it cannot be persisted in the" + " canonical core shape (issue #2232)." + ) + _PROJECTION_ERRORS = ( + ValidationError, + VultronValidationError, + ValueError, + TypeError, + ) + try: + return cast("BaseModel", to_core()) + except _PROJECTION_ERRORS as exc: + raise VultronValidationError( + f"Cannot persist {type(obj).__name__}" + f" '{getattr(obj, 'id_', '')}': projecting it to core type" + f" '{type_}' failed ({exc}). A wire-shaped '{type_}' row must not" + " be stored — normalise at the wire→core boundary instead" + " (issue #2232)." + ) from exc + + +def _normalize_to_core(obj: PersistableModel) -> PersistableModel: + """Return the core-shaped equivalent of *obj*, or *obj* unchanged. + + A wire vocabulary class whose bare ``type_`` shadows a ``CORE_VOCABULARY`` + entry would otherwise be written into a core-typed row in the wire field + shape, so whichever class reads the row back decides what the data means + (issue #2232). For the types in :data:`_NORMALIZE_WIRE_TO_CORE` the + difference is structural — core ``ParticipantStatus`` nests + ``rm: RmDimension`` where the wire shape carries a flat ``rm_state`` — so + the row is normalised here, at the persistence boundary, and no + wire-shaped row is ever stored. + + Both the object itself **and its direct children** are projected. Only + checking the top level left the invariant unmet in the case that motivated + it: a ``VulnerabilityCase`` row stores its ``case_participants`` inline, so + a wire-shaped participant nested inside a core-shaped case still persisted a + flat ``rm_state``. One level of child projection is sufficient because + ``to_core()`` recurses — projecting an ``as_CaseParticipant`` also projects + its ``as_ParticipantStatus`` children. + + Raises: + VultronValidationError: when a wire object (at either level) cannot be + projected to its core counterpart. + """ + if not isinstance(obj, BaseModel): + return obj + model = _project_shadowing_wire_obj(obj) + updates: dict[str, Any] = {} + for field_name in type(model).model_fields: + value = getattr(model, field_name, None) + if isinstance(value, BaseModel): + projected = _project_shadowing_wire_obj(value) + if projected is not value: + updates[field_name] = projected + elif isinstance(value, list) and value: + items = [ + ( + _project_shadowing_wire_obj(item) + if isinstance(item, BaseModel) + else item + ) + for item in value + ] + if any(new is not old for new, old in zip(items, value)): + updates[field_name] = items + if not updates: + return cast(PersistableModel, model) + # ``model_copy`` rather than re-validation: the parent's field is declared + # with the *wire* child type, so validating a core child against it would + # fail. ``model_dump(serialize_as_any=True)`` in ``from_obj`` serialises + # each child by its runtime type, so the core shape is what reaches the row. + return cast(PersistableModel, model.model_copy(update=updates)) + + class Record(StorableRecord): """Record wrapper stored in TinyDB. @@ -219,6 +350,11 @@ def from_obj(cls, obj: PersistableModel) -> "Record": "Object 'type_' attribute cannot start with 'as_' for Record conversion" ) + # Wire ``type_`` values are bare, so the guard above cannot catch a + # wire class shadowing a core type. Normalise those to the canonical + # core shape before serialising (issue #2232). + obj = _normalize_to_core(obj) + record = Record( id_=obj.id_, type_=obj.type_, diff --git a/vultron/adapters/driving/fastapi/routers/actors/_inbox.py b/vultron/adapters/driving/fastapi/routers/actors/_inbox.py index eda2ab02b..e13b56169 100644 --- a/vultron/adapters/driving/fastapi/routers/actors/_inbox.py +++ b/vultron/adapters/driving/fastapi/routers/actors/_inbox.py @@ -31,6 +31,7 @@ from vultron.core.models.actor import CoreActor from vultron.core.models.protocols import PersistableModel from vultron.core.ports.datalayer import DataLayer, StorableRecord +from vultron.errors import VultronValidationError from vultron.wire.as2.errors import ( VultronParseError, VultronParseMissingTypeError, @@ -194,8 +195,24 @@ def _store_nested_inbox_object( try: dl.create(object_to_record(typed_nested)) + except VultronValidationError: + # A shape/projection failure, NOT an "already exists" collision — the + # object cannot be persisted in the canonical core shape at all + # (issue #2232). Swallowing this silently alongside the duplicate case + # left the row absent and downstream nodes reporting a misleading + # "participant not found", so it is logged loudly instead. + logger.error( + "Not pre-storing inline %s %s from ingress: it cannot be projected" + " to the canonical core shape.", + nested.type_, + getattr(nested, "id_", ""), + exc_info=True, + ) except ValueError: - pass + logger.debug( + "Inline object %s already exists in shared DL; skipping re-store.", + getattr(nested, "id_", ""), + ) def _store_inbox_activity(dl: DataLayer, activity: as_Activity) -> None: diff --git a/vultron/core/behaviors/case/nodes/leave.py b/vultron/core/behaviors/case/nodes/leave.py index 08d20650c..48d4881e2 100644 --- a/vultron/core/behaviors/case/nodes/leave.py +++ b/vultron/core/behaviors/case/nodes/leave.py @@ -40,6 +40,9 @@ from vultron.core.behaviors.helpers import DataLayerAction from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_participant import CaseParticipant +from vultron.core.models.participant_status import ( + participant_status_rm_state, +) from vultron.core.states.rm import RM logger = logging.getLogger(__name__) @@ -109,9 +112,7 @@ def update(self) -> Status: # Idempotency: skip if already at RM.CLOSED for ps in participant.participant_statuses: - rm_dim = getattr(ps, "rm", None) - rm_state = getattr(rm_dim, "state", None) if rm_dim else None - if rm_state == RM.CLOSED: + if participant_status_rm_state(ps) == RM.CLOSED: self.logger.debug( "%s: participant '%s' already at RM.CLOSED — no-op", self.name, @@ -212,9 +213,7 @@ def update(self) -> Status: # Idempotency: skip if already at RM.CLOSED for ps in participant.participant_statuses: - rm_dim = getattr(ps, "rm", None) - rm_state = getattr(rm_dim, "state", None) if rm_dim else None - if rm_state == RM.CLOSED: + if participant_status_rm_state(ps) == RM.CLOSED: self.logger.debug( "%s: case actor '%s' already at RM.CLOSED — no-op", self.name, diff --git a/vultron/core/behaviors/case/nodes/participant/common.py b/vultron/core/behaviors/case/nodes/participant/common.py index d52533077..1d90b6900 100644 --- a/vultron/core/behaviors/case/nodes/participant/common.py +++ b/vultron/core/behaviors/case/nodes/participant/common.py @@ -28,6 +28,8 @@ ParticipantStatus, coerce_cvd_roles, coerce_em_consent_state, + participant_status_rm_state, + participant_status_vfd_state, ) from vultron.core.models.case import VulnerabilityCase from vultron.core.models.report_case_link import VultronReportCaseLink @@ -41,7 +43,10 @@ from vultron.core.states.cs import CS_vfd from vultron.core.states.rm import RM, is_rm_at_least from vultron.enums.roles import CVDRole -from vultron.core.models._helpers import _as_id, _report_phase_status_id +from vultron.core.models._helpers import ( + _as_id, + _report_phase_status_id, +) if TYPE_CHECKING: from vultron.core.ports.trigger_activity import TriggerActivityPort @@ -193,7 +198,19 @@ def resolve_participant_state_from_dl( dl: CasePersistence, participant_id: str, ) -> tuple[RM, CS_vfd]: - """Return (current_rm, current_vfd) from the participant's latest status.""" + """Return (current_rm, current_vfd) from the participant's latest status. + + ``(RM.START, CS_vfd.vfd)`` is returned only when the participant genuinely + has no recorded status — never as a fallback for a status that could not be + read. Substituting an initial state on an unreadable status silently reset + a participant's ladder (#2264, a symptom of #2232); a shape mismatch now + raises instead (ARCH-15-001, ARCH-15-002). Both dimensions go through + their canonical reader: leaving VFD on the old ``hasattr``/``isinstance`` + degrade would have kept the identical defect alive one dimension over. + + Raises: + VultronValidationError: when the latest status is not core-shaped. + """ participant_obj = dl.read(participant_id) if participant_obj is not None and hasattr( participant_obj, "participant_statuses" @@ -201,13 +218,10 @@ def resolve_participant_state_from_dl( statuses = getattr(participant_obj, "participant_statuses") if statuses: latest = statuses[-1] - raw_rm = latest.rm.state if hasattr(latest, "rm") else RM.START - raw_vfd = ( - latest.vfd.state if hasattr(latest, "vfd") else CS_vfd.vfd + return ( + participant_status_rm_state(latest), + participant_status_vfd_state(latest), ) - rm_state = raw_rm if isinstance(raw_rm, RM) else RM.START - vfd_state = raw_vfd if isinstance(raw_vfd, CS_vfd) else CS_vfd.vfd - return rm_state, vfd_state return RM.START, CS_vfd.vfd diff --git a/vultron/core/behaviors/helpers.py b/vultron/core/behaviors/helpers.py index 3be94bd6a..c7384f6b6 100644 --- a/vultron/core/behaviors/helpers.py +++ b/vultron/core/behaviors/helpers.py @@ -35,7 +35,7 @@ """ import logging -from typing import TYPE_CHECKING, Any, cast +from typing import TYPE_CHECKING, Any, cast, overload import py_trees from pydantic import BaseModel @@ -44,11 +44,16 @@ from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_participant import CaseParticipant +from vultron.core.models.participant_status import ( + participant_status_rm_state, +) from vultron.core.ports.case_persistence import ( CasePersistence, CaseOutboxPersistence, ) from vultron.core.ports.datalayer import DataLayer, StorableRecord +from vultron.core.states.rm import RM +from vultron.errors import VultronValidationError if TYPE_CHECKING: from vultron.core.ports.trigger_activity import TriggerActivityPort @@ -56,6 +61,67 @@ logger = logging.getLogger(__name__) +@overload +def read_rm_states( + node: py_trees.behaviour.Behaviour, + status: object, + /, +) -> tuple[RM] | None: ... + + +@overload +def read_rm_states( + node: py_trees.behaviour.Behaviour, + status: object, + other: object, + /, +) -> tuple[RM, RM] | None: ... + + +def read_rm_states( + node: py_trees.behaviour.Behaviour, + *statuses: object, +) -> tuple[RM, ...] | None: + """Return the RM states of *statuses*, or ``None`` to signal FAILURE. + + A ``ParticipantStatus`` whose RM dimension is unreadable is a shape + mismatch, not an absence. Substituting a default (``RM.START``, ``None``) + let an invalid transition through unchecked and silently reset a + participant's RM ladder (#2264, a symptom of #2232), so ARCH-15-001 and + ARCH-15-002 require FAILURE instead of a degraded SUCCESS. + + Callers for whom *absence* is legitimate (e.g. a participant with no + recorded status) must handle that case before calling. + + The one- and two-status arities are declared as ``@overload``\\ s with + fixed-length return tuples, so a caller that unpacks the result + (``new, current = states``) is arity-checked statically instead of raising + ``ValueError: too many values to unpack`` at runtime — where a BT node's + blanket handler would report it only as an opaque FAILURE. + + Args: + node: The calling BT node. Its ``feedback_message`` and ``logger`` are + used to report the mismatch, and its ``participant_id`` (when + present) identifies the participant in the message. + *statuses: Status objects to read, in the caller's preferred order. + + Returns: + A tuple of :class:`RM` states positionally matching *statuses*, or + ``None`` when any status is not core-shaped. On ``None`` the caller + must return ``Status.FAILURE``. + """ + try: + return tuple(participant_status_rm_state(s) for s in statuses) + except VultronValidationError as exc: + participant_id = getattr(node, "participant_id", "") + node.feedback_message = ( + "Non-canonical ParticipantStatus shape for participant" + f" '{participant_id}': {exc}" + ) + node.logger.error(f"{node.name}: {node.feedback_message}") + return None + + class DataLayerCondition(py_trees.behaviour.Behaviour): """ Base class for BT condition nodes that check state from DataLayer. diff --git a/vultron/core/behaviors/status/nodes/append.py b/vultron/core/behaviors/status/nodes/append.py index 872508edc..2f66a2313 100644 --- a/vultron/core/behaviors/status/nodes/append.py +++ b/vultron/core/behaviors/status/nodes/append.py @@ -26,7 +26,11 @@ import py_trees from py_trees.common import Status -from vultron.core.behaviors.helpers import DataLayerAction, DataLayerCondition +from vultron.core.behaviors.helpers import ( + DataLayerAction, + DataLayerCondition, + read_rm_states, +) from vultron.core.models.case_participant import CaseParticipant from vultron.core.models.participant_status import ParticipantStatus from vultron.core.models.protocols import PersistableModel @@ -299,19 +303,15 @@ def update(self) -> Status: ) return Status.FAILURE - new_rm_state = ( - status_obj.rm.state if hasattr(status_obj, "rm") else None - ) current_status = getattr(participant, "participant_status", None) - - if new_rm_state is None or current_status is None: - self.logger.debug( - "ValidateRMTransitionNode: no current status or new RM state," - " skipping validation" - ) + if current_status is None: + self.logger.debug("ValidateRMTransitionNode: no current status") return Status.SUCCESS - current_rm = current_status.rm.state + states = read_rm_states(self, status_obj, current_status) + if states is None: + return Status.FAILURE + new_rm_state, current_rm = states if current_rm == RM.CLOSED: self.feedback_message = ( "Participant is already in terminal RM.CLOSED state" @@ -464,9 +464,10 @@ def update(self) -> Status: if current_status is None: return Status.SUCCESS - current_rm = ( - current_status.rm.state if hasattr(current_status, "rm") else None - ) + states = read_rm_states(self, current_status) + if states is None: + return Status.FAILURE + (current_rm,) = states if current_rm != RM.CLOSED: return Status.SUCCESS diff --git a/vultron/core/behaviors/sync/nodes/effects.py b/vultron/core/behaviors/sync/nodes/effects.py index 1d70a689a..a2f343987 100644 --- a/vultron/core/behaviors/sync/nodes/effects.py +++ b/vultron/core/behaviors/sync/nodes/effects.py @@ -49,7 +49,10 @@ from vultron.core.models._helpers import _as_id from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_participant import CaseParticipant -from vultron.core.models.participant_status import ParticipantStatus +from vultron.core.models.participant_status import ( + ParticipantStatus, + participant_status_rm_state, +) logger = logging.getLogger(__name__) @@ -451,8 +454,7 @@ def update(self) -> Status: participant = self.datalayer.read(participant_id) if isinstance(participant, CaseParticipant): for ps in participant.participant_statuses: - rm_dim = getattr(ps, "rm", None) - if getattr(rm_dim, "state", None) == RM.CLOSED: + if participant_status_rm_state(ps) == RM.CLOSED: self.logger.debug( "%s: departing actor '%s' already at RM.CLOSED — no-op", self.name, diff --git a/vultron/core/behaviors/sync/nodes/fanout.py b/vultron/core/behaviors/sync/nodes/fanout.py index 0d39f76c6..7d47a5820 100644 --- a/vultron/core/behaviors/sync/nodes/fanout.py +++ b/vultron/core/behaviors/sync/nodes/fanout.py @@ -32,7 +32,10 @@ from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_ledger_entry import VultronCaseLedgerEntry from vultron.core.models.case_participant import CaseParticipant -from vultron.core.models.participant_status import ParticipantStatus +from vultron.core.models.participant_status import ( + ParticipantStatus, + participant_status_rm_state, +) from vultron.core.states.rm import RM from vultron.core.ports.sync_activity import SyncActivityPort @@ -75,11 +78,7 @@ def _is_rm_closed(self, participant_id: str) -> bool: ps = ps_ref if not isinstance(ps, ParticipantStatus): continue - rm_dim = getattr(ps, "rm", None) - if ( - rm_dim is not None - and getattr(rm_dim, "state", None) == RM.CLOSED - ): + if participant_status_rm_state(ps) == RM.CLOSED: return True return False diff --git a/vultron/core/models/_wire_spelling.py b/vultron/core/models/_wire_spelling.py new file mode 100644 index 000000000..3d94aae19 --- /dev/null +++ b/vultron/core/models/_wire_spelling.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python + +# Copyright (c) 2026 Carnegie Mellon University and Contributors. +# - see Contributors.md for a full list of Contributors +# - see ContributionInstructions.md for information on how you can Contribute to this project +# Vultron Multiparty Coordinated Vulnerability Disclosure Protocol Prototype is +# licensed under a MIT (SEI)-style license, please see LICENSE.md distributed +# with this Software or contact permission@sei.cmu.edu for full terms. +# Created, in part, with funding and support from the United States Government +# (see Acknowledgments file). This program may include and/or can make use of +# certain third party source code, object code, documentation and other files +# (“Third Party Software”). See LICENSE.md for more details. +# Carnegie Mellon®, CERT® and CERT Coordination Center® are registered in the +# U.S. Patent and Trademark Office by Carnegie Mellon University + +# Copyright + +"""Detection of wire-spelled (camelCase) keys in core-type input. + +A core type validated against a wire-shaped payload drops every key whose only +spelling is snake_case, because Pydantic v2 ignores unknown keys. That is how a +whole RM ladder disappeared without a trace in issue #2232. This module +computes, per model class, the set of camelCase spellings that would be dropped, +so a ``model_validator(mode="before")`` can reject them loudly instead +(ARCH-15-001, ARCH-15-002). + +Kept out of ``vultron/core/models/_helpers.py`` deliberately: that module cannot +import from ``vultron.core.states`` (circular import through +``states/__init__.py``), and while this module happens not to need those imports +today, colocating shape guards with the helpers that do need them would +re-create the cycle the first time one of them grew a state reference. +""" + +from typing import Any + +from pydantic import BaseModel +from pydantic.alias_generators import to_camel + +from vultron.errors import VultronValidationError + +#: Per-class cache for :func:`wire_spelled_keys`. Keyed by the exact class, so +#: a subclass that adds a field gets its own mapping rather than inheriting a +#: map computed from its base — the silent-drop hole that a single shared +#: module-level map would leave open. +_CACHE: dict[type[BaseModel], dict[str, str]] = {} + + +def wire_spelled_keys(model: type[BaseModel]) -> dict[str, str]: + """Map each field's forbidden camelCase spelling to its canonical name. + + A field's camelCase form is *sanctioned* — and therefore excluded — when the + field declares it as an explicit ``validation_alias`` (``in_reply_to`` → + ``inReplyTo``). Names ending in ``_`` (``id_``, ``type_``, ``context_``) + are skipped: they carry their own aliases and have no camelCase form. + Fields whose camelCase form equals their snake_case form (``name``, + ``context``) cannot collide and are skipped too. + + Results are cached per exact class; call :func:`clear_cache` if a test + defines model classes dynamically and needs the cache reset. + """ + cached = _CACHE.get(model) + if cached is not None: + return cached + mapping: dict[str, str] = {} + for name, field in model.model_fields.items(): + if name.endswith("_"): + continue + camel = to_camel(name) + if camel == name: + continue + if isinstance(field.validation_alias, str) and ( + field.validation_alias == camel + ): + continue + mapping[camel] = name + _CACHE[model] = mapping + return mapping + + +def clear_cache() -> None: + """Drop the per-class cache. Intended for tests only.""" + _CACHE.clear() + + +def reject_wire_spelled_keys( + model: type[BaseModel], data: Any, boundary_hint: str +) -> Any: + """Return *data* unchanged, or raise if it carries wire-spelled keys. + + Args: + model: The class being validated. Its own ``model_fields`` decide + which spellings are forbidden, so a subclass that adds a field is + covered without any registration step. + data: The raw ``model_validator(mode="before")`` input. Non-dict + input is passed through untouched. + boundary_hint: The wire→core projection the caller should have used + instead, quoted back in the error message + (e.g. ``"as_CaseParticipant.to_core()"``). + + Raises: + VultronValidationError: when at least one forbidden spelling is present. + """ + if not isinstance(data, dict): + return data + forbidden = wire_spelled_keys(model) + offenders = sorted(key for key in forbidden if key in data) + if not offenders: + return data + canonical = ", ".join(f"{key} -> {forbidden[key]}" for key in offenders) + raise VultronValidationError( + f"{model.__name__} received wire-spelled (camelCase) key(s)" + f" {offenders}, which this core type does not accept and Pydantic would" + f" silently discard: {canonical}. Convert at the wire→core boundary" + f" ({boundary_hint}) instead of validating a wire-shaped payload" + " against a core type. See issue #2232." + ) diff --git a/vultron/core/models/case_participant.py b/vultron/core/models/case_participant.py index 7af9ba817..0d212496f 100644 --- a/vultron/core/models/case_participant.py +++ b/vultron/core/models/case_participant.py @@ -34,10 +34,11 @@ from __future__ import annotations import logging -from typing import Literal +from typing import Any, Literal from pydantic import Field, field_serializer, field_validator, model_validator +from vultron.core.models._wire_spelling import reject_wire_spelled_keys from vultron.core.models.base import CoreObject, NonEmptyString from vultron.core.models.dimensions import PecDimension, RmDimension from vultron.core.models.participant_status import ( @@ -79,6 +80,48 @@ class CaseParticipant(CoreObject): embargo_consent_state: PEC = Field(default=PEC.NO_EMBARGO) participant_case_name: NonEmptyString | None = None + @model_validator(mode="before") + @classmethod + def _reject_wire_spelled_keys(cls, data: Any) -> Any: + """Raise on camelCase keys that Pydantic would silently discard. + + This class declares no ``alias_generator``, so a wire-spelled key such + as ``participantStatuses`` is an *unknown* key. Pydantic v2 ignores + unknown keys by default, so it was silently dropped and + ``_init_participant_status_if_empty`` then re-seeded a single status at + ``RM.START``: a whole RM ladder vanished without a trace (issue #2232). + The same drop applied to every other snake-only field on this model + (``case_roles``, ``accepted_embargo_ids``, ``embargo_consent_state``, + ``participant_case_name``), so roles could be lost the same way. + + Wire→core conversion belongs at the boundary + (``as_CaseParticipant.to_core()``, which emits snake_case), not here. + This validator makes the mismatch loud instead of lossy + (ARCH-15-001, ARCH-15-002). + + Fields that declare an explicit camelCase ``validation_alias`` (e.g. + ``in_reply_to``/``inReplyTo``) are sanctioned spellings and are + accepted unchanged. + + **This guard is one level deep, by design and not by accident.** The + nested :class:`ParticipantStatus` *does* set + ``alias_generator=to_camel`` and accepts flat wire spellings + (``rmState``) through its own migration shim, so + ``{"participant_statuses": [{"rmState": "CLOSED"}]}`` is accepted here + and yields ``rm.state == RM.CLOSED``. That asymmetry is a known + deviation from ARCH-12-003 tracked in #1991 — the child's shim is what + makes this parent guard survivable in the first place — and it is not + a hole in the #2232 fix: an aliased child cannot *lose* the ladder, it + only spells it differently. Do not restate ARCH-12-003 as though it + held throughout this subtree; it does not yet. + + Raises: + VultronValidationError: when a wire-spelled key is present. + """ + return reject_wire_spelled_keys( + cls, data, "as_CaseParticipant.to_core()" + ) + @field_serializer("case_roles") def _serialize_case_roles(self, value: list[CVDRole]) -> list[str]: return serialize_roles(value) diff --git a/vultron/core/models/participant_status.py b/vultron/core/models/participant_status.py index a87468ce5..185694028 100644 --- a/vultron/core/models/participant_status.py +++ b/vultron/core/models/participant_status.py @@ -26,8 +26,11 @@ ) from pydantic.alias_generators import to_camel +from vultron.core.states.cs import CS_vfd from vultron.core.states.participant_embargo_consent import PEC +from vultron.core.states.rm import RM from vultron.enums.roles import CVDRole +from vultron.errors import VultronValidationError from vultron.core.models.base import CoreObject, NonEmptyString from vultron.core.models.case_status import CaseStatus from vultron.core.models.dimensions import ( @@ -151,3 +154,95 @@ def _serialize_cvd_role(self, roles: list[CVDRole]) -> list[str]: @classmethod def _validate_cvd_role(cls, v: object) -> list[CVDRole]: return coerce_cvd_roles(v) + + +def participant_status_rm_state(status: object) -> RM: + """Return the RM state of a single ``ParticipantStatus``. + + This is the canonical RM-dimension reader. Core :class:`ParticipantStatus` + carries a nested ``rm: RmDimension`` (ADR-0036, SDO-03-002); the wire + projection ``as_ParticipantStatus`` carries a flat ``rm_state: RM`` and no + ``rm`` attribute at all. Reading ``rm`` off a wire-shaped status therefore + yields ``None``, and every caller that tolerated that ``None`` silently + took a wrong branch — the defect behind issue #2232. + + A status object always has an RM state in the canonical shape (``rm`` has a + ``default_factory``), so there is no legitimate ``None`` outcome here: an + absent or unusable ``rm`` means the object is not core-shaped, and that is a + defect to surface rather than absorb (ARCH-15-001..004). + + Callers for whom *absence* is legitimate — e.g. a participant with an empty + ``participant_statuses`` list — must make that check themselves before + calling, per the lenient-helper rule in ``notes/domain-validation.md``. + + Args: + status: A single participant status object. + + Returns: + The :class:`RM` state recorded on *status*. + + Raises: + VultronValidationError: when *status* exposes no usable ``rm`` + dimension — typically because it is a wire-shaped status that + should have been normalised at the wire→core boundary. + """ + rm = getattr(status, "rm", None) + if rm is None: + raise VultronValidationError( + f"ParticipantStatus {getattr(status, 'id_', status)!r} has no 'rm'" + f" dimension (got a {type(status).__name__}). Core" + " ParticipantStatus uses a nested 'rm: RmDimension'; the wire" + " shape uses a flat 'rm_state'. Convert at the wire→core boundary" + " (as_ParticipantStatus.to_core()) instead of reading the wire" + " shape here. See issue #2232." + ) + state = getattr(rm, "state", None) + if not isinstance(state, RM): + raise VultronValidationError( + f"ParticipantStatus {getattr(status, 'id_', status)!r} has an 'rm'" + f" dimension with no valid RM state (got {state!r}). See issue" + " #2232." + ) + return state + + +def participant_status_vfd_state(status: object) -> CS_vfd: + """Return the VFD state of a single ``ParticipantStatus``. + + The VFD-dimension twin of :func:`participant_status_rm_state`, with the + same contract and for the same reason: core :class:`ParticipantStatus` + carries a nested ``vfd: VfdDimension`` while the wire projection carries a + flat ``vfd_state``, so reading ``vfd`` off a wire-shaped status yields + ``None``. Substituting the initial state (``CS_vfd.vfd``) silently reset a + participant's vendor-fix ladder exactly the way ``RM.START`` reset the RM + ladder (#2264, a symptom of #2232). + + Args: + status: A single participant status object. + + Returns: + The :class:`CS_vfd` state recorded on *status*. + + Raises: + VultronValidationError: when *status* exposes no usable ``vfd`` + dimension — typically because it is a wire-shaped status that + should have been normalised at the wire→core boundary. + """ + vfd = getattr(status, "vfd", None) + if vfd is None: + raise VultronValidationError( + f"ParticipantStatus {getattr(status, 'id_', status)!r} has no" + f" 'vfd' dimension (got a {type(status).__name__}). Core" + " ParticipantStatus uses a nested 'vfd: VfdDimension'; the wire" + " shape uses a flat 'vfd_state'. Convert at the wire→core boundary" + " (as_ParticipantStatus.to_core()) instead of reading the wire" + " shape here. See issue #2232." + ) + state = getattr(vfd, "state", None) + if not isinstance(state, CS_vfd): + raise VultronValidationError( + f"ParticipantStatus {getattr(status, 'id_', status)!r} has a 'vfd'" + f" dimension with no valid VFD state (got {state!r}). See issue" + " #2232." + ) + return state diff --git a/vultron/core/use_cases/_helpers.py b/vultron/core/use_cases/_helpers.py index 9f242d138..1ba0348a6 100644 --- a/vultron/core/use_cases/_helpers.py +++ b/vultron/core/use_cases/_helpers.py @@ -12,6 +12,9 @@ from vultron.core.models._helpers import _as_id from vultron.core.models.case import VulnerabilityCase from vultron.core.models.case_participant import CaseParticipant +from vultron.core.models.participant_status import ( + participant_status_rm_state, +) from vultron.core.models.report_case_link import VultronReportCaseLink from vultron.core.ports.case_persistence import ( CasePersistence, @@ -446,8 +449,11 @@ def current_participant_rm_state( statuses = participant.participant_statuses if not statuses: return RM.START - state = statuses[-1].rm.state - return state if isinstance(state, RM) else RM.START + # Canonical reader rather than an isinstance-guarded ``RM.START`` fallback: + # substituting the initial state for an unreadable one is the #2264 defect, + # and ``participant`` is an already-validated core CaseParticipant here, so + # its latest status always carries a usable ``rm`` dimension (issue #2232). + return participant_status_rm_state(statuses[-1]) def _resolve_case_manager_id( diff --git a/vultron/core/use_cases/received/case/_helpers.py b/vultron/core/use_cases/received/case/_helpers.py index 40a784f01..f69d32be2 100644 --- a/vultron/core/use_cases/received/case/_helpers.py +++ b/vultron/core/use_cases/received/case/_helpers.py @@ -2,6 +2,8 @@ import logging +from pydantic import ValidationError + from vultron.core.behaviors.case.nodes.participant.common import ( # noqa: F401 _ensure_reporter_participant, _upgrade_participant_to_accepted, @@ -10,9 +12,14 @@ find_excluded_actor_ids, ) from vultron.core.models.case import VulnerabilityCase +from vultron.core.models.case_participant import CaseParticipant +from vultron.core.models.participant_status import ( + participant_status_rm_state, +) from vultron.core.models.report_case_link import VultronReportCaseLink from vultron.core.ports.case_persistence import CasePersistence from vultron.core.states.rm import RM, is_monotonic_rm_forward +from vultron.errors import VultronValidationError logger = logging.getLogger(__name__) @@ -66,6 +73,11 @@ def _store_embedded_participants( Idempotent: ``dl.save()`` upserts so repeated calls are safe. + Each embedded participant is projected to the canonical core shape first + (see :func:`_project_to_core_participant`) — a received snapshot arrives in + the wire shape, and both the regression check below and every later reader + of the stored row require the core shape (issue #2232). + A received snapshot is a remote point-in-time view, so it must never regress local RM progress. Bootstrap and Announce activities are built before delivery and may arrive after the receiver has already advanced a @@ -86,9 +98,12 @@ def _store_embedded_participants( pid = getattr(participant_ref, "id_", None) if pid is None: continue - if _would_regress_participant(participant_ref, dl, pid, case_id): + participant = _project_to_core_participant(participant_ref, pid) + if participant is None: continue - dl.save(participant_ref) + if _would_regress_participant(participant, dl, pid, case_id): + continue + dl.save(participant) logger.debug( "store_embedded_participants: stored participant '%s'" " for case '%s' (CBT-05-005, #566)", @@ -97,26 +112,119 @@ def _store_embedded_participants( ) +def _project_to_core_participant( + participant_ref: object, pid: str +) -> CaseParticipant | None: + """Return *participant_ref* as a canonical core participant, or ``None``. + + This is the wire→core ingress boundary for embedded participants. A + received ``VulnerabilityCase`` snapshot is deserialised from AS2, so its + ``case_participants`` are wire objects (``as_CaseParticipant``) carrying + wire-shaped statuses with a flat ``rm_state`` — legitimate inbound data, not + a corrupt row. Every core-side reader below this point (the RM comparison + in :func:`_would_regress_participant`, and anything that later reads the + stored row) requires the canonical nested ``rm: RmDimension`` shape, so the + projection has to happen here rather than being discovered downstream + (issue #2232). + + Projecting at ingress rather than only at persistence also means the row + that lands in the DataLayer is core-shaped, which is what makes + ``dl.read()`` return a core object per DL-05-001. + + ``None`` means *this participant cannot be stored* and the caller must skip + it. A projection failure is logged at ERROR: core types are stricter than + wire types, so it means the sender's snapshot was never valid domain data. + Skipping one unprojectable participant is deliberately preferred over + letting the exception abort the whole received-case behavior tree — a single + malformed embedded participant must not cost the receiver the entire case + (and, because the HTTP inbox re-queues on exception, must not turn the + activity into an undrainable poison message). + + Args: + participant_ref: An embedded participant object from the snapshot, + either core-shaped already or a wire projection exposing + ``to_core()``. + pid: The participant's ID, for log context. + + Returns: + A core :class:`CaseParticipant` (possibly a role subclass), or ``None`` + when the object cannot be represented in the canonical core shape. + """ + if isinstance(participant_ref, CaseParticipant): + return participant_ref + to_core = getattr(participant_ref, "to_core", None) + if to_core is None: + logger.error( + "participant '%s' cannot be projected to the canonical core" + " shape and will be skipped: a" + " %s exposes no to_core() projection, so it cannot be stored in" + " the canonical core shape (issue #2232).", + pid, + type(participant_ref).__name__, + ) + return None + try: + projected = to_core() + except (ValidationError, VultronValidationError, ValueError, TypeError): + logger.error( + "participant '%s' cannot be projected to the canonical core shape" + " and will be skipped: its %s snapshot failed core validation" + " (issue #2232).", + pid, + type(participant_ref).__name__, + exc_info=True, + ) + return None + if not isinstance(projected, CaseParticipant): + logger.error( + "participant '%s' cannot be projected to the canonical core shape" + " and will be skipped: %s.to_core() returned a %s, not a core" + " CaseParticipant (issue #2232).", + pid, + type(participant_ref).__name__, + type(projected).__name__, + ) + return None + return projected + + def _participant_rm_state(participant: object) -> RM | None: - """Return the latest RM state recorded on *participant*, if any.""" + """Return the latest RM state recorded on *participant*, if any. + + ``None`` means *no status has been recorded yet* — a legitimate state that + callers must handle. It does **not** mean "the status was unreadable": + a status that exists but exposes no usable ``rm`` dimension raises, because + that is a shape mismatch rather than an absence (issue #2232, ARCH-15). + + Raises: + VultronValidationError: when the latest status is not core-shaped. + """ statuses = getattr(participant, "participant_statuses", None) or [] if not statuses: return None - rm = getattr(statuses[-1], "rm", None) - state = getattr(rm, "state", None) - return state if isinstance(state, RM) else None + return participant_status_rm_state(statuses[-1]) def _would_regress_participant( - incoming: object, dl: CasePersistence, pid: str, case_id: str + incoming: CaseParticipant, dl: CasePersistence, pid: str, case_id: str ) -> bool: """Return ``True`` when saving *incoming* would roll back local RM state. Only the RM dimension is compared: it is the dimension whose state machine rejects backward transitions outright, so a regression there is what actually breaks subsequent protocol progress. + + Both sides are read through the canonical RM reader, so both must be + core-shaped. *incoming* is projected by the caller; the stored side is + projected here because a legacy wire-shaped row can still be returned by + ``dl.read()`` via the DL-05-004 escape list. When the stored side cannot be + read, ``False`` is returned: an incoming canonical snapshot overwriting an + unreadable row is an improvement, not a regression. """ - existing = dl.read(pid) + stored = dl.read(pid) + if stored is None: + return False + existing = _project_to_core_participant(stored, pid) if existing is None: return False