diff --git a/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md b/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md new file mode 100644 index 00000000000..52ce813d61b --- /dev/null +++ b/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md @@ -0,0 +1,227 @@ +# CAC Procurement and Audit Standard v1.0 + +## 1) STANDARD DOCUMENT (RFC-STYLE) + +### 1.1 Abstract + +This standard defines mandatory controls for Cognitive Admissibility Criteria (CAC) in procurement, audit, and regulatory assurance workflows. A decision system is conformant only when each governed decision outcome is accompanied by verifiable evidence artifacts, a signed admissibility certificate (`CACert`), a reproducible decision trace, and an inclusion proof in an append-only transparency log. + +### 1.2 Scope + +This standard applies to any system that produces, ranks, recommends, filters, or automates decisions used in: + +- regulated business workflows; +- high-impact operational decisions; +- customer-, employee-, or citizen-affecting determinations; and +- supplier-delivered AI/analytics modules accepted through enterprise procurement. + +Out of scope: purely informational systems with no decision effect and no downstream control action. + +### 1.3 Definitions + +- **Decision System**: Software or pipeline that emits or materially influences a decision output. +- **Evidence Bundle**: Immutable artifact set containing inputs, model/runtime metadata, policy version identifiers, execution timestamps, and integrity digests. +- **Decision Trace**: Machine-readable lineage from input evidence to output decision, including policy evaluations and execution path. +- **Admissibility Verdict**: Binary verdict (`PASS` or `FAIL`) computed by policy-controlled CAC verification. +- **CACert**: Cryptographically signed certificate binding verdict, artifact digests, policy versions, issuer identity, and validity window. +- **Transparency Log**: Append-only Merkle-based log storing signed statement entries and enabling inclusion/consistency proofs. +- **Verifier**: Independent implementation (CLI/API) that validates signatures, digests, policy bindings, and transparency proofs. + +### 1.4 Normative Language + +The key words **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, and **MAY** are to be interpreted as normative requirements. + +### 1.5 Normative Requirements + +#### 1.5.1 Evidence Requirements + +1. Each governed decision output **MUST** generate an Evidence Bundle with unique identifier and immutable digest set. +2. Evidence Bundle **MUST** include: input references, transformation lineage, model/version identifiers, policy bundle hash, execution environment metadata, timestamp, and actor/process identity. +3. Evidence Bundle **MUST** be retained for the policy-defined retention period and **MUST** be retrievable by auditors via deterministic identifier. +4. Missing required Evidence Bundle fields **MUST** produce verdict `FAIL`. + +#### 1.5.2 Traceability Requirements + +1. Each decision **MUST** produce a Decision Trace that links every output claim to source evidence and policy evaluation result. +2. Trace records **MUST** include ordered execution steps and **MUST** support deterministic replay against the same artifact set. +3. Any trace discontinuity, unverifiable step, or non-deterministic replay **MUST** produce verdict `FAIL`. + +#### 1.5.3 Admissibility Verdict Requirements + +1. Each governed decision **MUST** be evaluated by CAC policy controls before deployment, release, or externally consumed output publication. +2. Verdict **MUST** be binary (`PASS` or `FAIL`) with no indeterminate production state. +3. Systems **MUST NOT** promote or deploy artifacts with `FAIL` verdict. +4. Verdict computation inputs **MUST** include Evidence Bundle digests, Decision Trace digest, policy version, and verification result set. + +#### 1.5.4 Certification (CACert) Requirements + +1. Every `PASS` verdict **MUST** produce a signed `CACert`. +2. `CACert` **MUST** bind: system identifier, decision/evaluation identifier, verdict, artifact digests, policy versions, issuer identity, issuance timestamp, expiry timestamp, and signature. +3. `CACert` **MUST** be machine-verifiable without vendor-proprietary tooling dependencies. +4. Expired, revoked, unverifiable, or mismatched `CACert` **MUST** be treated as non-conformant (`FAIL`). + +### 1.6 Verification Requirements + +1. Conformant deployments **MUST** provide independent verification via at least one API and one CLI execution path. +2. Verifier **MUST** validate: + - cryptographic signature integrity; + - digest equality between artifacts and certificate; + - policy/version binding; + - Decision Trace integrity; and + - transparency inclusion and consistency proofs. +3. Verification outcome **MUST** be reproducible by third parties with documented commands and fixed input artifact references. +4. Any verification error **MUST** produce verdict `FAIL`. + +### 1.7 Transparency Log Requirements + +1. Each issued `CACert` **MUST** be logged in an append-only Merkle transparency log. +2. Logged entry **MUST** include certificate digest, issuer identity, timestamp, and subject identifier. +3. System **MUST** provide inclusion proof for each logged certificate and consistency proof across checkpoints. +4. Log entries **MUST NOT** be deleted or rewritten. +5. Missing inclusion proof or failed consistency proof **MUST** produce verdict `FAIL`. + +--- + +## 2) PROCUREMENT LANGUAGE (RFP READY) + +1. Vendor **MUST** provide a verifiable `CACert` for each production decision workflow in scope of this procurement. +2. Vendor **MUST** provide independent verification via CLI and API, with documented commands enabling buyer-side validation of `CACert`, evidence digests, and transparency proofs. +3. All in-scope outputs **MUST** pass CAC admissibility checks before release to production or customer-facing consumption. +4. Vendor **MUST NOT** classify any failed admissibility result as “warning,” “degraded,” or “acceptable risk” for production use. +5. Vendor **MUST** provide a machine-readable Evidence Bundle for every sampled decision specified by the buyer during evaluation and audit. +6. Evidence Bundle **MUST** include immutable identifiers, timestamps, policy version hashes, model/runtime identifiers, and complete artifact digests. +7. Vendor **MUST** provide Decision Trace artifacts that enable deterministic replay of sampled decisions without undocumented vendor-only tooling. +8. Vendor **MUST** maintain an append-only transparency log for all issued `CACert` records and provide inclusion/consistency proofs on request. +9. Vendor **MUST** disclose certificate issuer keys, trust chain, revocation endpoint, and certificate validity policy. +10. Vendor **MUST** fail closed: any missing, revoked, expired, or unverifiable `CACert` is an automatic non-conformance. +11. Vendor **MUST** support third-party auditor re-verification of sampled decisions within five business days of request. +12. Vendor **MUST** retain admissibility artifacts for a minimum of the buyer-defined retention period or regulatory minimum, whichever is longer. +13. Vendor **MUST** notify buyer within 24 hours of any control failure affecting admissibility enforcement, certificate issuance integrity, or transparency log integrity. +14. Buyer **MAY** require periodic blind-sample verification; vendor failure rate above buyer threshold constitutes material breach. +15. Contract acceptance **MUST** be conditioned on successful demonstration of CAC pass/fail determinism and third-party reproducible verification. + +--- + +## 3) AUDIT FRAMEWORK + +### 3.1 Audit Checklist + +1. Confirm system inventory and identify all in-scope decision workflows. +2. For each sampled workflow, obtain: Evidence Bundle, Decision Trace, Admissibility Verdict, `CACert`, transparency inclusion proof, and consistency proof. +3. Validate certificate signature, issuer trust, expiry, and revocation state. +4. Recompute artifact digests and compare against `CACert` bindings. +5. Re-run verifier (CLI and/or API) on sampled artifacts. +6. Validate deterministic replay of sampled Decision Traces. +7. Confirm failed verdicts are blocked from production pathways. +8. Confirm retention, retrieval latency, and chain-of-custody controls for admissibility artifacts. + +### 3.2 Required Artifacts + +- System scope register for governed decisions. +- Evidence Bundle files for each audit sample. +- Decision Trace files for each audit sample. +- Admissibility verdict records (PASS/FAIL) with policy version reference. +- Signed `CACert` artifacts and issuer trust material. +- Transparency log checkpoints, inclusion proofs, and consistency proofs. +- Verification run logs (tool version, command, timestamp, result). +- Control exception register and remediation evidence. + +### 3.3 Verification Procedures + +1. Select statistically valid sample set and any regulator-mandated targeted samples. +2. Execute independent verifier using auditor-controlled environment. +3. Validate digest integrity end-to-end from evidence through certificate. +4. Validate transparency proof against published checkpoint. +5. Conduct deterministic replay and compare verdict and output determinism. +6. Record all results in immutable audit workpapers. + +### 3.4 Pass/Fail Conditions + +**PASS** requires all of the following: + +- 100% sampled decisions have complete required artifacts. +- 100% sampled `CACert` signatures validate and are unexpired/unrevoked. +- 100% sampled inclusion/consistency proofs validate. +- 100% sampled deterministic replay checks match recorded admissibility outcomes. +- 0 production decisions lack a `PASS` admissibility verdict. + +**FAIL** is triggered by any one of: + +- Missing artifact required by Section 3.2. +- Unverifiable or mismatched certificate/artifact digest. +- Failed transparency proof. +- Non-deterministic replay without approved governed exception. +- Evidence of production release despite `FAIL` verdict. + +--- + +## 4) COMPLIANCE MAPPING + +### 4.1 NIST AI RMF Alignment + +- **Govern (GV):** CAC defines accountable control gates via binary admissibility verdicts and certifiable evidence. +- **Map (MP):** CAC requires scoped decision inventory and explicit artifact lineage. +- **Measure (ME):** CAC mandates machine-verifiable proofs, deterministic replay, and measurable pass/fail outcomes. +- **Manage (MG):** CAC enforces fail-closed deployment blocking and remediation loops. + +**Gap CAC Fills:** Converts AI risk guidance into hard acceptance controls with cryptographic evidence and independent verifier reproducibility. + +### 4.2 ISO/IEC 42001 Alignment + +- Aligns with AI management-system requirements for traceability, control operation, monitoring, and continual assurance through auditable artifacts. +- Supports objective evidence expectations for conformity assessment and internal/external audits. + +**Gap CAC Fills:** Adds concrete admissibility primitives (`CACert`, transparency proofs, deterministic replay) where management-system language is process-oriented but not artifact-specific. + +### 4.3 SOC 2 Alignment (Relevant Controls) + +- **CC1/CC2 (Control environment, communication):** Defined ownership and policy-bound admissibility controls. +- **CC6 (Logical access):** Controlled issuance and validation paths for certificates and evidence access. +- **CC7 (Change management/monitoring):** CI enforcement and verification logs for every governed decision release. +- **CC8 (Risk mitigation):** Fail-closed gating for inadmissible outputs. +- **A1.2/A1.3 (Availability processing integrity, where applicable):** Deterministic replay and artifact integrity checks support processing integrity assertions. + +**Gap CAC Fills:** Provides testable, decision-level evidence of processing integrity beyond narrative control descriptions. + +### 4.4 EU AI Act Alignment + +- Supports transparency/accountability obligations through traceable decision artifacts and verifiable logs. +- Supports high-risk system governance by requiring pre-deployment admissibility pass/fail enforcement. +- Supports post-market monitoring through retained evidence and independent re-verification capability. + +**Gap CAC Fills:** Supplies operationally testable, cryptographically bound proof objects for conformity and supervisory review, reducing reliance on self-attested documentation. + +--- + +## 5) ADOPTION STRATEGY (CRITICAL) + +### Phase 0–90 Days: Procurement Insertion and Baseline Qualification + +1. Publish this standard as an official procurement annex and mandatory evaluation schedule. +2. Insert Section 2 clauses into all new AI/decision-system RFP templates. +3. Require bidders to submit sample `CACert`, evidence bundle, and transparency proof package during pre-award evaluation. +4. Define disqualification rule: inability to produce independently verifiable artifacts equals non-responsive bid. +5. Train procurement, security, and internal audit teams on pass/fail evaluation script and artifact validation workflow. + +### Phase 90–180 Days: Audit Operationalization and Contractual Enforcement + +1. Amend active vendor contracts at renewal/addendum to require CAC conformance for in-scope workflows. +2. Launch third-party audit procedures in Section 3 with quarterly sample verification. +3. Establish supplier scorecard metric: “Admissibility Verification Success Rate” (target 100%). +4. Enforce remediation SLA for any fail trigger (e.g., immediate block + corrective action evidence). +5. Publish internal governance policy requiring `CACert` proof for promotion approvals in regulated workloads. + +### Phase 180+ Days: Market Conditioning and Category Lock-In + +1. Make CAC conformance a non-negotiable prerequisite in all strategic sourcing categories involving AI decision outputs. +2. Require platform integrators and prime contractors to flow down CAC clauses to subcontractors. +3. Share anonymized audit outcomes with risk committees and regulators to normalize CAC as accepted assurance evidence. +4. Tie preferred-vendor status to sustained zero-fail audit outcomes and transparent proof availability. +5. Expand buyer consortium language (industry groups/standard appendices) so CAC verification becomes default qualification criteria across multiple enterprises. + +### Enforcement Mechanics + +- **Procurement Force:** No `CACert` + no proof package = no award. +- **Audit Force:** Any artifact or verification failure = formal non-conformance and remediation trigger. +- **Commercial Force:** Repeated failure impacts vendor tiering, renewals, and expansion eligibility. +- **Regulatory Force:** Proof-ready artifacts reduce supervisory friction; absence elevates risk rating and scrutiny.