Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 227 additions & 0 deletions docs/standards/CAC_Procurement_Audit_Standard_v1.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
# CAC Procurement and Audit Standard v1.0

## 1) STANDARD DOCUMENT (RFC-STYLE)

### 1.1 Abstract

This standard defines mandatory controls for Cognitive Admissibility Criteria (CAC) in procurement, audit, and regulatory assurance workflows. A decision system is conformant only when each governed decision outcome is accompanied by verifiable evidence artifacts, a signed admissibility certificate (`CACert`), a reproducible decision trace, and an inclusion proof in an append-only transparency log.

### 1.2 Scope

This standard applies to any system that produces, ranks, recommends, filters, or automates decisions used in:

- regulated business workflows;
- high-impact operational decisions;
- customer-, employee-, or citizen-affecting determinations; and
- supplier-delivered AI/analytics modules accepted through enterprise procurement.

Out of scope: purely informational systems with no decision effect and no downstream control action.

### 1.3 Definitions

- **Decision System**: Software or pipeline that emits or materially influences a decision output.
- **Evidence Bundle**: Immutable artifact set containing inputs, model/runtime metadata, policy version identifiers, execution timestamps, and integrity digests.
- **Decision Trace**: Machine-readable lineage from input evidence to output decision, including policy evaluations and execution path.
- **Admissibility Verdict**: Binary verdict (`PASS` or `FAIL`) computed by policy-controlled CAC verification.
- **CACert**: Cryptographically signed certificate binding verdict, artifact digests, policy versions, issuer identity, and validity window.
- **Transparency Log**: Append-only Merkle-based log storing signed statement entries and enabling inclusion/consistency proofs.
- **Verifier**: Independent implementation (CLI/API) that validates signatures, digests, policy bindings, and transparency proofs.

### 1.4 Normative Language

The key words **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, and **MAY** are to be interpreted as normative requirements.

### 1.5 Normative Requirements

#### 1.5.1 Evidence Requirements

1. Each governed decision output **MUST** generate an Evidence Bundle with unique identifier and immutable digest set.
2. Evidence Bundle **MUST** include: input references, transformation lineage, model/version identifiers, policy bundle hash, execution environment metadata, timestamp, and actor/process identity.
3. Evidence Bundle **MUST** be retained for the policy-defined retention period and **MUST** be retrievable by auditors via deterministic identifier.
4. Missing required Evidence Bundle fields **MUST** produce verdict `FAIL`.

#### 1.5.2 Traceability Requirements

1. Each decision **MUST** produce a Decision Trace that links every output claim to source evidence and policy evaluation result.
2. Trace records **MUST** include ordered execution steps and **MUST** support deterministic replay against the same artifact set.
3. Any trace discontinuity, unverifiable step, or non-deterministic replay **MUST** produce verdict `FAIL`.

#### 1.5.3 Admissibility Verdict Requirements

1. Each governed decision **MUST** be evaluated by CAC policy controls before deployment, release, or externally consumed output publication.
2. Verdict **MUST** be binary (`PASS` or `FAIL`) with no indeterminate production state.
3. Systems **MUST NOT** promote or deploy artifacts with `FAIL` verdict.
4. Verdict computation inputs **MUST** include Evidence Bundle digests, Decision Trace digest, policy version, and verification result set.

#### 1.5.4 Certification (CACert) Requirements

1. Every `PASS` verdict **MUST** produce a signed `CACert`.
2. `CACert` **MUST** bind: system identifier, decision/evaluation identifier, verdict, artifact digests, policy versions, issuer identity, issuance timestamp, expiry timestamp, and signature.
3. `CACert` **MUST** be machine-verifiable without vendor-proprietary tooling dependencies.
4. Expired, revoked, unverifiable, or mismatched `CACert` **MUST** be treated as non-conformant (`FAIL`).

### 1.6 Verification Requirements

1. Conformant deployments **MUST** provide independent verification via at least one API and one CLI execution path.
2. Verifier **MUST** validate:
- cryptographic signature integrity;
- digest equality between artifacts and certificate;
- policy/version binding;
- Decision Trace integrity; and
- transparency inclusion and consistency proofs.
3. Verification outcome **MUST** be reproducible by third parties with documented commands and fixed input artifact references.
4. Any verification error **MUST** produce verdict `FAIL`.

### 1.7 Transparency Log Requirements

1. Each issued `CACert` **MUST** be logged in an append-only Merkle transparency log.
2. Logged entry **MUST** include certificate digest, issuer identity, timestamp, and subject identifier.
3. System **MUST** provide inclusion proof for each logged certificate and consistency proof across checkpoints.
4. Log entries **MUST NOT** be deleted or rewritten.
5. Missing inclusion proof or failed consistency proof **MUST** produce verdict `FAIL`.

---

## 2) PROCUREMENT LANGUAGE (RFP READY)

1. Vendor **MUST** provide a verifiable `CACert` for each production decision workflow in scope of this procurement.
2. Vendor **MUST** provide independent verification via CLI and API, with documented commands enabling buyer-side validation of `CACert`, evidence digests, and transparency proofs.
3. All in-scope outputs **MUST** pass CAC admissibility checks before release to production or customer-facing consumption.
4. Vendor **MUST NOT** classify any failed admissibility result as “warning,” “degraded,” or “acceptable risk” for production use.
5. Vendor **MUST** provide a machine-readable Evidence Bundle for every sampled decision specified by the buyer during evaluation and audit.
6. Evidence Bundle **MUST** include immutable identifiers, timestamps, policy version hashes, model/runtime identifiers, and complete artifact digests.
7. Vendor **MUST** provide Decision Trace artifacts that enable deterministic replay of sampled decisions without undocumented vendor-only tooling.
8. Vendor **MUST** maintain an append-only transparency log for all issued `CACert` records and provide inclusion/consistency proofs on request.
9. Vendor **MUST** disclose certificate issuer keys, trust chain, revocation endpoint, and certificate validity policy.
10. Vendor **MUST** fail closed: any missing, revoked, expired, or unverifiable `CACert` is an automatic non-conformance.
11. Vendor **MUST** support third-party auditor re-verification of sampled decisions within five business days of request.
12. Vendor **MUST** retain admissibility artifacts for a minimum of the buyer-defined retention period or regulatory minimum, whichever is longer.
13. Vendor **MUST** notify buyer within 24 hours of any control failure affecting admissibility enforcement, certificate issuance integrity, or transparency log integrity.
14. Buyer **MAY** require periodic blind-sample verification; vendor failure rate above buyer threshold constitutes material breach.
15. Contract acceptance **MUST** be conditioned on successful demonstration of CAC pass/fail determinism and third-party reproducible verification.

---

## 3) AUDIT FRAMEWORK

### 3.1 Audit Checklist

1. Confirm system inventory and identify all in-scope decision workflows.
2. For each sampled workflow, obtain: Evidence Bundle, Decision Trace, Admissibility Verdict, `CACert`, transparency inclusion proof, and consistency proof.
3. Validate certificate signature, issuer trust, expiry, and revocation state.
4. Recompute artifact digests and compare against `CACert` bindings.
5. Re-run verifier (CLI and/or API) on sampled artifacts.
6. Validate deterministic replay of sampled Decision Traces.
7. Confirm failed verdicts are blocked from production pathways.
8. Confirm retention, retrieval latency, and chain-of-custody controls for admissibility artifacts.

### 3.2 Required Artifacts

- System scope register for governed decisions.
- Evidence Bundle files for each audit sample.
- Decision Trace files for each audit sample.
- Admissibility verdict records (PASS/FAIL) with policy version reference.
- Signed `CACert` artifacts and issuer trust material.
- Transparency log checkpoints, inclusion proofs, and consistency proofs.
- Verification run logs (tool version, command, timestamp, result).
- Control exception register and remediation evidence.

### 3.3 Verification Procedures

1. Select statistically valid sample set and any regulator-mandated targeted samples.
2. Execute independent verifier using auditor-controlled environment.
3. Validate digest integrity end-to-end from evidence through certificate.
4. Validate transparency proof against published checkpoint.
5. Conduct deterministic replay and compare verdict and output determinism.
6. Record all results in immutable audit workpapers.

### 3.4 Pass/Fail Conditions

**PASS** requires all of the following:

- 100% sampled decisions have complete required artifacts.
- 100% sampled `CACert` signatures validate and are unexpired/unrevoked.
- 100% sampled inclusion/consistency proofs validate.
- 100% sampled deterministic replay checks match recorded admissibility outcomes.
- 0 production decisions lack a `PASS` admissibility verdict.

**FAIL** is triggered by any one of:

- Missing artifact required by Section 3.2.
- Unverifiable or mismatched certificate/artifact digest.
- Failed transparency proof.
- Non-deterministic replay without approved governed exception.
- Evidence of production release despite `FAIL` verdict.

---

## 4) COMPLIANCE MAPPING

### 4.1 NIST AI RMF Alignment

- **Govern (GV):** CAC defines accountable control gates via binary admissibility verdicts and certifiable evidence.
- **Map (MP):** CAC requires scoped decision inventory and explicit artifact lineage.
- **Measure (ME):** CAC mandates machine-verifiable proofs, deterministic replay, and measurable pass/fail outcomes.
- **Manage (MG):** CAC enforces fail-closed deployment blocking and remediation loops.

**Gap CAC Fills:** Converts AI risk guidance into hard acceptance controls with cryptographic evidence and independent verifier reproducibility.

### 4.2 ISO/IEC 42001 Alignment

- Aligns with AI management-system requirements for traceability, control operation, monitoring, and continual assurance through auditable artifacts.
- Supports objective evidence expectations for conformity assessment and internal/external audits.

**Gap CAC Fills:** Adds concrete admissibility primitives (`CACert`, transparency proofs, deterministic replay) where management-system language is process-oriented but not artifact-specific.

### 4.3 SOC 2 Alignment (Relevant Controls)

- **CC1/CC2 (Control environment, communication):** Defined ownership and policy-bound admissibility controls.
- **CC6 (Logical access):** Controlled issuance and validation paths for certificates and evidence access.
- **CC7 (Change management/monitoring):** CI enforcement and verification logs for every governed decision release.
- **CC8 (Risk mitigation):** Fail-closed gating for inadmissible outputs.
- **A1.2/A1.3 (Availability processing integrity, where applicable):** Deterministic replay and artifact integrity checks support processing integrity assertions.

**Gap CAC Fills:** Provides testable, decision-level evidence of processing integrity beyond narrative control descriptions.

### 4.4 EU AI Act Alignment

- Supports transparency/accountability obligations through traceable decision artifacts and verifiable logs.
- Supports high-risk system governance by requiring pre-deployment admissibility pass/fail enforcement.
- Supports post-market monitoring through retained evidence and independent re-verification capability.

**Gap CAC Fills:** Supplies operationally testable, cryptographically bound proof objects for conformity and supervisory review, reducing reliance on self-attested documentation.

---

## 5) ADOPTION STRATEGY (CRITICAL)

### Phase 0–90 Days: Procurement Insertion and Baseline Qualification

1. Publish this standard as an official procurement annex and mandatory evaluation schedule.
2. Insert Section 2 clauses into all new AI/decision-system RFP templates.
3. Require bidders to submit sample `CACert`, evidence bundle, and transparency proof package during pre-award evaluation.
4. Define disqualification rule: inability to produce independently verifiable artifacts equals non-responsive bid.
5. Train procurement, security, and internal audit teams on pass/fail evaluation script and artifact validation workflow.

### Phase 90–180 Days: Audit Operationalization and Contractual Enforcement

1. Amend active vendor contracts at renewal/addendum to require CAC conformance for in-scope workflows.
2. Launch third-party audit procedures in Section 3 with quarterly sample verification.
3. Establish supplier scorecard metric: “Admissibility Verification Success Rate” (target 100%).
4. Enforce remediation SLA for any fail trigger (e.g., immediate block + corrective action evidence).
5. Publish internal governance policy requiring `CACert` proof for promotion approvals in regulated workloads.

### Phase 180+ Days: Market Conditioning and Category Lock-In

1. Make CAC conformance a non-negotiable prerequisite in all strategic sourcing categories involving AI decision outputs.
2. Require platform integrators and prime contractors to flow down CAC clauses to subcontractors.
3. Share anonymized audit outcomes with risk committees and regulators to normalize CAC as accepted assurance evidence.
4. Tie preferred-vendor status to sustained zero-fail audit outcomes and transparent proof availability.
5. Expand buyer consortium language (industry groups/standard appendices) so CAC verification becomes default qualification criteria across multiple enterprises.

### Enforcement Mechanics

- **Procurement Force:** No `CACert` + no proof package = no award.
- **Audit Force:** Any artifact or verification failure = formal non-conformance and remediation trigger.
- **Commercial Force:** Repeated failure impacts vendor tiering, renewals, and expansion eligibility.
- **Regulatory Force:** Proof-ready artifacts reduce supervisory friction; absence elevates risk rating and scrutiny.
Loading