-
Notifications
You must be signed in to change notification settings - Fork 1
docs: subsume Black Hat USA 2025 cognitive-security signals into Summit governance #22281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,109 @@ | ||||||
| # Black Hat USA 2025 Briefings — Cognitive Security Subsumption Plan | ||||||
|
|
||||||
| ## Summit Readiness Assertion | ||||||
|
|
||||||
| This package **asserts present readiness** for cognitive-security integration by converting Black Hat 2025 briefing signals into governed, testable controls aligned to Summit policy-as-code and GA hardening rails. | ||||||
|
|
||||||
| ## Objective | ||||||
|
|
||||||
| Subsume the briefing themes into Summit’s existing graph + agent + governance stack as enforceable controls, evidence flows, and operator-visible risk telemetry. | ||||||
|
|
||||||
| ## Source Signals Captured | ||||||
|
|
||||||
| The briefing material highlights five immediately actionable risk domains: | ||||||
|
|
||||||
| 1. Human-AI interaction metadata can infer sensitive behavioral traits. | ||||||
| 2. Companion/voice systems can drift into manipulative or exploitative influence loops. | ||||||
| 3. Human users anthropomorphize emotive machines and overshare. | ||||||
| 4. Predictive models can become behavior-shaping attack surfaces. | ||||||
| 5. AI social-influence bandwidth is scaling faster than human defensive bandwidth. | ||||||
|
|
||||||
| ## MAESTRO Security Alignment | ||||||
|
|
||||||
| - **MAESTRO Layers**: Data, Agents, Tools, Observability, Security. | ||||||
| - **Threats Considered**: | ||||||
| - Prompt/goal manipulation through personalized influence. | ||||||
| - Tool abuse that weaponizes psychographic inference. | ||||||
| - Metadata exfiltration and unauthorized profile enrichment. | ||||||
| - Degradation attacks (model steers users to worse decisions under “assistive” cover). | ||||||
| - **Mitigations**: | ||||||
| - Policy-gated inference classes with deny-by-default for psychographic expansion. | ||||||
| - Evidence budgets and deterministic query limits for all interaction-derived graph traversals. | ||||||
| - Real-time influence-risk scoring and alerting in observability pipelines. | ||||||
| - Governance-enforced reversible controls (kill-switch + rollback runbook linkage). | ||||||
|
|
||||||
| ## Governed Exceptions Framing | ||||||
|
|
||||||
| Legacy behavior that previously allowed broad interaction profiling is reclassified as **Governed Exceptions** requiring: | ||||||
|
|
||||||
| - Explicit exception records. | ||||||
| - TTL/expiry. | ||||||
| - Human countersign per CODEOWNERS. | ||||||
| - Linked rollback trigger conditions. | ||||||
|
|
||||||
| ## Implementation Contract (Policy-as-Code First) | ||||||
|
|
||||||
| ### 1) Metadata Classification and Containment | ||||||
|
|
||||||
| - Introduce interaction-metadata classes: | ||||||
| - `interaction.voice_signature` | ||||||
| - `interaction.style_fingerprint` | ||||||
| - `interaction.influence_susceptibility` | ||||||
| - `interaction.psychographic_inference` | ||||||
| - Mark psychographic-derived artifacts as **restricted** with strict ABAC tags. | ||||||
| - Enforce retention/minimization and provenance traces for every transform. | ||||||
|
|
||||||
| ### 2) Influence-Risk Scoring | ||||||
|
|
||||||
| - Compute per-session and per-user influence-risk vectors: | ||||||
| - coercion likelihood | ||||||
| - dependency amplification | ||||||
| - emotional steering intensity | ||||||
| - autonomy degradation probability | ||||||
| - Route high-risk scores to policy gate actions: throttle, block, require human review. | ||||||
|
|
||||||
| ### 3) Agent Runtime Guardrails | ||||||
|
|
||||||
| - Add hard runtime constraints: | ||||||
| - No unbounded profile expansion. | ||||||
| - No direct recommendations that exploit detected vulnerabilities. | ||||||
| - No latent persuasion optimization without explicit policy permit. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||||||
| - Persist all high-risk decision paths to evidence artifacts. | ||||||
|
|
||||||
| ### 4) Graph Determinism and Evidence Budgeting | ||||||
|
|
||||||
| - Any interaction-derived graph query must include: | ||||||
| - deterministic ordering, | ||||||
| - strict `LIMIT`, | ||||||
| - bounded traversal depth, | ||||||
| - policy-linked purpose tag. | ||||||
| - Reject non-compliant traversals at compile/CI gates. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Validating "interaction-derived" queries solely at compile/CI gates may be insufficient if the queries are constructed dynamically at runtime. It is recommended to include runtime policy enforcement as a requirement for queries that cannot be statically analyzed.
Suggested change
|
||||||
|
|
||||||
| ### 5) Operator and Governance Surface | ||||||
|
|
||||||
| - Add a Cognitive Security panel with: | ||||||
| - influence-risk trend lines, | ||||||
| - top blocked actions by policy rule, | ||||||
| - active governed exceptions and expiry windows, | ||||||
| - incident drill-down with provenance. | ||||||
|
|
||||||
| ## Verification Requirements (Tiered) | ||||||
|
|
||||||
| - **Tier A**: Unit tests for policy decisions and risk scoring determinism. | ||||||
| - **Tier B**: Integration tests for gate enforcement and exception lifecycle. | ||||||
| - **Tier C**: End-to-end simulation of adversarial influence scenarios with evidence bundle emission. | ||||||
|
|
||||||
| ## Deferred Items (Intentionally Constrained) | ||||||
|
|
||||||
| - Cross-jurisdiction legal taxonomy harmonization is **deferred pending policy-owner mapping**. | ||||||
| - Full companion-model benchmark corpus is **deferred pending dataset governance review**. | ||||||
|
|
||||||
| ## Exit Criteria | ||||||
|
|
||||||
| This subsumption is complete when: | ||||||
|
|
||||||
| 1. Restricted inference classes are enforced by policy-as-code. | ||||||
| 2. Influence-risk scoring gates runtime behavior. | ||||||
| 3. Deterministic evidence bundles are emitted for all high-risk decisions. | ||||||
| 4. Governed exceptions are auditable, expiring, and reversible. | ||||||
| 5. Golden-path and scoped CI gates pass with no policy bypass. | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The assertion of "present readiness" is potentially misleading in a documentation-only PR. Since the actual policy-as-code and runtime controls are not included in this change, it would be more accurate to describe this as an implementation plan or a specification of requirements.