feat(narrative-ci): add fixtures, OPA gates, deterministic pipeline scaffolding, and evidence bundles#19505
Conversation
BrianCLong
added
the
codex
ChatGPT Codex Connector
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request lays the foundational framework for a new 'Narrative CI' system, designed to ensure the integrity and reproducibility of narrative-related data. It achieves this by introducing a fixtures-first development approach, implementing robust policy enforcement through OPA gates, and providing a structured pipeline for processing and bundling evidence. The primary goal is to establish clear output contracts and prevent data inconsistencies, thereby enhancing the reliability of information for critical updates. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Ignored Files
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (49)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces significant scaffolding for a new Narrative CI pipeline, including OPA policies, data fixtures, deterministic pipeline steps, and evidence bundling. A critical logic error was identified in the traceability policy, specifically in the every_receipt_has_artifact rule, which uses an existential ('some') check instead of a universal ('every') check. This could allow provenance gaps to bypass the gate by only verifying that at least one receipt has an artifact, rather than ensuring all are complete. Additionally, the review identified a critical issue in the evidence bundling logic and several instances where hardcoded values should be externalized to configuration files.
| type EvidenceIndex = { version: number; evidence: Record<string, { files: string[] }> }; | ||
|
|
||
| function addEvidence(index: EvidenceIndex, id: string, files: string[]) { | ||
| index.evidence[id] = { files: [...files].sort() }; | ||
| } |
There was a problem hiding this comment.
The EvidenceIndex type and the addEvidence function do not match the actual structure of evidence/index.json. The JSON file defines evidence as an array of objects, where each object has an evidence_id and a files object (with keys like report, metrics). The current implementation incorrectly types evidence as a Record<string, { files: string[] }> and addEvidence passes an array of strings for files. This will result in a corrupted evidence/index.json file.
| every_receipt_has_artifact { | ||
| some r | ||
| r := input.receipts[_] | ||
| count(r.sources) > 0 | ||
| r.sources[0].artifact_id != "" | ||
| r.sources[0].content_sha256 != "" | ||
| } |
There was a problem hiding this comment.
The every_receipt_has_artifact rule in .github/policies/narrative_ci/traceability.rego is implemented using an existential ('some') check instead of a universal ('every') check. This logic error means the policy will pass if at least one receipt has an artifact, even if others have no sources, leading to traceability gaps. This defeats the purpose of the traceability gate, which aims to ensure all governed outputs have complete provenance. Furthermore, the current implementation could fail if no receipts are present, even when none are required. The rule should be rewritten to ensure all receipts are properly validated.
every_receipt_has_artifact {
not exists_receipt_without_artifact
}
exists_receipt_without_artifact {
some r
r := input.receipts[_]
not receipt_has_artifact(r)
}
receipt_has_artifact(r) {
count(r.sources) > 0
r.sources[0].artifact_id != ""
r.sources[0].content_sha256 != ""
}
| export function writeJsonDeterministic(path: string, obj: unknown) { | ||
| const dir = path.split("/").slice(0, -1).join("/"); | ||
| if (dir && !existsSync(dir)) { | ||
| mkdirSync(dir, { recursive: true }); | ||
| } | ||
| writeFileSync(path, `${stableStringify(obj)}\n`, "utf8"); | ||
| } | ||
|
|
||
| export function writeStampJson(path: string) { | ||
| const dir = path.split("/").slice(0, -1).join("/"); | ||
| if (dir && !existsSync(dir)) { | ||
| mkdirSync(dir, { recursive: true }); | ||
| } | ||
| // timestamps allowed ONLY here | ||
| writeFileSync(path, JSON.stringify({ ts_utc: new Date().toISOString() }) + "\n", "utf8"); | ||
| } |
There was a problem hiding this comment.
The logic for creating a directory is duplicated in writeJsonDeterministic and writeStampJson. This should be extracted into a shared helper function to avoid code duplication and improve maintainability. Additionally, using path.split('/') to get the directory name is not robust across different operating systems. It's better to use path.dirname() from Node's built-in path module.
| const tierJumpScore = Math.min(1, (rank(maxTier) - rank("fringe")) / 50); | ||
|
|
||
| const hedging = countMarkers(clm.text_norm, lex.register_markers?.hedging ?? []); | ||
| const legalistic = countMarkers(clm.text_norm, lex.register_markers?.legalistic ?? []); | ||
| const registerShiftScore = Math.min(1, (hedging + legalistic) / 6); | ||
|
|
||
| const citationCircularityScore = 0; | ||
|
|
||
| const score = Math.max( | ||
| 0, | ||
| Math.min(1, 0.45 * tierJumpScore + 0.35 * registerShiftScore + 0.2 * citationCircularityScore), | ||
| ); |
There was a problem hiding this comment.
This scoring logic contains several hardcoded 'magic numbers' (e.g., 50 on line 64, 6 on line 68, and the weights 0.45, 0.35, 0.2 on line 74). These should be extracted into a configuration file like intelgraph/pipelines/narrative_ci/config/thresholds.yml. This will make the scoring algorithm more configurable and easier to understand without reading the code.
| if (handoffScore >= 0.8) { | ||
| to_state = "Institutionalized"; | ||
| } else if (handoffScore >= 0.6 && from_state === "Seeded") { | ||
| to_state = "Contested"; | ||
| } |
There was a problem hiding this comment.
The score thresholds for state transitions (0.8 and 0.6) are hardcoded. The thresholds.yml config file already defines institutionalize_handoff_score: 0.80. This script should read the configuration file and use the values from there to avoid inconsistencies and improve configurability. A new threshold for contesting a narrative should also be added to the config.
|
|
||
| const new_claims = [...curClaims.keys()].filter((hash) => !prevClaims.has(hash)); | ||
| const removed_claims = [...prevClaims.keys()].filter((hash) => !curClaims.has(hash)); | ||
| const updated_claims: string[] = []; |
There was a problem hiding this comment.
The updated_claims array is initialized as empty and never populated. While this might be intentional for the initial scaffolding, it's worth noting that claim updates are not being tracked. If this is intended to be implemented later, a // TODO comment would be helpful to track this.
| if (candidate.score >= 0.7) { | ||
| indicators.push({ | ||
| indicator_id: `ew_${sha256Hex(`${candidate.narrative_id}:${candidate.to_tier}`).slice(0, 12)}`, | ||
| narrative_id: candidate.narrative_id, | ||
| kind: "tier_handoff_watch", | ||
| severity: candidate.score >= 0.85 ? "high" : "medium", | ||
| rationale: | ||
| "Handoff score exceeded threshold; monitor for institutional uptake and policy discussion artifacts.", | ||
| }); |
There was a problem hiding this comment.
The thresholds for creating an early warning indicator (0.7) and setting its severity (0.85) are hardcoded. The handoff.alert_score from thresholds.yml could be used for the 0.7 threshold. These values should be read from a configuration file to make them easier to manage and adjust.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Temporarily closing to reduce Actions queue saturation and unblock #22241. Reopen after the golden-main convergence PR merges. |
1 similar comment
|
Temporarily closing to reduce Actions queue saturation and unblock #22241. Reopen after the golden-main convergence PR merges. |
1 participant
Motivation
Description
.github/policies/narrative_ci/fordeterminism.rego,traceability.rego, andtier_taxonomy.regoplus corresponding pass/fail fixture JSONs.narrative-ci-verify.ymlthat runs OPA evaluations against fixtures and a determinism linter (checksout/**), and uploads the fixtures as artifacts.fixtures/feb07_2026/*), deterministic pipeline scaffolding underintelgraph/pipelines/narrative_ci/(libs:hash.ts,io.ts,json_stable.ts,schema_validate.stub.ts; steps:31_score_handoff.ts,40_state_machine.ts,50_bundle_evidence.ts,60_extract_delta.ts,70_early_warning.ts), and a README describing inputs/outputs.intelgraph/schema/narrative.graph.ymlandschemas/narrative/*.json), evidence bundles underevidence/EVD-SITUPDATE-2026-02-07-*/, and updateevidence/index.jsonanddocs/roadmap/STATUS.jsonto register the initiative.Testing
node scripts/check-boundaries.cjslocally to validate parallelization/boundary constraints; it completed with no violations.narrative-ci-verifythat will runopa evalagainst the added fixtures and perform the determinism linter in CI (the workflow is added but its run is executed by GitHub Actions).schema_validate.stub.tsplaceholder for later schema validation and is intentionally scaffolded for further automated tests.Codex Task