Critic agent for hypothesis generation: exclude 1-hop GWAS genes + dedup hypotheses

[HyunjunA]

Goal

Add a critic step to hypothesis generation that (1) prevents the model from leaning on already-known GWAS genes, and (2) guarantees no duplicate hypotheses are produced.

Tasks

• Exclude 1-hop GWAS hit genes from the model. Genes that are direct (1-hop) GWAS hits for the target disease must NOT be used as model features, so the pipeline does not just rediscover known associations and is pushed toward novel candidates.
• Critic check for duplicate hypotheses — remove duplicates. Implemented as a standalone public:hypothesis-dedup skill: exact match on normalized text + cosine similarity (GET /api/hypotheses/similar, threshold 0.85) against both already-stored and in-batch hypotheses; duplicates are dropped before storage. Wired into the hypothesize stage.

Notes

• Dedup currently runs as an agent self-check inside the hypothesize stage; it could be promoted to a dedicated critic stage later.

[mikeherv]

Push the logic (For the Critique agent, currently hard-coded - allow the user to have options to use the critique agent). Make it optional, default if OFF.

[mikeherv]

These should be in a PostEvaluation agent - (after Test and Validate -> Veredict (Critique agent) - then execute the 2 items below:
Currently hard-coded in the orchestrator (as the Analysis agent) :

• Assess the verdict_results.json and run api calls to the backend to update the hypothesis table database. (Currently hardcoded in the Orchestrator).
• Update the profile.md and insights.md with whatever important nuance was observed in the report.md from the job

[mikeherv]

Specific to Cedars, have it as part of the Command Line:
HPC skill for job submission and/or full deployment of ecoxai (5)

Be able to send jobs to the HPC and retrieve results. Will have to extract results by downloading all the assets using rsync.

Easiest way is probably to do a command line friendly EcoXAI. Just have ecoXAI and the dataset uploaded to your common folder, then run the ecoxai on it with the skills already altered. Probably needs a command line integration of downloading skills or management of workflow.

[HyunjunA]

Status — critic stage (on feature/critic-agent; main frozen for the paper)

What the critic does (domain-agnostic):

• Always applies the general memory gates: R1 (duplicate of an already-accepted hypothesis) and R3 (previously rejected).
• Rejected hypotheses are PATCHed to status=critic_rejected, and the applied gates are recorded in critique.yaml.
• It does not judge hypothesis quality (falsifiability, etc.) — it only filters for duplication and novelty.

Why R2 (known genes) is designed this way:

• The known-gene check is inherently domain-specific (AlzKB / Alzheimer's), so putting it inside the general critic would tie the critic to a single domain.
• Therefore R2's substance lives in a separate skill, alzkb-ad-genes (genes 1-hop associated with the disease in AlzKB via GENEASSOCIATESWITHDISEASE), kept outside the critic.
• At runtime the critic reads the task domain (DATASET_DOMAIN / context) and applies only the domain gates that match — it enables alzkb-ad-genes only for Alzheimer's / genomic tasks.
• This lets new domain gates be added as skills without modifying the critic, keeping the pipeline general rather than Alzheimer-specific.

Note on main: main is intentionally frozen for the existing EcoXAI bioinformatics application paper, so this critic work is kept on feature/critic-agent and is not merged into main.