Fix scrambled-molecule docking bug + populate Layer 1.7 blind validation

BENCHMARKS.md

@@ -37,21 +37,38 @@ python -u tests/<path>/<test>.py    # or cellsim <subcommand> …
 
 ## 1.3 Docking: pose-recovery on bundled cocrystals
 
-Re-docking gate — canonical Astex/PDBBind convention (top-1 ≤ 2.5 Å
-**AND** best-of-top-3 < 2.0 Å).
-
-| System | ligand | top-1 RMSD | top-3-best | status | notes |
-|---|---|:-:|:-:|:-:|---|
-| 1STP streptavidin | biotin | 2.02 Å | 1.99 Å | ✅ PASS | canonical benchmark since AutoDock 1.0 |
-| 3PTB trypsin | benzamidine | 1.30 Å | 1.30 Å | ✅ PASS | classic S1-pocket test |
-| 1M17 EGFR kinase | erlotinib | 4.23 Å | 4.23 Å | ❌ FAIL | Vina scoring-function failure; known hard kinase case |
-
-**Aggregate: 2/3 = 67 %** at canonical gate.
-Vina papers report ~75–85 % on Astex Diverse Set (85 systems) at
-exhaustiveness=32; our 3-cocrystal set is deliberately harder
-(includes a known failure case — erlotinib).
-
-Reproducer: `python tests/dock/test_mini_bench.py`.
+> **CORRECTION (2026-07):** the per-system RMSDs previously tabulated
+> here (1STP 2.02 Å, 1M17 4.23 Å, …) were computed on molecules
+> *scrambled* by a Meeko atom-order bug in the pose reconstruction —
+> `_build_pose_mol` / `pose_rmsd` assumed Meeko preserves SMILES atom
+> order, which it does not. The bug is fixed (docked poses are now
+> reconstructed via Meeko's reverse conversion) and the numbers below
+> are recomputed on correct molecules. It made good poses look bad —
+> e.g. biotin/1STP went from a reported 2.02 Å to the real **0.69 Å**.
+
+**Blind pose recovery — 15 diverse cocrystals**, structures + ligand
+SMILES fetched from RCSB at run time (not hard-coded, so not circular
+on our own inputs). Reproducer:
+`python scripts/run_blind_dock_bench.py benchmarks/pdbbind/blind_set.yaml`.
+
+| Metric | Result | Campaign-1 gate |
+|---|:-:|:-:|
+| PoseBusters physical validity (#3) | **100 %** (15/15) | ✅ ≥ 95 % |
+| Pose recovery, top-1 ≤ 2 Å (#1) | **73 %** (11/15) | ⚠️ ≥ 75 % |
+| Pose recovery, top-3 ≤ 2 Å | **87 %** (13/15) | — |
+
+The top-1/top-3 gap is the headline: Vina's **sampling** finds the
+correct pose 87 % of the time, but its **scoring** ranks it #1 only
+73 %. The remaining top-1 misses are two ranking failures (1M17, 1HPX —
+correct pose at rank 2–3) and two genuine sampling near-misses (1FKG,
+1DWD, top-3 best 2.26 / 2.34 Å). A UFF-strain re-rank does *not* recover
+the ranking failures (the correct pose is not lower-strain), so top-1 is
+a Vina-scoring limitation — the same root as the kinase-ranking result
+below, which is what FEP addresses.
+
+The legacy 3-cocrystal re-dock smoke (`tests/dock/test_mini_bench.py`)
+still runs as an API-regression gate; the 15-cocrystal blind set above
+is the real accuracy measurement.
 
 ---
 

README.md

@@ -68,12 +68,12 @@ omit `--center` / `--box` and CellSim auto-detects pockets via fpocket
 |---|---|---|
 | **1.1 Chem** | SMILES → OpenFF-parametrised system (AM1-BCC charges) | ✅ 9/10 full tier, 10/10 RDKit tier |
 | **1.2 MD** | Classical Langevin MD, solvated protein loader (AMBER14 + TIP3P) | ✅ 1 ps ubiquitin Cα RMSD 0.74 Å |
-| **1.3 Docking** | Vina + Meeko + PoseBusters + fpocket auto-site | ✅ mini-bench 2/3 canonical gate |
+| **1.3 Docking** | Vina + Meeko + PoseBusters + fpocket auto-site | ✅ blind 15-cocrystal: top-3 87 %, PoseBusters 100 %; top-1 73 % |
 | **1.3 FEP** | Alchemical ΔG_hyd + ΔG_bind (DDM); `cellsim fep-binding {dg,ddg,bench,validate}` + `fep-report` + `bench-all` | ✅ **Milestone A PASS** (FreeSolv-12 MAE 1.42 kcal/mol on `milestone-a-pilot-3`, Pearson r +0.913); pipeline complete end-to-end; 107+ smoke tests; Milestone B binding pending GPU |
 | **1.4 Quantum** | xTB GFN2 single-point + CYP3A4 SoM predictor (BDE) | ✅ 10/10 sane + 3/3 SoM smoke |
 | **1.5 Coarse-grained** | Martini 3 membrane / bilayer MD | ⏳ scaffold only |
 | **1.6 UQ** | Monte-Carlo / Sobol / split-conformal for ΔG bounds | ✅ triad shipped |
-| **1.7 Blind harness** | PDBBind scale gate + red-team slot | ⏳ 3-cocrystal mini-bench shipped; PDBBind scale pending |
+| **1.7 Blind harness** | RCSB-fetched blind pose recovery + PoseBusters | ✅ 15-cocrystal blind set; PoseBusters #3 PASS (100 %); CASF/full-PDBBind scale pending |
 | **x-cut cache** | SQLite physics-prior memoisation (AM1-BCC, Vina, xTB) | ✅ shipped; wired into Layers 1.1 / 1.3 / 1.4 |
 
 **Cross-cutting UX:**
@@ -168,8 +168,14 @@ per-row reproducers.
 
 Headline numbers today:
 
-- **3-cocrystal mini-bench:** 2/3 = 67 % pose recovery at canonical
-  top-3 < 2 Å gate.
+- **Blind pose recovery (15 diverse cocrystals, RCSB-fetched):**
+  top-1 ≤ 2 Å = **73 %**, top-3 ≤ 2 Å = **87 %**, PoseBusters physical
+  validity = **100 %**. The top-1/top-3 gap is Vina's *scoring* (it
+  ranks the correct pose #1 only 73 % of the time) vs its *sampling*
+  (it generates the correct pose 87 % of the time). Reproduce:
+  `python scripts/run_blind_dock_bench.py benchmarks/pdbbind/blind_set.yaml`.
+  (Earlier "2/3 mini-bench" numbers were computed on molecules
+  scrambled by a Meeko atom-order bug, now fixed — see BENCHMARKS.md.)
 - **Streptavidin calibration:** Spearman ρ = **1.00** across
   14 orders of magnitude of K_d (but Pearson r = 0.98 hides the
   fact that Vina's absolute ΔG saturates on tight binders — MAE

benchmarks/pdbbind/.gitignore

@@ -0,0 +1 @@
+structures/

benchmarks/pdbbind/blind_set.yaml

@@ -0,0 +1,21 @@
+# Layer 1.7 blind pose-recovery + PoseBusters validation set.
+# Diverse, standard re-docking cocrystals across protein families.
+# SMILES + coordinates are fetched from RCSB at run time (never
+# hard-coded), so this benchmark is not circular on our own inputs.
+# `resname` is given only where auto-pick could grab a cofactor.
+entries:
+  - {pdb: 1stp, resname: BTN, name: streptavidin_biotin}
+  - {pdb: 3ptb, resname: BEN, name: trypsin_benzamidine}
+  - {pdb: 1m17, resname: AQ4, name: egfr_erlotinib}
+  - {pdb: 1hvr, resname: XK2, name: hivprotease_xk263}
+  - {pdb: 2hyy, resname: STI, name: abl_imatinib}
+  - {pdb: 3ert, resname: OHT, name: er_hydroxytamoxifen}
+  - {pdb: 1err, resname: RAL, name: er_raloxifene}
+  - {pdb: 1oyt, name: thrombin}
+  - {pdb: 1fkg, name: fkbp12}
+  - {pdb: 1ke5, name: cdk2}
+  - {pdb: 1cbx, name: carboxypeptidase_a}
+  - {pdb: 1dwd, name: thrombin_napap}
+  - {pdb: 1hpx, name: hivprotease_kni272}
+  - {pdb: 1e1v, name: cdk2_inhibitor}
+  - {pdb: 1lpz, name: factor_xa}

benchmarks/pdbbind/blind_set_results.csv

@@ -0,0 +1,16 @@
+pdb,name,resname,smiles,n_lig_heavy,dG_kcalmol,top1_rmsd_A,top3_best_rmsd_A,posebusters_ok,wall_s,error
+1stp,streptavidin_biotin,BTN,C1[C@H]2[C@@H]([C@@H](S1)CCCCC(=O)O)NC(=O)N2,16,-7.4,0.69,0.69,True,9.7,
+3ptb,trypsin_benzamidine,BEN,[H]/N=C(\c1ccccc1)/N,9,-6.02,1.13,1.1,True,4.9,
+1m17,egfr_erlotinib,AQ4,COCCOc1cc2c(cc1OCCOC)ncnc2Nc3cccc(c3)C#C,29,-7.25,2.61,1.71,True,19.3,
+1hvr,hivprotease_xk263,XK2,c1ccc(cc1)C[C@@H]2[C@@H]([C@H]([C@H](N(C(=O)N2Cc3ccc4ccccc4c3)Cc5ccc6ccccc6c5)Cc7ccccc7)O)O,46,-11.63,0.74,0.74,True,137.2,
+2hyy,abl_imatinib,STI,Cc1ccc(cc1Nc2nccc(n2)c3cccnc3)NC(=O)c4ccc(cc4)CN5CCN(CC5)C,37,-11.89,0.8,0.8,True,42.3,
+3ert,er_hydroxytamoxifen,OHT,CC/C(=C(\c1ccc(cc1)O)/c2ccc(cc2)OCCN(C)C)/c3ccccc3,29,-9.74,1.08,1.0,True,20.8,
+1err,er_raloxifene,RAL,c1cc(ccc1c2c(c3ccc(cc3s2)O)C(=O)c4ccc(cc4)OCCN5CCCCC5)O,34,-10.85,0.6,0.6,True,27.8,
+1oyt,thrombin,FSN,c1cc(ccc1CN2C(=O)[C@H]3[C@@H]4CCC[N@@]4[C@H]([C@H]3C2=O)c5ccc(cc5)C(=[NH2+])N)F,30,-10.92,0.26,0.26,True,14.2,
+1fkg,fkbp12,SB3,CCC(C)(C)C(=O)C(=O)N1CCCC[C@H]1C(=O)OC(CCc2ccccc2)c3ccccc3,33,-7.96,2.52,2.26,True,31.3,
+1ke5,cdk2,LS1,CNS(=O)(=O)c1ccc(cc1)N\C=C/2\c3ccccc3NC2=O,23,-9.45,1.19,0.32,True,10.1,
+1cbx,carboxypeptidase_a,BZS,c1ccc(cc1)C[C@H](CC(=O)O)C(=O)O,15,-7.51,0.32,0.32,True,10.5,
+1dwd,thrombin_napap,MID,[H]/N=C(/c1ccc(cc1)C[C@H](C(=O)N2CCCCC2)NC(=O)CNS(=O)(=O)c3ccc4ccccc4c3)\N,37,-9.96,3.12,2.34,True,37.1,
+1hpx,hivprotease_kni272,KNI,CC(C)(C)NC(=O)[C@@H]1CSCN1C(=O)[C@H]([C@H](Cc2ccccc2)NC(=O)[C@H](CSC)NC(=O)COc3cccc4c3ccnc4)O,46,-9.3,3.0,1.61,True,94.6,
+1e1v,cdk2_inhibitor,CMG,c1[nH]c2c(n1)c(nc(n2)N)OCC3CCCCC3,18,-7.55,1.03,1.03,True,9.0,
+1lpz,factor_xa,CMB,Cc1cccc2c1cc(n2Cc3cccc(c3)C(=N)N)C(=O)NCc4cc(cc(c4)Cl)Cl,32,-10.29,0.77,0.77,True,22.6,

benchmarks/pdbbind/blind_set_results.summary.json

@@ -0,0 +1,11 @@
+{
+  "n_entries": 15,
+  "n_scored": 15,
+  "n_errored": 0,
+  "pose_recovery_top1_frac": 0.7333333333333333,
+  "pose_recovery_top3_frac": 0.8666666666666667,
+  "pose_recovery_gate": 0.75,
+  "posebusters_frac": 1.0,
+  "posebusters_gate": 0.95,
+  "errored_pdbs": []
+}

docs/campaign1_status.md

@@ -9,13 +9,13 @@ the realistic timeline is to Stage 1 completion on CPU-only.
 |---|---|---|---|
 | **1.1 Chem foundation** (RDKit + OpenFF + AM1-BCC) | ✅ DONE | 10k ChEMBL round-trip, ADMET + AM1-BCC cache, profile dashboard | — |
 | **1.2 Classical MD** (OpenMM + ff14SB + TIP3P) | ✅ DONE (MVP) | 1 ps ubiquitin Cα RMSD 0.74 Å; 100 ns ubiquitin gate awaits GPU | full 100 ns gate (GPU) |
-| **1.3a Docking** (Vina + Meeko + PoseBusters + fpocket) | ✅ DONE | mini-bench 2/3 canonical re-dock; off-target panel; CYP3A4 SoM; strain-gate; triage rules + viewer | — |
+| **1.3a Docking** (Vina + Meeko + PoseBusters + fpocket) | ✅ DONE (with caveat) | blind 15-cocrystal: PoseBusters 100 % (#3 PASS), pose recovery top-3 87 % / top-1 73 % (#1 marginal — Vina *scoring*, not sampling); off-target panel; CYP3A4 SoM; strain-gate; triage rules + viewer. NB fixed a pose-reconstruction bug that had corrupted all prior RMSD/PoseBusters numbers. | lift top-1 via FEP ranking or re-scope #1 to top-3 |
 | **1.3b FEP Milestone A** (hydration ΔG) | ✅ **PASS** (today) | FreeSolv-12 MAE **1.42 kcal/mol** (gate ≤ 1.5), Pearson r +0.913, GHMC 99%, tag `milestone-a-pilot-3` | 12/12 closure on 2 polar fails (structural, doc'd; not blocking) |
 | **1.3c FEP Milestone B** (binding ΔG) | ⏳ in flight | streptavidin bench RUNNING on CPU (PID 75132, started 2026-05-30 22:15, ETA ~20 h); EGFR queued | both benches → fep-report → tarball |
 | **1.4 Quantum** (xTB GFN2 + PySCF DFT) | ✅ DONE (MVP) | 10/10 sanity + 3/3 CYP3A4 SoM; 2/3 on literature validation (aspirin ✓, midazolam ✓, diazepam doc'd-fail) | optional: extend literature set to 20 |
 | **1.5 Coarse-grained Martini 3** | ⏳ scaffold only | `src/cg/{bilayer,protein_cg}.py` are NotImplementedError stubs (~90 LOC total) | full bilayer builder + martinize2 wrapper + OpenMM-Martini MD driver + area-per-lipid validation |
 | **1.6 UQ** (Sobol + MC + MAPIE) | ✅ DONE | MC-dock seeds, Sobol sensitivity, conformal quantiles, streptavidin/trypsin/EGFR calibration bundles | — |
-| **1.7 Blind harness** (PDBBind / CASF / PoseBusters / ChEMBL) | ⏳ partial | PoseBusters integrated, fpocket integrated, 3-cocrystal mini-bench shipped; `benchmarks/{pdbbind,casf}/` are empty dirs | PDBBind 500-compound CPU subset (full 5k needs GPU); CASF-2016 ranking gate |
+| **1.7 Blind harness** (PDBBind / CASF / PoseBusters / ChEMBL) | ⏳ partial | `scripts/run_blind_dock_bench.py` fetches structures + ligand SMILES from RCSB and scores pose recovery + PoseBusters on a 15-cocrystal blind set (`benchmarks/pdbbind/blind_set.yaml`). #3 PoseBusters PASSES (100 %); #1 pose recovery measured honestly (top-1 73 %, top-3 87 %). | scale the blind set (30–50); CASF-2016 ranking gate |
 
 ## CPU-only timeline to Stage 1 complete