diff --git a/AGENTS.md b/AGENTS.md index d85b67b6..59aafb1a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,7 +5,7 @@ **Author**: Paul Calnon **License**: MIT License **Version**: 0.5.0 -**Last Updated**: 2026-06-30 +**Last Updated**: 2026-07-04 --- @@ -260,7 +260,7 @@ For any new `prometheus_client` `Counter` / `Gauge` / `Histogram` / `Summary` / - `register_info_or_update(name, description, **labels)` — sugar for the `Info` two-step register-then-`.info({...})` pattern. - `lazy_register_or_reuse(...)` — for the lazy-init-with-`None`-sentinel pattern. -Tests touching these collectors should use `juniper_observability.testing.reset_prometheus_registry`. Existing examples in this repo: `src/observability.py:_ensure_canopy_metrics`, `src/main.py` (browser WS metrics), `src/adapter_validation.py`, `src/frontend/dashboard_manager.py`. See [the design doc in juniper-ml](https://github.com/pcalnon/juniper-ml/blob/main/notes/observability/REGISTER_OR_REUSE_HELPER_DESIGN_2026-05-05.md) for the rationale. +Tests touching these collectors should use `juniper_observability.testing.reset_prometheus_registry`. Existing examples in this repo: `src/observability.py:_ensure_canopy_metrics`, `src/main.py` (browser WS metrics), `src/adapter_validation.py`, `src/frontend/dashboard_manager.py`. See [the design doc in juniper-ml](https://github.com/pcalnon/juniper-ml/blob/main/notes/observability/JUNIPER_2026-05-05_JUNIPER-ML_REGISTER-OR-REUSE-HELPER-DESIGN.md) for the rationale. ### CI/CD diff --git a/CHANGELOG.md b/CHANGELOG.md index 702634e6..32abf8fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -248,7 +248,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 compatible model), exercised via an injectable `models=` param. The **optional search box (§5.2) is deferred** — it is a FR12 scale affordance with no value at the current two-model population. Design-of-record: juniper-ml - [`notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md) + [`notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-06-17_JUNIPER-CANOPY_MODEL-DATASET-SELECTION-DESIGN.md) (§5.3 / §5.8). Tests: `src/tests/unit/test_model_registry.py` (`dataset_model_hint`) + `src/tests/regression/test_model_table.py` (degenerate recovery, hint handler/seed, callback wiring). - **Recurrence model goes live + D8 Train-gating (A1-iv-5, #368)**: the `recurrence` (LMU) model is @@ -275,7 +275,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 gains a `search=` parameter. This is the §5.2 scale affordance (browse-and-compare at dozens-to-hundreds of model variants); it has no functional effect at today's two models but completes the surface. Design-of-record: juniper-ml - [`notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md) + [`notes/JUNIPER_CANOPY_MODEL_DATASET_SELECTION_DESIGN_2026-06-17.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-06-17_JUNIPER-CANOPY_MODEL-DATASET-SELECTION-DESIGN.md) (§5.2). Tests: `model_matches_search` (`test_model_registry.py`) + search filtering / no-match message / open-honours-search + search-rebuilds-keeping-open (`test_model_table.py`). **This completes the A1 model-selection feature end-to-end.** @@ -292,7 +292,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 (Prometheus `juniper_canopy_build` Info metric) and the shared `ReadinessResponse`. Foundation for the ecosystem stale-image-detection effort — see juniper-ml - [`notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md). + [`notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-06-14_JUNIPER-ECOSYSTEM_BUILD-PROVENANCE-DESIGN.md). Requires `juniper-observability>=0.4.0`. - **STATUS BAR — show cascor `completion_reason` (converged vs stalled) on a completed run (Issue #3 diagnosability follow-up, consumes cascor #320)**: a finished training run rendered a bare "Completed" regardless of *why* growth stopped, so a genuine convergence was indistinguishable from a 0-unit stall. cascor #320 now emits a `completion_reason` on `/v1/training/status`; this wires it through canopy end-to-end. `ServiceBackend.get_status` (`src/backend/service_backend.py`) carries the top-level `completion_reason` into the flat `StatusResult` (mirroring the existing `pending_dataset` pass-through; `StatusResult` in `src/backend/protocol.py` gains the field), and `_build_unified_status_bar_content` (`src/frontend/dashboard_manager.py`) appends a short label to the status when `status == "Completed"` via a new `_completion_reason_label` helper: `residual_collapsed`/`below_threshold` → **"Completed — converged"**, `no_candidate` → **"Completed — stalled (0 new units)"**, `early_stopped` → **"Completed — early stopped"**, `max_iterations` → **"Completed — max iterations"**. Display-only (the status color still keys off the base "Completed"); an unknown or missing reason yields a plain "Completed", so a canopy talking to a cascor that predates #320 degrades gracefully. Regression coverage: `src/tests/unit/frontend/test_completion_reason_status_bar.py` (label mapping + the five completed-run suffixes + not-completed / unknown / missing cases) and two `test_service_backend.py` cases (carry-through + `None` when absent). @@ -367,7 +367,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **SEC-F19 / D4 WS cap rollback — globally rejected sockets no longer leak reserved per-IP/per-session slots**: the endpoints reserve the per-IP/session counters before awaiting `WebSocketManager.connect()`, but the new stack-absolute global cap can reject later inside `connect()` when `max_connections` is already full. That rejection path closes the socket before it enters `active_connections`, so the endpoint's normal `disconnect()` cleanup no-ops and the reserved counters stayed inflated, letting repeated over-global attempts strand a browser session/IP at its cap until process restart. `connect()` now returns whether it actually registered the socket, and all WS endpoints release the reserved cap slots when registration is rejected or fails before activation. Regression coverage: `src/tests/unit/test_ws_connection_caps.py::TestGlobalConnectionCap::test_global_cap_rejection_releases_reserved_session_slots`. -- **STALE-VERSION SHADOW — `.dockerignore` now excludes *nested* `**/*.egg-info/` so the image stops reporting a stale package version**: `importlib.metadata.version("juniper-canopy")` (the source of `APP_VERSION` → `/v1/health` `version`, the Prometheus `juniper_canopy_build` metric, and the Sentry release) resolved to **0.4.0** while `pyproject.toml` was **0.5.0**. Root cause: a stale, git-untracked `src/juniper_canopy.egg-info` build artifact was COPYed into the image by the Dockerfile's `COPY src/ ./src/`; with `ENV PYTHONPATH=/app/src` ahead of site-packages, `importlib.metadata` resolved that egg-info's `PKG-INFO` (0.4.0) instead of the freshly-installed `juniper_canopy-0.5.0.dist-info`. The existing `.dockerignore` carried `*.egg-info/`, but that pattern only matches the **context root** and silently missed the nested `src/*.egg-info`. Fix: add the `**/`-prefixed `**/*.egg-info/` and `**/*.dist-info/` forms so nested build-metadata dirs are excluded from the build context at any depth (verified against a context containing `src/X.egg-info` — excluded, while real source is kept). Surfaced by the build-provenance `make doctor` work (juniper-ml [`notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md)): doctor correctly reported the image **FRESH** (`git_sha` == source HEAD) while the version string lied — exactly why `git_sha` is the reliable staleness signal. Takes effect on the next canopy image rebuild. Regression coverage: `src/tests/regression/test_dockerignore_egg_info.py` pins the nested-exclusion patterns. +- **STALE-VERSION SHADOW — `.dockerignore` now excludes *nested* `**/*.egg-info/` so the image stops reporting a stale package version**: `importlib.metadata.version("juniper-canopy")` (the source of `APP_VERSION` → `/v1/health` `version`, the Prometheus `juniper_canopy_build` metric, and the Sentry release) resolved to **0.4.0** while `pyproject.toml` was **0.5.0**. Root cause: a stale, git-untracked `src/juniper_canopy.egg-info` build artifact was COPYed into the image by the Dockerfile's `COPY src/ ./src/`; with `ENV PYTHONPATH=/app/src` ahead of site-packages, `importlib.metadata` resolved that egg-info's `PKG-INFO` (0.4.0) instead of the freshly-installed `juniper_canopy-0.5.0.dist-info`. The existing `.dockerignore` carried `*.egg-info/`, but that pattern only matches the **context root** and silently missed the nested `src/*.egg-info`. Fix: add the `**/`-prefixed `**/*.egg-info/` and `**/*.dist-info/` forms so nested build-metadata dirs are excluded from the build context at any depth (verified against a context containing `src/X.egg-info` — excluded, while real source is kept). Surfaced by the build-provenance `make doctor` work (juniper-ml [`notes/BUILD_PROVENANCE_DESIGN_2026-06-14.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-06-14_JUNIPER-ECOSYSTEM_BUILD-PROVENANCE-DESIGN.md)): doctor correctly reported the image **FRESH** (`git_sha` == source HEAD) while the version string lied — exactly why `git_sha` is the reliable staleness signal. Takes effect on the next canopy image rebuild. Regression coverage: `src/tests/regression/test_dockerignore_egg_info.py` pins the nested-exclusion patterns. - **TRAINING-CONTROL ERROR SURFACING — a rejected Start/Pause/Stop/Resume/Reset now shows a danger alert instead of silently bouncing the button (the "dead button" class)**: clicking a training-control button that the backend then rejected produced **no** user-visible feedback — the button flipped to its optimistic "pending" state and silently re-enabled, with the failure (and its reason) reaching only a server log or the browser console. This was the canopy half of the cascor dual-path #319 incident: a 401/502 live dataset swap (cascor#331) and an FSM-rejected Start (409) were both invisible from the dashboard. Two transports were silent: (1) the **production-default clientside WS path** (`PHASE_D_TRAINING_BUTTONS_CLIENTSIDE_JS`, gated `enable_ws_control_buttons=True`) returned `success: true` *synchronously* and resolved the real WS/REST outcome only to `console.warn`; (2) the server-side handler (`_handle_training_buttons_handler`) computed `success: False` into `training-control-action` but **nothing consumed it** and the response body (the reason) was discarded. Fix — one shared outcome surface fed by both transports: a new fixed-position `training-control-outcome-alert` div (offset below `live-switch-outcome-alert`) is filled by a single unconditionally-registered render callback (`_surface_training_control_outcome_handler`) that renders a dismissable `dbc.Alert(color="danger", duration=8000)` naming the command and the reason on failure, and clears on success. The server-side handler now captures the rejection detail via a new `_extract_training_error_detail` helper (prefers cascor's structured `{"error": {"message": …}}` body — which cascor#332 made specific, e.g. "Training cannot be started: Training data not provided" — then raw text, then the exception string; never raises) and stores `{success, command, detail}`. The clientside JS pushes the **real** async outcome into the same store via `dash_clientside.set_props('training-control-action', …)` (the established Phase D §S10 pattern) from the REST-fallback failure branches, so WS rejections, WS-down→REST, and pure-REST failures all surface. All edits are additive on failure branches that previously dead-ended in the console; the success path, optimistic button state, debounce, and timeout sweeper are untouched. Design + root cause: [`notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md`](notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md). Regression coverage: `test_dashboard_manager.py` (handler failure now carries `command`+`detail`; `_extract_training_error_detail` JSON / text / bare-exception / never-raises; `_surface_training_control_outcome_handler` clear-on-success / render-danger-on-failure / fallback-detail) and `test_phase_d_button_clientside.py` (JS contains the `set_props` reporting wired into the REST fallback; render callback registered under both transport flags). **Live-verification gate:** the `set_props`→render round-trip is not Python-unit-testable, so this must be confirmed on the deployed stack (force a Start the FSM will reject; confirm the red alert appears with the cascor reason) before merge. **Deferred (noted in the design doc):** short-circuiting the REST double-send on a *definitive* WS command rejection, and an optional green success confirmation. - **WS-CONTROL PONG — `/ws/control` accepts an inbound heartbeat pong instead of erroring (closes the WS-KEEPALIVE latent note)**: the `/ws/control` receive loop (`src/main.py`) handled inbound `{"type": "ping"}` (replies with a pong) but had no branch for `{"type": "pong"}`. A pong frame carries no `command` key, so it fell through to the command dispatch as `command == ""` and the endpoint replied `Unknown command: ` (`code="unknown_command"`). Dormant today — the server Phase-F heartbeat only pings `/ws/training`, never `/ws/control` (the WS-KEEPALIVE entry below scoped it to `channel="training"` *specifically* to avoid this misfire) — but it would trip the moment the heartbeat is extended to the control channel, or any client sends an unsolicited pong. The loop now treats `{"type": "pong"}` as a silent no-op (debug-logged), mirroring `/ws/training`, which already ignores non-ping frames. Removes the blocker noted in WS-KEEPALIVE so a future control-channel heartbeat is safe. Regression coverage in `test_websocket_control.py::TestWebSocketControlIntegration::test_control_pong_is_noop_not_unknown_command` (send a pong then a valid `start`; the next command response is the `start` success — proving the pong produced no `ok: False` error). @@ -381,7 +381,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security -- **SEC-F22 / D2 — startup loopback bind-guard (the loopback bind is now an enforced invariant)**: canopy's browser training-control gate (`/api/train/*`, `/ws/control`) authenticates the same-origin browser by `Origin` + CSRF, both of which are forgeable by an in-network **non-browser** client (spoofable `Origin`, anonymously-mintable CSRF token — audit HO-6), so the **only** effective control is the loopback bind. That bind was an implicit default, not an enforced invariant — flipping `BIND_HOST=0.0.0.0` silently made the control surface in-network- (or internet-) reachable. A new startup guard (`src/security.py`: `is_loopback_host` / `enforce_loopback_bind_guard` / `NonLoopbackBindError`, called from `main.lifespan`, mirroring the E-8 `enforce_dependency_floors` fail-loud idiom) now **refuses to start** (CRITICAL log + raise; fail-closed) when `settings.server.host` (`JUNIPER_CANOPY_SERVER__HOST`) is a non-loopback interface (anything not in `127.0.0.0/8`, `::1`, or `localhost`) **unless** the new `settings.fronting_auth_attested` (`JUNIPER_CANOPY_FRONTING_AUTH_ATTESTED`, default `False`) is `True` — an operator attestation that a fronting authenticating proxy is present (the attested non-loopback path logs a loud WARNING). Loopback binds (the default) start normally, so this is zero-UX for the shipped posture. Implemented **inline in canopy** (no new dependency). Regression coverage: `src/tests/unit/test_bind_guard.py`. Design-of-record: juniper-ml [`notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md) §4 / §8 (D2); implementation note: [`notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md`](notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md). +- **SEC-F22 / D2 — startup loopback bind-guard (the loopback bind is now an enforced invariant)**: canopy's browser training-control gate (`/api/train/*`, `/ws/control`) authenticates the same-origin browser by `Origin` + CSRF, both of which are forgeable by an in-network **non-browser** client (spoofable `Origin`, anonymously-mintable CSRF token — audit HO-6), so the **only** effective control is the loopback bind. That bind was an implicit default, not an enforced invariant — flipping `BIND_HOST=0.0.0.0` silently made the control surface in-network- (or internet-) reachable. A new startup guard (`src/security.py`: `is_loopback_host` / `enforce_loopback_bind_guard` / `NonLoopbackBindError`, called from `main.lifespan`, mirroring the E-8 `enforce_dependency_floors` fail-loud idiom) now **refuses to start** (CRITICAL log + raise; fail-closed) when `settings.server.host` (`JUNIPER_CANOPY_SERVER__HOST`) is a non-loopback interface (anything not in `127.0.0.0/8`, `::1`, or `localhost`) **unless** the new `settings.fronting_auth_attested` (`JUNIPER_CANOPY_FRONTING_AUTH_ATTESTED`, default `False`) is `True` — an operator attestation that a fronting authenticating proxy is present (the attested non-loopback path logs a loud WARNING). Loopback binds (the default) start normally, so this is zero-UX for the shipped posture. Implemented **inline in canopy** (no new dependency). Regression coverage: `src/tests/unit/test_bind_guard.py`. Design-of-record: juniper-ml [`notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-07-03_JUNIPER-CANOPY_CONTROL-SURFACE-AUTH-AND-NAT-DESIGN.md) §4 / §8 (D2); implementation note: [`notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md`](notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md). - **SEC-F19 / D4 — global + per-session WebSocket connection caps (kills the shared-NAT self-DoS)**: Docker NAT collapses every WS client to the bridge-gateway IP (audit HO-3), so the existing per-IP cap (`max_connections_per_ip=5`) is shared across all users behind the gateway — one client's five sockets exhaust the cap for everyone (a live self-DoS). `src/communication/websocket_manager.py` now adds, alongside the per-IP cap: (a) the stack-absolute **global** cap `max_connections` (=50) enforced in `connect()` — the single admission choke point shared by `/ws/training`, `/ws/control`, `/ws` — rejecting the N+1th connection stack-wide with close code `1013`; and (b) a **per-session** cap `max_connections_per_session` (=5, new `WebSocketSettings` field) keyed on the anonymous `canopy_session` cookie read from the WS handshake, restoring per-client fairness where the per-IP cap is inert (one session can no longer starve another behind the same gateway). A cookieless first connection is allowed and left to the global cap as the backstop. The three endpoints call a new `check_connection_limits()` (per-IP then per-session, rolling back the per-IP slot on a per-session rejection so a rejected attempt can't leak the per-IP counter); each endpoint keeps its existing close-reason (`/ws/control` stays opaque per M-SEC-06). The per-IP cap is retained but re-scoped honestly (a code comment + this note): it is **inert behind NAT** — DoS-dampening, **not** authentication. Regression coverage: `src/tests/unit/test_ws_connection_caps.py`. Design-of-record §5 / §8 (D4). **Deferred (Phase 4, owner-gated, NOT in this PR):** X-Forwarded-For-from-trusted-proxy (D6) and a real dashboard login / fronting proxy (D7) — the only mechanisms that restore genuine per-client identity / close SEC-F22 for the remote/multi-user case. diff --git a/docs/ENVIRONMENT_SETUP.md b/docs/ENVIRONMENT_SETUP.md index 0efa59f0..12ec55d0 100644 --- a/docs/ENVIRONMENT_SETUP.md +++ b/docs/ENVIRONMENT_SETUP.md @@ -86,7 +86,7 @@ This guide covers complete environment setup for Juniper Canopy development and > Juniper conda environments are **versioned across rebuilds**. This project's > live environment is currently **`JuniperCanopy1`**; each full rebuild (per the > Juniper ecosystem conda-env rebuild procedure, -> `juniper-ml/notes/JUNIPER_CONDA_ENV_REBUILD_PROCEDURE.md`) creates a new +> `juniper-ml/notes/JUNIPER_2026-05-03_JUNIPER-ECOSYSTEM_CONDA-ENV-REBUILD-PROCEDURE.md`) creates a new > suffixed environment (`JuniperCanopy1` → `JuniperCanopy2` → …) and renames the > previous one to `*-DEPRECATED`. A `*-DEPRECATED` environment has a broken or > partial toolchain (e.g. a broken `torch` import) — **never activate one**. diff --git a/notes/FRONTEND_ISSUES_PLAN_2026-05-09.md b/notes/FRONTEND_ISSUES_PLAN_2026-05-09.md index 2fd6fce2..0cb9f1ee 100644 --- a/notes/FRONTEND_ISSUES_PLAN_2026-05-09.md +++ b/notes/FRONTEND_ISSUES_PLAN_2026-05-09.md @@ -3,7 +3,7 @@ * **Author**: Paul Calnon (drafted by Claude Code Opus 4.7) * **Status**: Reviewed — open questions resolved 2026-05-09 (see §10 Resolution log) * **Scope**: Six user-reported issues affecting the juniper-canopy Dash UI and its interaction with the juniper-cascor backend. -* **Companion pointer**: `juniper-ml/notes/canopy_frontend_issues_plan_2026-05-09.md` +* **Companion pointer**: `juniper-ml/notes/JUNIPER_2026-05-09_JUNIPER-CANOPY_FRONTEND-ISSUES-PLAN.md` ## Revisions diff --git a/notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md b/notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md index b094a00d..cd9725c9 100644 --- a/notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md +++ b/notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md @@ -123,7 +123,7 @@ But that handler runs only **after** the middleware admits the request. The midd | Doc | Relevance | | --- | --- | -| `juniper-ml/notes/observability/CANOPY_DASHBOARD_SELF_CALL_REFACTOR_2026-05-10.md` | **Most relevant.** Docs Opt-B (X-API-Key injection in **server-side** self-calls, shipped canopy#265) vs deferred Opt-C (in-process calls). Writes why **browser-originated** calls (this bug) were never covered. Cross-repo. | +| `juniper-ml/notes/observability/JUNIPER_2026-05-10_JUNIPER-CANOPY_DASHBOARD-SELF-CALL-REFACTOR.md` | **Most relevant.** Docs Opt-B (X-API-Key injection in **server-side** self-calls, shipped canopy#265) vs deferred Opt-C (in-process calls). Writes why **browser-originated** calls (this bug) were never covered. Cross-repo. | | `notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md` | Designs the "`{label} failed.`" danger-alert surfacing that renders this 401; both transports feed `training-control-action`. Explains the banner mechanism (not the auth cause). | | `notes/fixes/FIX_FRONTEND_REGRESSIONS_2026-05-30.md` | Frontend-regression remediation observed on the **deployed docker stack** (Bug-4 class, canopy↔cascor); same deploy context as this failure. | | `notes/CANOPY_RUNTIME_CLIENT_FLOOR_DRIFT_ROOT_CAUSE_2026-06-26.md` | Prior "green tests / dead app" root-cause writeup (client-wheel floor drift). Different root cause, but the same "tests pass, runtime breaks" methodology applies. | diff --git a/notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md b/notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md index 4ae9ed0f..d48b89a0 100644 --- a/notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md +++ b/notes/JUNIPER_CANOPY_CONTROL-SURFACE-HARDENING_SEC-F22-F19_NOTE_2026-07-04.md @@ -10,7 +10,7 @@ **Scope**: Records the two now-enforced control-surface invariants shipped in this PR and, explicitly, what remains deferred. > Design of record (read first): juniper-ml -> [`notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md) +> [`notes/JUNIPER_CANOPY_CONTROL_SURFACE_AUTH_AND_NAT_DESIGN_2026-07-03.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-07-03_JUNIPER-CANOPY_CONTROL-SURFACE-AUTH-AND-NAT-DESIGN.md) > — §4 (SEC-F22 Option A), §5 (SEC-F19 Option B), §7 Phases 1–2, §8 decisions D2 + D4, §9 testing. > Companion: [`JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md`](JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md) > §7.3 stated the load-bearing loopback precondition this PR now enforces. diff --git a/notes/JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md b/notes/JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md index ea5e90ea..09036f82 100644 --- a/notes/JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md +++ b/notes/JUNIPER_CANOPY_TRAINING-CONTROL-AUTH_DESIGN_2026-06-30.md @@ -57,7 +57,7 @@ banner via `_surface_training_control_outcome_handler` (`src/frontend/dashboard_ | C5 | **`/v1/*` and other non-browser API surfaces must remain key-gated.** External / programmatic clients can and do hold a key. | `/v1/*` is not exempt (`src/canopy_constants.py:355-367`); the audit confirms live `GET /v1/` → 401. | **Non-goals.** (a) Option C (in-process self-calls) from -`juniper-ml/notes/observability/CANOPY_DASHBOARD_SELF_CALL_REFACTOR_2026-05-10.md` +`juniper-ml/notes/observability/JUNIPER_2026-05-10_JUNIPER-CANOPY_DASHBOARD-SELF-CALL-REFACTOR.md` — it addresses the *server-side* self-call cost, not the *browser* path, and does not fix this 401. (b) Enabling the SEC-06 bearer subprotocol (`ws_auth_enabled`, `src/settings.py:341`) — it still requires the token *in the browser* (§4d). (c) Re-architecting the dashboard to a separate process. (d) Changing the secret's value or rotation. @@ -504,7 +504,7 @@ curl -sS -i http://localhost:8050/v1/health # expect 200 **Companion / cross-repo:** - [`notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md`](JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md) — the root-cause audit (H1/H1b, the toggle table, the call path). -- `juniper-ml/notes/observability/CANOPY_DASHBOARD_SELF_CALL_REFACTOR_2026-05-10.md` — Option B (server-side `X-API-Key` injection, shipped canopy#265) vs deferred Option C; §7 explicitly notes browser-side data fetching as out of scope, which is why browser-originated calls were never covered. +- `juniper-ml/notes/observability/JUNIPER_2026-05-10_JUNIPER-CANOPY_DASHBOARD-SELF-CALL-REFACTOR.md` — Option B (server-side `X-API-Key` injection, shipped canopy#265) vs deferred Option C; §7 explicitly notes browser-side data fetching as out of scope, which is why browser-originated calls were never covered. - [`notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md`](CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md) — the danger-alert surfacing that renders this 401 (both transports feed `training-control-action`). - juniper-deploy `/home/pcalnon/Development/python/Juniper/juniper-deploy/docker-compose.yml` — canopy service (`:545`), `CANOPY_API_KEY_FILE` (`:576`), loopback bind (`:557`), rate limit on (`:572`), `secrets: - canopy_api_key` (`:616-617`), secret def (`:932-933`); cascor WS origins (`:234,:384`, **not** canopy inbound). - juniper-deploy `/home/pcalnon/Development/python/Juniper/juniper-deploy/.env.secrets.enc:10` (encrypted `CANOPY_API_KEY`), `.env.secrets.example:13` (empty template), `./secrets/canopy_api_key.txt` (43 bytes). diff --git a/notes/development/CONSOLIDATED_DEVELOPMENT_HISTORY.md b/notes/development/CONSOLIDATED_DEVELOPMENT_HISTORY.md index 25a82004..11e6e9a7 100644 --- a/notes/development/CONSOLIDATED_DEVELOPMENT_HISTORY.md +++ b/notes/development/CONSOLIDATED_DEVELOPMENT_HISTORY.md @@ -401,7 +401,7 @@ Three files in `notes/development/` are broken symlinks whose targets no longer | File | Target | Status | |---------------------------------------|-----------------------------------------------------------------------------|-------------------| -| `DASHBOARD_AUGMENTATION_PLAN.md` | `../../../juniper-ml/notes/DASHBOARD_AUGMENTATION_PLAN.md` | ⚠️ Target missing | +| `DASHBOARD_AUGMENTATION_PLAN.md` | `../../../juniper-ml/notes/JUNIPER_2026-04-20_JUNIPER-CANOPY_DASHBOARD-AUGMENTATION-PLAN.md` | ⚠️ Target missing | | `DATASET_DISPLAY_FAILURE_ANALYSIS.md` | `../../../juniper-ml/notes/development/DATASET_DISPLAY_FAILURE_ANALYSIS.md` | ⚠️ Target missing | | `DATASET_DISPLAY_FIX_PLAN.md` | `../../../juniper-ml/notes/development/DATASET_DISPLAY_FIX_PLAN.md` | ⚠️ Target missing | diff --git a/notes/development/REPLAY_V2_FAQ.md b/notes/development/REPLAY_V2_FAQ.md index 458eee2e..190f9313 100644 --- a/notes/development/REPLAY_V2_FAQ.md +++ b/notes/development/REPLAY_V2_FAQ.md @@ -18,7 +18,7 @@ why decision-boundary playback "lags" the metric scrubber. For implementation roadmap and design rationale see the parent plan at -[`juniper-ml/notes/PHASE_6E_DEFERRED_CAN-015GH_DESIGN.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/PHASE_6E_DEFERRED_CAN-015GH_DESIGN.md). +[`juniper-ml/notes/JUNIPER_2026-05-04_JUNIPER-ECOSYSTEM_PHASE-6E-DEFERRED-CAN-015GH-DESIGN.md`](https://github.com/pcalnon/juniper-ml/blob/main/notes/JUNIPER_2026-05-04_JUNIPER-ECOSYSTEM_PHASE-6E-DEFERRED-CAN-015GH-DESIGN.md). --- diff --git a/notes/history/CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md b/notes/history/CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md index 821dfb8a..93213470 100644 --- a/notes/history/CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md +++ b/notes/history/CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md @@ -2,7 +2,7 @@ **Date**: 2026-04-12 **Version Reviewed**: 0.4.0 -**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/R5-01_canonical_development_plan.md) +**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_R5-01-CANONICAL-DEVELOPMENT-PLAN.md) **Supersedes**: [CODE_REVIEW_ANALYSIS_2026-04-04.md](CODE_REVIEW_ANALYSIS_2026-04-04.md) --- diff --git a/notes/history/CODE_REVIEW_AUDIT_PLAN_2026-04-12_R5-01-aligned.md b/notes/history/CODE_REVIEW_AUDIT_PLAN_2026-04-12_R5-01-aligned.md index 4b82a36b..06765419 100644 --- a/notes/history/CODE_REVIEW_AUDIT_PLAN_2026-04-12_R5-01-aligned.md +++ b/notes/history/CODE_REVIEW_AUDIT_PLAN_2026-04-12_R5-01-aligned.md @@ -2,7 +2,7 @@ **Date**: 2026-04-12 **Version**: 0.4.0 -**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/R5-01_canonical_development_plan.md) +**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_R5-01-CANONICAL-DEVELOPMENT-PLAN.md) **Supersedes**: [CODE_REVIEW_AUDIT_PLAN_2026-04-12.md](CODE_REVIEW_AUDIT_PLAN_2026-04-12.md) **Companion Documents**: diff --git a/notes/history/CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-12_R5-01-aligned.md b/notes/history/CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-12_R5-01-aligned.md index 90a92d91..6e289bcc 100644 --- a/notes/history/CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-12_R5-01-aligned.md +++ b/notes/history/CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-12_R5-01-aligned.md @@ -2,7 +2,7 @@ **Date**: 2026-04-12 **Version**: 0.4.0 -**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/R5-01_canonical_development_plan.md) +**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_R5-01-CANONICAL-DEVELOPMENT-PLAN.md) **Supersedes**: [CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-04.md](CODE_REVIEW_DEVELOPMENT_ROADMAP_2026-04-04.md) **Companion Documents**: @@ -154,7 +154,7 @@ Track PAR is otherwise complete. ## 4. R5-01 Canonical Phase Execution -This section summarizes the 11 R5-01 phases and their canopy impact. See [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/R5-01_canonical_development_plan.md) for full detail. +This section summarizes the 11 R5-01 phases and their canopy impact. See [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_R5-01-CANONICAL-DEVELOPMENT-PLAN.md) for full detail. ### Phase 0-cascor: Seq, Replay, Resume (2.0 days) diff --git a/notes/history/CODE_REVIEW_PLAN_2026-04-12_R5-01-aligned.md b/notes/history/CODE_REVIEW_PLAN_2026-04-12_R5-01-aligned.md index 82d1e372..f599b141 100644 --- a/notes/history/CODE_REVIEW_PLAN_2026-04-12_R5-01-aligned.md +++ b/notes/history/CODE_REVIEW_PLAN_2026-04-12_R5-01-aligned.md @@ -2,7 +2,7 @@ **Date**: 2026-04-12 **Version**: 0.4.0 -**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/R5-01_canonical_development_plan.md) +**Source of Truth**: [R5-01 Canonical Development Plan](../../juniper-ml/notes/interface_proposals/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_R5-01-CANONICAL-DEVELOPMENT-PLAN.md) **Companion**: [CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md](CODE_REVIEW_ANALYSIS_2026-04-12_R5-01-aligned.md) **Supersedes**: [CODE_REVIEW_PLAN_2026-04-04.md](CODE_REVIEW_PLAN_2026-04-04.md) diff --git a/prompts/thread-handoff_automated-prompts/HANDOFF_2026-06-29_canopy-cascor-training-401-apikey.md b/prompts/thread-handoff_automated-prompts/HANDOFF_2026-06-29_canopy-cascor-training-401-apikey.md index 31138e1c..306565cb 100644 --- a/prompts/thread-handoff_automated-prompts/HANDOFF_2026-06-29_canopy-cascor-training-401-apikey.md +++ b/prompts/thread-handoff_automated-prompts/HANDOFF_2026-06-29_canopy-cascor-training-401-apikey.md @@ -109,7 +109,7 @@ grep -n 'enable_ws_control_buttons' src/settings.py # = True (src/settings.py ## Reference docs - [`notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md`](../../notes/JUNIPER_CANOPY_CASCOR-TRAINING-401-APIKEY_AUDIT_2026-06-29.md) — **primary source**: full ranked root-cause, toggle table, call-path. -- `juniper-ml/notes/observability/CANOPY_DASHBOARD_SELF_CALL_REFACTOR_2026-05-10.md` (cross-repo) — **most relevant**: Option B (server-side `X-API-Key` injection, shipped canopy#265) vs deferred Option C; explains why **browser-originated** calls were never covered. +- `juniper-ml/notes/observability/JUNIPER_2026-05-10_JUNIPER-CANOPY_DASHBOARD-SELF-CALL-REFACTOR.md` (cross-repo) — **most relevant**: Option B (server-side `X-API-Key` injection, shipped canopy#265) vs deferred Option C; explains why **browser-originated** calls were never covered. - [`src/frontend/internal_api.py`](../../src/frontend/internal_api.py) (module docstring) — states the helper is for **server-side** self-calls only; points to the refactor doc above. - [`notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md`](../../notes/CANOPY_TRAINING_CONTROL_ERROR_SURFACING_DESIGN_2026-06-14.md) — designs the "`{label} failed.`" danger-alert that renders this 401 (the banner mechanism, not the auth cause). - [`notes/fixes/FIX_FRONTEND_REGRESSIONS_2026-05-30.md`](../../notes/fixes/FIX_FRONTEND_REGRESSIONS_2026-05-30.md) — frontend-regression remediation on the **deployed docker stack** (same deploy context). diff --git a/src/backend/demo_backend.py b/src/backend/demo_backend.py index a30be7fc..4b99d8bd 100644 --- a/src/backend/demo_backend.py +++ b/src/backend/demo_backend.py @@ -28,7 +28,7 @@ # ##################################################################################################################################################################################################### # References: -# - juniper-ml/notes/MICROSERVICES-ARCHITECTURE_DEVELOPMENT-ROADMAP.md §5.5 +# - juniper-ml/notes/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_MICROSERVICES-ARCHITECTURE-DEVELOPMENT-ROADMAP.md §5.5 # ##################################################################################################################################################################################################### # TODO : diff --git a/src/backend/protocol.py b/src/backend/protocol.py index db597e1e..5270cf08 100644 --- a/src/backend/protocol.py +++ b/src/backend/protocol.py @@ -26,12 +26,12 @@ ##################################################################################################################################################################################################### # Notes: # Phase 5 of the Microservices Architecture Development Roadmap. -# See: juniper-ml/notes/MICROSERVICES-ARCHITECTURE_DEVELOPMENT-ROADMAP.md +# See: juniper-ml/notes/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_MICROSERVICES-ARCHITECTURE-DEVELOPMENT-ROADMAP.md # ##################################################################################################################################################################################################### # References: # - PEP 544 — Protocols: Structural subtyping (static duck typing) -# - juniper-ml/notes/MICROSERVICES-ARCHITECTURE_DEVELOPMENT-ROADMAP.md §5.4 +# - juniper-ml/notes/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_MICROSERVICES-ARCHITECTURE-DEVELOPMENT-ROADMAP.md §5.4 # ##################################################################################################################################################################################################### # TODO : diff --git a/src/backend/service_backend.py b/src/backend/service_backend.py index b02ad58b..31a29b36 100644 --- a/src/backend/service_backend.py +++ b/src/backend/service_backend.py @@ -29,7 +29,7 @@ # ##################################################################################################################################################################################################### # References: -# - juniper-ml/notes/MICROSERVICES-ARCHITECTURE_DEVELOPMENT-ROADMAP.md §5.6 +# - juniper-ml/notes/JUNIPER_2026-04-20_JUNIPER-ECOSYSTEM_MICROSERVICES-ARCHITECTURE-DEVELOPMENT-ROADMAP.md §5.6 # ##################################################################################################################################################################################################### # TODO : diff --git a/src/demo_mode.py b/src/demo_mode.py index 7c2fb45f..1f88ac4c 100644 --- a/src/demo_mode.py +++ b/src/demo_mode.py @@ -598,7 +598,7 @@ def __init__(self, update_interval: float = None): # Phase 3 progress fields (mirror cascor TrainingState semantics so the # canopy progress UI displays non-zero values during demo training). - # See juniper-ml/notes/code-review/CANOPY_CASCOR_INTERFACE_ROADMAP_2026-04-08.md §5. + # See juniper-ml/notes/code-review/JUNIPER_2026-04-08_JUNIPER-ECOSYSTEM_CANOPY-CASCOR-INTERFACE-ROADMAP.md §5. self._best_correlation_state: float = 0.0 self._candidates_trained_count: int = 0 self._candidates_total_count: int = 0 diff --git a/src/frontend/internal_api.py b/src/frontend/internal_api.py index 7e5b52b8..a46cdabe 100644 --- a/src/frontend/internal_api.py +++ b/src/frontend/internal_api.py @@ -31,7 +31,7 @@ ``self._api_base_url`` URLs. Long-term direction (Option C, deferred — see -juniper-ml/notes/observability/CANOPY_DASHBOARD_SELF_CALL_REFACTOR_2026-05-10.md): +juniper-ml/notes/observability/JUNIPER_2026-05-10_JUNIPER-CANOPY_DASHBOARD-SELF-CALL-REFACTOR.md): the HTTP self-call indirection should be replaced with direct in-process function calls into the FastAPI route handlers, eliminating the serialize/deserialize round-trip and the metric-noise contribution. This diff --git a/src/tests/ui/test_l3_native_setter_poc.py b/src/tests/ui/test_l3_native_setter_poc.py index 0729c7e5..a1aa5f64 100644 --- a/src/tests/ui/test_l3_native_setter_poc.py +++ b/src/tests/ui/test_l3_native_setter_poc.py @@ -2,7 +2,7 @@ native-value-setter does NOT drive Dash's React-controlled ``dbc.Input(type=number)``. This is the reproducible artifact behind the audit doc's §5.3 / §6 claim -(`juniper-ml/notes/JUNIPER_CANOPY_AUDIT_REGRESSIONS_AND_MODEL_SELECTION_2026-06-15.md`). +(`juniper-ml/notes/JUNIPER_2026-06-15_JUNIPER-CANOPY_AUDIT-REGRESSIONS-AND-MODEL-SELECTION.md`). It is a **strict xfail**: the assertion that the set value reaches the backend is expected to FAIL (Apply pushes the *default*, not the set value), proving the POC #2 wall empirically and continuously. If a future Dash/dbc release fixes the diff --git a/src/tests/unit/test_demo_backend.py b/src/tests/unit/test_demo_backend.py index ef0230dc..06210ec6 100644 --- a/src/tests/unit/test_demo_backend.py +++ b/src/tests/unit/test_demo_backend.py @@ -312,7 +312,7 @@ class TestPhase4TypedContract: """Phase 4 / P5-RC-18: DemoBackend conforms to the same TypedDict contract as ServiceBackend. Field-presence assertions only — TypedDicts erase to plain dicts at runtime. - See juniper-ml/notes/code-review/CANOPY_CASCOR_INTERFACE_ROADMAP_2026-04-08.md §6.1. + See juniper-ml/notes/code-review/JUNIPER_2026-04-08_JUNIPER-ECOSYSTEM_CANOPY-CASCOR-INTERFACE-ROADMAP.md §6.1. """ def test_apply_params_returns_envelope(self, demo_backend): diff --git a/src/tests/unit/test_demo_mode_advanced.py b/src/tests/unit/test_demo_mode_advanced.py index 56a1a979..1a6d1264 100644 --- a/src/tests/unit/test_demo_mode_advanced.py +++ b/src/tests/unit/test_demo_mode_advanced.py @@ -243,7 +243,7 @@ def test_cascade_unit_addition(self): class TestPhase3ProgressFields: """Phase 3 metrics granularity: TrainingState progress fields are exposed by DemoMode so the canopy progress UI displays non-zero values during demo - training. See juniper-ml/notes/code-review/CANOPY_CASCOR_INTERFACE_ROADMAP_2026-04-08.md §5. + training. See juniper-ml/notes/code-review/JUNIPER_2026-04-08_JUNIPER-ECOSYSTEM_CANOPY-CASCOR-INTERFACE-ROADMAP.md §5. """ PHASE3_FIELDS = ( diff --git a/src/tests/unit/test_service_backend.py b/src/tests/unit/test_service_backend.py index 5048fe9f..ef3f6521 100644 --- a/src/tests/unit/test_service_backend.py +++ b/src/tests/unit/test_service_backend.py @@ -389,7 +389,7 @@ class TestPhase4TypedContract: """Phase 4 / P5-RC-18: BackendProtocol method returns conform to the declared TypedDict shapes. These are field-presence contract tests, not runtime type assertions — TypedDicts erase to plain dicts at runtime. - See juniper-ml/notes/code-review/CANOPY_CASCOR_INTERFACE_ROADMAP_2026-04-08.md §6.1. + See juniper-ml/notes/code-review/JUNIPER_2026-04-08_JUNIPER-ECOSYSTEM_CANOPY-CASCOR-INTERFACE-ROADMAP.md §6.1. """ def test_start_training_envelope_keys(self, service_backend):