feat: 實現 IP Camera 多路即時串流監測與 UI 看板系統（乾淨版本） - #452
Open
andy371643 wants to merge 20 commits into
Open
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
This adds user-controlled quality markers for screen glare and occlusion, persists them to MongoDB, and wires the values through APIs and UI rendering so the labels remain consistent across reloads. Co-authored-by: Cursor <cursoragent@cursor.com>
…nt_with_newwidget
Introduces helper-dashboard/.claude/agents/anomaly-builder.md, a Claude Code subagent that runs a Code->Observe->Decide->Check loop to build proactive anomaly detection for the neonatal cerebral-oximetry (rSO2) monitor. The subagent writes the stateful evaluator, scheduler, webhook notifier, synthetic neonatal data source, and decision_flow widget directly. The in-app rescue_extend pipeline is structurally limited to six widget files and developer_fix is dev-token gated, so the build runs through this Claude subagent instead (no human developer path, per project decision). It encodes the clinical-safety invariants surfaced by this session's audit: - data-integrity gate first; fail closed on mock/stale Prometheus data (the data `source` flag is currently written but never read) - breach = value < 0.80 * avg_over_time(metric[24h]) sustained 5m - shadow mode by default; golden must-fire / must-not-fire as the gate - Discord webhook is a TEST TUNNEL only, never a clinical channel; URL from env var, never committed - full audit trail of agent reads/edits/decisions and Helper<->Big-guy comms - break-glass manual control surface, for recovery only Scoped to the subagent definition only; no runtime code built yet. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…en harness AlertRuleSpec (mock source rejected at validation, extra=forbid, reused _UNSAFE_PATTERN/FORBIDDEN_WIDGET_FIELDS/PromQL denylist) and AnomalyEvaluationReport/AlertEvent (pending/firing/resolved/signal_lost/ insufficient_baseline). Synthetic neonatal rSO2 generator with injectable scenarios + freshness + OCR-quality labels. Pure injected-clock evaluator core: data-integrity gate first (fail-closed on mock/unreachable/stale), baseline sufficiency, breach-duration state machine, shadow-by-default paging. Golden harness + 4 required traces all green (11 passed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e + heartbeat/watchdog Wrap the INC1 AnomalyEvaluatorCore in a runnable loop: - AnomalyEvaluatorService.tick(now): injectable-clock, one observation per rule, runs the pure core, persists durable state. FastAPI lifespan drives it on a real timer (opt-in via ANOMALY_EVALUATOR_ENABLED, shadow-only). - AlertStateStore: durable, fail-closed lifecycle store (breach_start + pending/firing/resolved/acked history); atomic write+fsync; clinical persist failure raises; corrupt load is surfaced, never reset to healthy. Breach timer is rehydrated on restart so the 5m clock is never silently reset. - DataProviders: SimDataProvider (deterministic, drives the whole loop) and PrometheusDataProvider that READS the client's source flag so the core fails closed on the silent mock fallback. - Heartbeat + watchdog: a stalled/crashed tick is detectable; source==mock, unreachable and stale are alarmable SIGNAL_LOST, not 'no anomaly'. - Integration suite steps several simulated ticks (injected time): full lifecycle normal->breach->firing@5m->resolved; mid-trace real->mock => signal_lost; restart rehydration; watchdog stall; shadow never pages. Rules remain SHADOW (record would-fire, never page — notifier is INC3). Gates green: tests/anomaly_golden + tests/spec_validation + tests/anomaly_service = 76 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… + dispatch policy Deliver alerts out of band without weakening any clinical-safety invariant: - anomaly_notifier.py: Notifier interface + DiscordWebhookNotifier (TEST TUNNEL only, reads ANOMALY_TEST_WEBHOOK_URL, urllib POST shell=False, no user argv, token masked via INC1 mask_url, at-least-once + ack + retries) + LocalFileReceiver fallback when the env var is absent. Every message carries the [TEST TUNNEL · NON-DIAGNOSTIC] banner + non-diagnostic disclaimer. - anomaly_notification_dispatch.py: policy layer. Shadow records would-fire only (never pages); promoted rule pages once per firing edge (dedup + hysteresis via resolve re-arm); SIGNAL_LOST raises its own distinct page; a display/severity filter can never silence a page (SuppressionRefused); every delivery attempt audited with the masked target. - anomaly_evaluator_service.py: optional dispatcher wired AFTER the clinical verdict is persisted; delivery never gates or delays the verdict. - tests_support/default_rules.py: promoted (mode=active) rule helper for the paging path (production promotion stays supervisor-gated). Gate: tests/anomaly_notification (15) asserts exactly one delivery on a real firing, zero in shadow, zero clinical fire on signal_lost, dedup+hysteresis, suppression refused, banner on every message, Discord adapter via injected opener (never touches the network) with token masked. Combined anomaly_golden + spec_validation + anomaly_service + anomaly_notification: 91 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… + multi-input + renderer/component)
Add the clinical decision-support flowchart widget in lockstep across
backend and frontend:
- widget_spec.py: new WidgetType.decision_flow; nested extra='forbid'
FlowNode/FlowEdge/FlowStep + DecisionFlowSpec; multi-input labeled
QueryInput over neonatal signals (rSO2/SpO2/HR/MAP/FiO2), prometheus-only
(source=mock rejected); model_validator enforces decision_flow config is
required for the type and forbidden on any other; first node MUST be the
data-integrity gate; edges/steps resolve; unknown signal/label rejected.
- widget_schema_doc.py: teach the type, its nodes/edges/steps/inputs config,
and the "gate first / real events / non-diagnostic" rules.
- frontend widget-toolkit/DecisionFlowWidget.tsx: reuse WidgetFrame, client-side
step state, reads REAL alert events from the INC2/INC3 store (never
AlertListWidget MOCK_ALERTS), holds the clinician on the gate until it
passes, surfaces SIGNAL_LOST / INSUFFICIENT_BASELINE / store-unreachable
loudly, persistent non-suppressible non-diagnostic disclaimer.
(frontend/lib/{spec-schema.ts,renderer.tsx,api.ts} edited on disk too but
frontend/lib is gitignored, so not committed.)
- tests/spec_validation/test_decision_flow.py: tier0 — validates with
nodes/edges/steps+multi-input; rejects raw_html/component/code smuggling,
source=mock input, non-gate first node, and config/type mismatch.
Evidence: test_decision_flow 23 passed; anomaly_golden+spec_validation 90
passed; related backend suites 135 passed; `npx tsc --noEmit` exit 0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…teness Break-glass (kill-switch/recovery hatch, only human-touchable surface, no routine maintenance): start/stop/restart evaluator+data source, force a rule into shadow (kill paging), force SIGNAL_LOST, recover from last-known-good. A stopped loop refuses to tick (never a fake no-anomaly); break-glass can only make things safer (never promotes to paging), and every action is audited with the operator identity, fail-closed. End-to-end wiring: the neonatal rSO2 rule (value < 0.80*avg_over_time[24h], for 5m) flows evaluator -> alert-state store -> notifier (TEST TUNNEL) -> decision_flow widget projection, with AnomalyEvaluationReport feeding the decide evidence. project_report reflects (never re-derives) the verdict; a degraded signal parks the flow on the loud signal_lost node with the persistent non-diagnostic banner. Audit completeness: new anomaly_lifecycle_audit stream captures rule activation, every state transition, delivery/would-fire, ack, suppressed, and MISSED must-fire (top-severity), fail-closed on clinical events. AlertStateStore gains last-known-good backup/restore for recovery. CHECK: 122 anomaly+spec+e2e tests green (incl. 8 new e2e); full suite 400 passed, 1 pre-existing unrelated Playwright failure. New rules remain SHADOW by default; source=mock still rejected at spec + decision_flow layers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin compatible HuggingFace versions and harden runtime handling so GPU extension loading uses the correct libraries while stream analysis no longer reports success when frame processing fails. Co-authored-by: Cursor <cursoragent@cursor.com>
…r race fix Per user directive 2026-07-06 (extends LD-1/LD-2 to the whole user path): - Review-loop rescue tickets no longer block the user: the validated draft is delivered with an honest caveat; the ticket persists as a diagnostic record only. All false "team has been notified" wording replaced with "logged a diagnostic". - New backend/app/helper/auto_fix.py: every persisted ticket schedules a gated background developer_fix run (injection check, daily quota, single-flight, snapshot+rollback, clinical protected-path byte guard, JSONL audit). bin/opencode gains tool-using mode for developer_fix with a render-layer-only write allow-list; clinical anomaly paths denied at both trust boundaries. - browser_evaluator: retry with a 25s wait on the all-widgets-missing / no-real-errors signature (the Jetson Next.js commit race that sank the rSO2 line-chart request). - CLAUDE.md (first tracked version): Big-guy row updated for the user-approved auto-fix path; SECURITY_BOUNDARIES §5b documents the boundaries; big-guy agent prompt updated for both modes. - tests: tests/auto_fix/ (gates, rollback, clinical guard), CLI allow-list tests, deliver-on-ticket + schedules-auto-fix regression tests; conftest autouse fixture keeps auto-fix off in tests unless opted in. Tier 0: 473 passed, 1 known-environmental failure (test_backend_down). Tier 2: 10/10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ons)
Pre-existing uncommitted work from earlier sessions, committed with
explicit user approval ("git everything", 2026-07-06). Working tree
verified green together with the auto-fix commit (Tier 0: 473 passed,
1 known-environmental failure; Tier 2: 10/10).
Includes: anomaly service/spec/test refinements, break-glass +
notification dispatch updates, neonatal publisher + its tests,
/api/anomaly route, and the first tracked version of the agent-ops
institution docs (LOCKED_DECISIONS, A_DIAGNOSIS, MAINTENANCE_PROTOCOL,
LESSONS, LETTER_TO_FUTURE_SESSIONS, DELEGATION_TEMPLATES).
Deliberately NOT committed: backend/app/storage/** runtime + audit
artifacts, docs/USER_SESSION_LOG.md (machine noise),
docs/SESSION_CHAT_HISTORY_2026-07-01.md and docs/agent-ops/backups/
(PII / masked webhook), frontend/tsconfig.tsbuildinfo (build
artifact), dashboard_store.py + test_draft_shadow.py (in-flight
background task).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…0 Prometheus New NON-CLINICAL rule domain (user request 2026-07-06): "alert me when CPU is above 90% for 5 minutes" in Helper chat now creates a validated system alert rule. Zero edits to the protected clinical files; the neonatal goldens are untouched (12/12). - specs/system_rule_spec.py: SystemAlertRuleSpec — curated metric catalog (cpu/disk-io/disk-usage/memory/load1 -> hardcoded PromQL templates; no user/LLM query text ever reaches Prometheus), absolute threshold + comparator, for 1m-1h, mode structurally Literal["shadow"], source locked to "prometheus". - services/system_rules.py: rule store, SystemMetricProvider (source- flag branching per CLAUDE.md; freshness via timestamp(node_time_ seconds)), SystemThresholdCore (gate-first state machine; never pages), SystemRuleService reusing a second AlertStateStore for durable state + append-only history. - api/system_rules.py + main.py: /api/system-rules read surface, validated CRUD, wall-clock tick loop (SYSTEM_RULES_ENABLED=0 kill switch; enabled-check inside the startup hook so pytest collection can never bake the loop in). - Chat wiring: AlertRuleIntent in user_message outputs; new author_alert_rule operation (runtime allow-list + bin/opencode heuristic + LLM contract + agent prompt docs); orchestrator validates via Python (authoritative), hot-adds the rule, and replies honestly about SHADOW + SIGNAL_LOST semantics. - tests/system_rules/: 19 tests — spec bounds/forced-shadow, gate- first core (mock/stale never fire), duration state machine, store round-trip, service durable state, provider mock-source honesty, chat end-to-end (mock runtime), API surface. Live-verified on the Jetson: real opencode/OpenRouter chat request created sys-cpu-above-90; evaluator ticks it and correctly reports SIGNAL_LOST(no_data) until node_exporter is fixed. Tier 0: 492 passed, 1 known-environmental failure. Tier 2: 10/10. Anomaly goldens: 12/12. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add manual-upload control and preprocessing to sanitize detected screen crops before VLM analysis, then persist both original and VLM-input image paths so the details view can compare what the model actually saw. Co-authored-by: Cursor <cursoragent@cursor.com>
Ensure database-page deletion removes related analysis files and safely purges stream_<session_id[:8]> directories for stream sources, and update the UI warning copy to reflect image file deletion. Co-authored-by: Cursor <cursoragent@cursor.com>
…rols Allow detail-page CSV exports to switch between raw and corrected output so manual corrections are preserved in downloads, and add an in-modal timeline window slider to keep dense histogram series readable. Co-authored-by: Cursor <cursoragent@cursor.com>
Add google/gemini-3.1-flash-lite to model selector lists and make it the default selection in details, upload, and stream monitor views for consistent OpenRouter defaults. Co-authored-by: Cursor <cursoragent@cursor.com>
Align downloaded medical-value CSV names with operator context by prioritizing task_name plus session_id, with session-only fallback and safe filename sanitization. Co-authored-by: Cursor <cursoragent@cursor.com>
Automatically remove source files in uploads after successful manual analysis while guarding path scope and skip-on-stop behavior to reduce disk growth safely. Co-authored-by: Cursor <cursoragent@cursor.com>
…heduling Add a gear-driven global interval setting persisted via API/MongoDB, include interval metadata in stream sessions/templates, and start first analysis at the next whole minute before continuing with fixed intervals. Co-authored-by: Cursor <cursoragent@cursor.com>
phreed
added a commit
to phreed/GroundingDINO
that referenced
this pull request
Aug 14, 2026
…suite Merges every pull request open at github.com/IDEA-Research/GroundingDINO/pulls, retaining only what belongs to the library, its documentation or its tests. Against upstream main 856dde2: 63 files changed, 37 added, 26 modified, none deleted. Upstream's own 67 files are all still present. Earlier states are preserved as tags: "pre-integration-backup" is upstream main untouched, "pre-squash" is the full 138-commit history with each PR a separate merge commit carrying its original authorship, and "full-integration" is this work with the PR IDEA-Research#452 application still included. ======================================================================= 1. PRs merged ======================================================================= 29 applied cleanly; 8 needed hand resolution. IDEA-Research#277 Bugfix/prompt ending IDEA-Research#279 Batched inference API and support for float16 inference IDEA-Research#295 Fix gradio demo IDEA-Research#299 Update inference.py IDEA-Research#313 Update inference.py IDEA-Research#319 Fix/bert uncased IDEA-Research#329 Edited docker_test.py file IDEA-Research#346 Quick fix to avoid error when not enough proposals post augmentation IDEA-Research#349 MPS Fix: masked_fill_ returning incorrect result on mac GPU IDEA-Research#357 Update docker_test.py IDEA-Research#371 Flexible requirements (opencv-python or opencv-python-headless) IDEA-Research#372 Update fixed README.md typo IDEA-Research#376 Single class train and coco format dataset loading IDEA-Research#380 onnx输出支持 (ONNX export support) IDEA-Research#381 Fix Dockerfile to ensure proper build IDEA-Research#383 Cuda 11.8 support IDEA-Research#393 make it compatible with torch 2.6.0 IDEA-Research#409 Add torch2.6 support for ms_deform_attn_cuda IDEA-Research#414 [fix] Change 'pyy' to 'py' IDEA-Research#415 Fix for pytorch 2.8 - Replace value.type() with value.scalar_type IDEA-Research#417 Bug Fixes for CUDA Custom Ops and timm Layer Import IDEA-Research#419 Add pyproject.toml IDEA-Research#423 make sure right_idx bigger than left_idx IDEA-Research#424 Support 5090 IDEA-Research#433 Fix typo in docker_test.py IDEA-Research#447 Fix logical error in true negative calculation IDEA-Research#449 Add ROCm inference support for MsDeformAttn IDEA-Research#452 IP Camera 多路即時串流監測與 UI 看板系統 IDEA-Research#453 [Fix] inference_on_a_image: UnboundLocalError text_prompt (IDEA-Research#390) IDEA-Research#455 fix: update BoxAnnotator API for supervision >=0.16 IDEA-Research#456 Fix load_model ignoring device parameter IDEA-Research#458 fix: restore compatibility with transformers >= 4.49 IDEA-Research#459 fix: include cuda_fp16.h in the MsDeformAttn CUDA kernel IDEA-Research#460 feat: expose __version__ on the groundingdino package IDEA-Research#461 feat: extend the SwinB config with training parameters IDEA-Research#462 fix: replace deprecated Tensor::type() so the CUDA kernel compiles IDEA-Research#463 chore: add uv-based local setup script PR IDEA-Research#452 contributed roughly 500 files: an IP-camera monitoring application and three dashboards, none of which GroundingDINO imports. It was merged, examined, then dropped as a consumer of the library rather than part of it. Its one vendored third-party component, a verbatim copy of github.com/grafana/mcp-grafana carrying that project's own go.mod, LICENSE, CODEOWNERS and CI, went with it; conda-forge packages it as mcp-grafana if it is ever wanted back. ======================================================================= 2. Conflict resolutions that changed the outcome ======================================================================= IDEA-Research#409, IDEA-Research#415 and IDEA-Research#462 are recorded merges with no tree change. Each re-does the Tensor::type() -> scalar_type() modernisation IDEA-Research#383 had already landed, in an older form (AT_ASSERTM, .data<T>(), non-half dispatch). IDEA-Research#380 replaced GroundingDINO.forward() with a signature taking pre-computed BERT output, which would have broken all four caption-based callers. Main already had a forward()/forward_nn() split, so the caption API was kept and forward_nn() gained an optional last_hidden_state: pass it to skip the internal BERT call, omit it for the previous behaviour. Its torch.amp.autocast(str(tgt.device)) was corrected to tgt.device.type, since 'cuda:0' is not a device type. Its global use_checkpoint=False was reverted, as that regresses training added by IDEA-Research#376 and IDEA-Research#461; load_model() instead gained a config_overrides argument. IDEA-Research#452 renamed groundingdino/ to groundingdino_source/ -- a local workaround per its own development log, contradicted by its own setup.py. Not carried over. Its LICENSE replacement was MediaMTX's MIT licence, Copyright (c) 2019 aler9, evidently pasted by accident; Apache-2.0 is kept. Its transformers==4.33.2 pin would have undone IDEA-Research#458. IDEA-Research#393, IDEA-Research#417, IDEA-Research#449 and IDEA-Research#279 resolved by keeping whichever side was newer, and unioning where both changes were wanted. ======================================================================= 3. Defects found and fixed ======================================================================= ExportFlag.is_export read off the class returns the property object, which is always truthy. The guard from IDEA-Research#277, carried into IDEA-Research#449's use_custom_kernel, therefore evaluated "not ExportFlag.is_export" as False every time, silently forcing every forward pass down the pure-PyTorch fallback even on CUDA. Now read through ExportFlag.current(). ExportFlag.__enter__ also stored the bound classmethod instead of calling it. APOPMeter.update() never recorded false negatives. PR IDEA-Research#447 corrected the mask to (pred==0, gt==1) but kept adding the result to self.tn, so fn was always zero and tn inflated. Completed here; the four counters now sum to the sample count. torch.amp.autocast was passed a device index rather than a device type (IDEA-Research#380). Two defects in PR IDEA-Research#452 code were fixed before that code was dropped, and are recorded here only because the same bugs may resurface if it is ever restored: get_class_mapping() did not skip blank lines, and convert_to_yolo_format() clamped centre and size independently so boxes could still leave the image. ======================================================================= 4. What is added ======================================================================= Inside the library: groundingdino/version.py IDEA-Research#460 groundingdino/util/export_flag.py IDEA-Research#277 groundingdino/config/GroundingDINO_SwinT_OGC_export.py IDEA-Research#380 groundingdino/models/.../ms_deform_attn_hip.h IDEA-Research#449, ROCm forward kernel groundingdino/models/.../ms_deform_attn_hip_forward.cu Alongside it: export_onnx/ IDEA-Research#277, IDEA-Research#380 ONNX and TensorRT export, plus test/cat.jpg. Note export_model.py and export.py are two independent exporters and are not reconciled. training/ IDEA-Research#376 single-class training and COCO loading requirements/ IDEA-Research#371 runtime/cv2/cv2-headless split that setup.py reads bin/ IDEA-Research#463 uv-based setup script demo/ IDEA-Research#279 batched float16 notebook and nine sample images scripts/setup_environment.py diagnoses groundingdino._C loading pyproject.toml IDEA-Research#279/IDEA-Research#417/IDEA-Research#419 pixi.toml conda dependencies for default, export and test ======================================================================= 5. Tests ======================================================================= Upstream had no test suite: two files named test_*, neither containing an assertion, and no CI. tests/test_upstream_regressions.py demonstrates that the merged changes were needful, on the standard that a change is needful only if its absence is observable. Every test was also run against a worktree of pre-integration-backup, where the file reports 11 failed and 1 passed -- the pass being the deliberate control. Nine PRs are covered this way. The decisive case is IDEA-Research#458: on upstream main build_model() raises "AttributeError: 'BertModel' object has no attribute 'get_head_mask'" under transformers 5.x, so the library cannot construct a model at all. Result: 12 passed. Coverage and its limits are recorded in tests/README.md. The CUDA and HIP kernel work (IDEA-Research#383, IDEA-Research#409, IDEA-Research#415, IDEA-Research#424, IDEA-Research#449, IDEA-Research#459, IDEA-Research#462), the MPS fixes (IDEA-Research#349) and the fp16 kernel path (IDEA-Research#279) cannot be exercised without the hardware. IDEA-Research#417 and IDEA-Research#393 were checked and are forward-looking rather than currently broken: timm 1.0.28 still ships the timm.models.layers shim, and torch 2.13 only warns about meshgrid indexing= and checkpoint use_reentrant=. ======================================================================= 6. Verification ======================================================================= groundingdino imports and reports version 0.1.0; build_model() succeeds on CPU; forward() and forward_nn() produce bitwise-identical output; tests 12 passed; find_packages() returns the 7 groundingdino.* packages; pixi resolves 3 environments; documentation links resolve. Not verified: the CUDA and HIP kernels are uncompiled, no GPU or nvcc being available, and the ONNX export path was not run end to end. ======================================================================= Attribution ======================================================================= Squashing removes the per-PR authorship the "pre-squash" tag still carries. The upstream contributors whose work is included: Abb :D, Alex, Andrew Choi, andy371643, Artemis Panagopoulou, ASHWIN UNNIKRISHNAN, awpbash, ayushpixis5941, dcaustin33, Deltaheavy6, dpkimball-cursor, Dr. Artificial曾小健, Freax Ruby, Functionhx, hujian, Jiani Huang, jinwoo, joncrall, jvarshini99, Kenshi Takayama, Mohamad Al Mdfaa, pftq, PierreMarieCurie, Ren Tianhe, robostoneDK, Salvador Belenguer, Skwarson96, szsteven008, Xiaoke JIANG, Yu-Chung Wang, Yushuo Sun, zhengjia, Ziqi Gao (Roy), zjli2013, 叨叨 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.