Agentic builds: dsynth evidence capture hooks#1517
Open
tuxillo wants to merge 56 commits into
Open
Conversation
Add dsynth hook scripts that snapshot distilled build errors and relevant port metadata on failures, grouped by run, so debugging can stay build-driven without keeping full workdirs. Document the bounded evidence contract and the planned opencode integration/central queue model for asynchronous triage.
Add observe-only state server for remote UI integration: - REST API for runs, jobs, bundles, ports, artifacts - SSE event stream with replay support - SQLite persistence for full history - Filesystem reconciler for live updates Validated on DragonFlyBSD VM - all endpoints tested.
- Add vanilla JS Bootstrap 5 UI served by state-server - Live SSE event stream with replay/reconnect - Views: Overview, Events, Jobs, Runs, Ports, Bundles - Artifact viewer for markdown, diffs, logs - SSE improvements: after_id, tail query params, ts in payloads
- Add /bundles API endpoint listing recent bundles - Add #/bundles route with renderBundles() view - Add Bundles nav item to navbar - Update Phase 9 docs with completion status and new route
- agent-queue-runner: add apply job type and iteration tracking - apply-patch: add DragonFly local mode, --no-push flag, BSD-compatible patch - hook_common.sh: detect rebuild iterations, track previous bundles - Add KEDB entry for DragonFly source patch conventions
Makefiles use tabs, not spaces. The agent was generating patches with spaces which caused patch application failures. Added rule #8 to emphasize preserving exact whitespace from the bundle context.
When retrying a patch application, the branch may already exist from a previous failed attempt. Delete it first to allow the retry.
Stop extraction when hitting common section markers like 'Rationale', 'Files Modified', etc. Also detect when prose text starts after hunks. This prevents non-diff content from being included in patch.diff.
The agent was generating patches with incorrect hunk line counts. Added detailed instructions on unified diff format with example.
- Change dports-patch prompt to request complete file contents - Add extract_files_from_response() to parse FILE content blocks - Add generate_unified_diff() to create diffs programmatically - Add generate_combined_diff() for multi-file patches - Update write_patch_outputs() to try new format first, fallback to legacy This fixes the malformed diff issue - LLMs are good at generating file content but struggle with unified diff syntax and line counts.
The agent was outputting diff syntax inside FILE blocks for Makefile.DragonFly. Make it explicit that Makefile.DragonFly should be raw makefile content, while dragonfly/patch-* files are actual diffs. Also add specific hint for the IFM_IEEE80211_VHT5G error.
…er UI - Add activity_log and runner_status tables to state-server schema - Add /activity and /runner-status API endpoints with SSE events - Update agent-queue-runner to log activities at all job stages - Add heartbeat thread for runner liveness detection (5s interval) - UI: Add Activity Log panel showing last 10 runner activities - UI: Add Runner Status indicator with staleness detection (>15s) - UI: Add back button for artifact navigation in bundle view - UI: Hide session_id.txt files from artifact lists
…b error display - state-server: Only emit runner_status SSE events when status/job_id/stage changes, not on every heartbeat update_at change - app.js: Don't trigger full re-render for runner_status/activity events (fixes bundle tab reset issue), only re-render on overview page - app.js: Add renderJobDetail() with prominent error display and related activity log entries for failed jobs - agent-queue-runner: Write .job.error files before moving failed jobs, move error files along with job files
Added bundle_upserted, artifact_put, and user_context_updated event types that were being emitted by artifact-store but not registered in the UI's EventSource listeners.
Added explicit guidance to prevent agents from outputting raw diff text or writing patch files directly. Agents must use dports_genpatch and dports_install_patches tools to generate proper dragonfly/patch-* files.
- Updated workspace paths to use lowercase (freebsd-ports, deltaports, dports) - Fixed dsynth profile to use agentic-workspace directories - Added -p agentic flag to dsynth test command - Added optional DP_*_DIR environment variable overrides - Fixed dsynth.ini indentation and paths
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
We are designing a system to automatically (agent-assisted) fix ports while keeping the existing, build-driven workflow intact:
dsynthstays the authoritative build executor.What this PR adds (foundation)
scripts/dsynth-hooks/:hook_run_start/hook_run_endgroup failures per build run and snapshot dsynth summary lists.hook_pkg_failurecreates a per-failure evidence bundle with:logs/errors.txt(high-signal extract, capped at 200KB)logs/full.log.gz(full log preserved for humans)port/*snapshot (Makefile/distinfo/pkg-plist/patches, etc.)meta.txtand basic dsynth profile/config snapshotsdocs/AGENTIC_BUILDS.mddescribing:What this PR does not do (yet)
Those are intentionally deferred so this PR can land the core evidence-capture mechanism safely and independently.
How to try it
scripts/dsynth-hooks/hook_*andscripts/dsynth-hooks/hook_common.shinto dsynth’s config base (/etc/dsynth/or/usr/local/etc/dsynth/) and making them executable.dsynthnormally.${Directory_logs}/evidence/runs/.../ports/.../for the evidence bundle.Why this matters for automated fixing
Reliable, size-capped evidence capture is the prerequisite for an automated port-fixing system:
errors.txt+ port metadata)dsynth-driven, and automation can be layered on without destabilizing build infrastructure