Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ Type `/help` in the panel for full signatures and flags. The most useful ones:
| `/schedule [prompt]` | Create a scheduled task |
| `/watch [--keep] [--secs <30-120>] [--long \| --short] <condition and action> [/beep]` | Poll the current page and act when a condition is met |
| `/workflow` · `/workflow --save <name>` | Manage saved workflows or compile the last successful run |
| `/teach --start <name>` · `/teach --end` | Learn a reusable workflow from your demonstrated actions |
| `/memory --add <text>` | Save a user preference |
| `/screenshot [--full-page]` | Capture the tab, or the full scrollable page |
| `/record [--transcribe]` | Record the current tab, optionally saving a transcript |
Expand Down
8 changes: 8 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -531,6 +531,14 @@ action CSS selectors, coordinates, query strings, fragments, and typed values. E
value becomes a declared runtime parameter; unsupported or failed actions are
skipped and reported to the user as save warnings.

Teacher mode is a second, value-free compiler input for the same schema.
`content/teacher-capture.js` observes only trusted top-document clicks, field
completion, checkbox/radio changes, Enter submissions, and navigation while a
tab-scoped session is active. Field values are never read or sent; a field
action becomes a runtime parameter marker as soon as it crosses the capture
boundary. The sanitized session survives navigation in `storage.session` and
is removed when `/teach --end` compiles and stores the workflow.

Each compiled step contains semantic target metadata (role, accessible name,
label, field identity, link, or placeholder), an expected postcondition, and
the origin/path family observed before that action. `/workflow` renders a
Expand Down
3 changes: 3 additions & 0 deletions docs/fr/slash-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ d'une espace ouvre l'autocomplétion de ses options disponibles.
| `/workflow --delete <id>` | Supprimer un workflow enregistré |
| `/workflow --export <id>` | Télécharger un fichier JSON portable `webbrain-workflow/1` assaini |
| `/workflow --import --file` | Importer un fichier de workflow portable comme nouveau workflow local |
| `/teach` | Afficher l’état du mode d’apprentissage et le nombre d’actions capturées dans l’onglet actuel |
| `/teach --start <nom>` | Commencer à apprendre un workflow à partir de vos clics et modifications de champs |
| `/teach --end` | Arrêter l’apprentissage et compiler les actions en workflow enregistré sans valeurs saisies |
| `/allow-api` | **Dérogation de mutation API par conversation.** Voir [plus bas](#allow-api). |
| `/foreground [invite]` | Exécuter une tâche locale au premier plan pour assurer la compatibilité visuelle |
| `/dangerously-skip-permissions` | **Contournement global des demandes d'autorisation.** Désactive `Ask before consequential actions` sans ouvrir les Paramètres. WebBrain agira sans demandes par site jusqu'à ce que vous réactiviez le réglage. |
Expand Down
9 changes: 9 additions & 0 deletions docs/privacy-and-data-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,15 @@ postconditions, and parameter descriptors. It does not contain typed field
values, raw historical `ref_id` values, action CSS selectors, coordinates, URL query strings, or URL
fragments.

`/teach --start <name>` records a user demonstration in temporary,
tab-scoped session storage. The page capture code never reads field values:
it sends only the field's semantic identity, and the compiler immediately
represents that action as a runtime parameter. Click/field target labels and
sanitized URL origin/path families are retained because replay needs them to
find the same controls. `/teach --end` removes the temporary session whether
compilation succeeds or fails; a successful compilation writes the same
`webbrain-workflow/1` format described above.

`/workflow --run <id>` collects declared values in a temporary side-panel form
and sends them directly to the background replay executor. The values are not
written to the workflow, chat text, retry payload, user memory, replay trace,
Expand Down
3 changes: 3 additions & 0 deletions docs/slash-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ for its available flags.
| `/workflow --delete <id>` | Delete a saved workflow |
| `/workflow --export <id>` | Download a sanitized portable `webbrain-workflow/1` JSON file |
| `/workflow --import --file` | Import a portable workflow file as a new local saved workflow |
| `/teach` | Show the current tab's Teacher mode status and captured action count |
| `/teach --start <name>` | Start learning a saved workflow from your demonstrated clicks and field edits |
| `/teach --end` | Stop teaching and compile the captured actions into a value-free saved workflow |
| `/allow-api` | **Per-conversation API mutation override.** See [below](#allow-api). |
| `/foreground [prompt]` | Run one local task in the foreground for visual compatibility |
| `/dangerously-skip-permissions` | **Global permission-prompt bypass.** Turns off `Ask before consequential actions` without opening Settings. WebBrain will act without per-site prompts until you re-enable the setting. |
Expand Down
3 changes: 3 additions & 0 deletions docs/zh-CN/slash-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ WebBrain 接受作为输入框某行开头的斜杠命令。在面板内输入 `
| `/workflow --delete <id>` | 删除已保存的工作流 |
| `/workflow --export <id>` | 下载经过清洗的可移植 `webbrain-workflow/1` JSON 文件 |
| `/workflow --import --file` | 将可移植工作流文件导入为新的本地工作流 |
| `/teach` | 显示当前标签页的示教状态和已捕获操作数 |
| `/teach --start <名称>` | 开始从您演示的点击和字段编辑中学习工作流 |
| `/teach --end` | 结束示教,并将捕获的操作编译为不含实际输入值的已保存工作流 |
| `/allow-api` | **按对话的 API 变更覆盖。** 参见[下文](#allow-api)。 |
| `/foreground [提示词]` | 为视觉兼容性在前台运行一次本地任务 |
| `/dangerously-skip-permissions` | **全局权限提示绕过。** 无需打开设置即可关闭「操作前询问」。在你重新启用该设置之前,WebBrain 将不再按站点弹出提示。 |
Expand Down
1 change: 1 addition & 0 deletions src/chrome/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
"js": [
"src/content/rich-text-toolbar-heuristic.js",
"src/content/accessibility-tree.js",
"src/content/teacher-capture.js",
"src/content/content.js",
"src/content/agent-visual-indicator.js",
"src/content/ollama-launch-handoff.js"
Expand Down
55 changes: 42 additions & 13 deletions src/chrome/src/agent/agent.js
Original file line number Diff line number Diff line change
Expand Up @@ -558,6 +558,7 @@ export class Agent extends LoopDetector {
this._recentSubmitClicks = new Map(); // tabId -> recent submit click timestamps
this._formValidationBlocks = new Map(); // tabId -> validation state that must change before another submit
this._runningTabs = new Set(); // tabIds with an active processMessage/Stream in flight
this._runStartGuard = null;
// Ordinary runs capture and act in their original tab without activating
// it. Managed cloud runs and the explicit /foreground compatibility
// override retain the old Page.bringToFront behavior for their lifetime.
Expand Down Expand Up @@ -896,6 +897,38 @@ export class Agent extends LoopDetector {
this._conversationScopeChangeListener = typeof listener === 'function' ? listener : null;
}

setRunStartGuard(guard) {
this._runStartGuard = typeof guard === 'function' ? guard : null;
}

_runEntryKind(defaultKind, runOptions = {}) {
if (runOptions?.cloudRun === true) return 'cloud';
if (runOptions?.scheduledRun === true) return 'scheduled';
return defaultKind;
}

async assertRunStartAllowed(tabId, defaultKind = 'interactive', runOptions = {}) {
await this._runStartGuard?.(tabId, {
kind: this._runEntryKind(defaultKind, runOptions),
runOptions,
});
}

async _claimRunEntry(tabId, defaultKind, runOptions = {}) {
if (this._runningTabs.has(tabId)) {
throw new Error('An agent run is already in progress for this tab.');
}
// Claim synchronously before awaiting the external guard so teacher-mode
// startup cannot race a run whose persisted teacher-state check is pending.
this._runningTabs.add(tabId);
try {
await this.assertRunStartAllowed(tabId, defaultKind, runOptions);
} catch (error) {
this._runningTabs.delete(tabId);
throw error;
}
}

isRunning(tabId) {
return this._runningTabs.has(tabId);
}
Expand Down Expand Up @@ -15577,11 +15610,16 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}

async replaySavedWorkflow(tabId, workflow, parameters = {}, onUpdate = () => {}, runOptions = {}) {
if (this._runningTabs.has(tabId)) throw new Error('An agent run is already in progress for this tab.');
if (!workflow?.id || !Array.isArray(workflow.steps) || !workflow.steps.length) {
throw new Error('Saved workflow is missing or invalid.');
}
await this._hydrate(tabId);
await this._claimRunEntry(tabId, 'workflow', runOptions);
try {
await this._hydrate(tabId);
} catch (error) {
this._runningTabs.delete(tabId);
throw error;
}
// Deterministic workflow replay is an independent task even when it never
// falls back to processMessage. Clear stale selected-text grounding before
// any replay action so a successful replay cannot poison the next turn.
Expand All @@ -15598,7 +15636,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
this.permissions.beginTurn(tabId);
this.conversationModes.set(tabId, 'act');
const completionRunToken = this._beginCompletionInvariant(tabId);
this._runningTabs.add(tabId);
const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions);
const startUrl = await this._currentUrl(tabId);
const conversationId = await this.ensureConversationId(tabId, 'act');
Expand Down Expand Up @@ -21948,10 +21985,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
}

async processMessage(tabId, userMessage, onUpdate = () => {}, mode = 'ask', attachments = [], runOptions = {}) {
if (this._runningTabs.has(tabId)) {
throw new Error('An agent run is already in progress for this tab.');
}
this._runningTabs.add(tabId);
await this._claimRunEntry(tabId, 'interactive', runOptions);
try {
// Hydration has to run before the toolbar-ledger reset below, so a
// persisted obligation cannot outlive the run that cleared it. It is
Expand All @@ -21972,7 +22006,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId);
this._clickAxCdpFallbacks.delete(tabId);
const completionRunToken = this._beginCompletionInvariant(tabId);
this._runningTabs.add(tabId);
if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId);
this._runUpdateCallbacks.set(tabId, onUpdate);
const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions);
Expand Down Expand Up @@ -22874,10 +22907,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
* Process a message with streaming output.
*/
async processMessageStream(tabId, userMessage, onUpdate = () => {}, mode = 'ask', runOptions = {}) {
if (this._runningTabs.has(tabId)) {
throw new Error('An agent run is already in progress for this tab.');
}
this._runningTabs.add(tabId);
await this._claimRunEntry(tabId, 'interactive', runOptions);
try {
// Hydration has to run before the toolbar-ledger reset below, so a
// persisted obligation cannot outlive the run that cleared it. It is
Expand All @@ -22898,7 +22928,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d
if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId);
this._clickAxCdpFallbacks.delete(tabId);
const completionRunToken = this._beginCompletionInvariant(tabId);
this._runningTabs.add(tabId);
if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId);
this._runUpdateCallbacks.set(tabId, onUpdate);
const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions);
Expand Down
47 changes: 40 additions & 7 deletions src/chrome/src/agent/scheduler.js
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,10 @@ export class ScheduledJobManager {
this.restoreAlarms().catch((e) => console.warn('[WebBrain] restore scheduled alarms failed:', e));
}

isRunning(tabId) {
return this._runningTabs.has(tabId);
}

async _getJobs() {
const stored = await this.api.storage.local.get(SCHEDULED_JOBS_KEY);
const jobs = stored?.[SCHEDULED_JOBS_KEY];
Expand Down Expand Up @@ -1523,20 +1527,49 @@ export class ScheduledJobManager {
}

let tabId;
let reservedTabId = null;
const releaseReservation = () => {
if (reservedTabId == null) return;
this._runningTabs.delete(reservedTabId);
reservedTabId = null;
};
const reserveTab = async (candidateTabId) => {
if (candidateTabId == null) return;
const numericTabId = Number(candidateTabId);
const resolvedTabId = Number.isFinite(numericTabId) ? numericTabId : candidateTabId;
if (this._runningTabs.has(resolvedTabId) || this.agent.isRunning(resolvedTabId)) {
throw new Error('An agent run is already in progress for this tab.');
}
this._runningTabs.add(resolvedTabId);
reservedTabId = resolvedTabId;
try {
await this.agent.assertRunStartAllowed?.(resolvedTabId, 'scheduled', { scheduledRun: true });
} catch (error) {
releaseReservation();
throw error;
}
};
try {
const candidateTabId = job.kind === 'resume' || job.target?.type === 'current_tab'
? (job.tabId || job.target?.tabId)
: job.target?.tabId;
await reserveTab(candidateTabId);
tabId = await this._resolveTab(job);
if (reservedTabId !== tabId) {
releaseReservation();
await reserveTab(tabId);
}
await this._validateConversation(job, tabId);
await this._validateTaskTarget(job, tabId);
} catch (e) {
await this._markFailed(job, e.message);
return;
}

if (this._runningTabs.has(tabId) || this.agent.isRunning(tabId)) {
await this._requeue(job, 'The target tab already has an active WebBrain run.');
releaseReservation();
if (isActiveRunError(e)) {
Comment thread
esokullu marked this conversation as resolved.
Outdated
await this._requeue(job, 'The target tab already has an active WebBrain run.');
} else {
await this._markFailed(job, e.message);
}
return;
}
this._runningTabs.add(tabId);

const running = await this._updateJobIf(job.id, (prev) => (
['pending', 'queued'].includes(prev.status)
Expand Down
Loading
Loading