Skip to content

feat: add value-free teacher mode - #2699

Merged
webbrain-one merged 5 commits into
webbrain-one:mainfrom
alectimison-maker:feat/teacher-mode
Aug 8, 2026
Merged

feat: add value-free teacher mode#2699
webbrain-one merged 5 commits into
webbrain-one:mainfrom
alectimison-maker:feat/teacher-mode

Conversation

@alectimison-maker

Copy link
Copy Markdown
Contributor

Summary

  • add /teach --start <name>, /teach, and /teach --end
  • capture trusted user demonstrations in a tab-scoped temporary session
  • compile demonstrations into the existing webbrain-workflow/1 format
  • reuse the saved workflow store and guarded replay executor
  • add Chrome/Firefox parity, localized UI messages, and documentation

Design

Teacher mode captures trusted top-document clicks, field completion,
checkbox/radio changes, Enter submissions, and navigation.

Field values are never read or stored. A demonstrated field is converted
directly into a runtime workflow parameter at the capture/compiler boundary.
URLs are reduced to sanitized origins, path families, and query-free replay
destinations.

Temporary sessions live in storage.session, survive navigation and service
worker restarts, and are always removed by /teach --end, including failed
compilations. Unsafe targets and actions beyond the workflow limit are skipped
with explicit warnings.

This introduces no new workflow schema or browser permissions.

Scope

  • captures the top-level document only
  • cross-origin iframe demonstrations are intentionally deferred
  • supports clicks, text/select fields, checkbox/radio state, Enter submission,
    and explicit navigation
  • limits demonstrations to 100 recorded actions

Testing

  • node test/run.js: 1518 passed, 1 existing upstream failure
    • existing failure: package version 26.2.2 is newer than the latest
      changelog entry 26.2.0
  • security injection corpus: 60/60 passed
  • rich-text toolbar guard: 33 passed
  • Chrome/Firefox syntax and parity checks passed
  • trusted-event browser harness passed for fields, checkboxes, Enter submission,
    pending-field flush, duplicate-submit suppression, and value non-retention
  • contenteditable privacy harness confirmed entered text is not retained

Closes #197

@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

@webbrain-one

Copy link
Copy Markdown
Owner

Review finding: request changes before merge.

Teacher mode prevents /teach --start when an agent is already running, but it does not prevent an agent, scheduled job, cloud run, or saved-workflow run from starting after the teacher session is active. The capture script treats any event.isTrusted input as a user demonstration, while WebBrain deliberately dispatches CDP mouse/key events that are isTrusted. Those automated actions and navigations can therefore be recorded and compiled as if the user demonstrated them, which breaks the PR's trusted-user-demonstration boundary and can produce an incorrect saved workflow.

Suggested fix:

  • Enforce teacher/run mutual exclusion at a central run-entry boundary covering interactive, scheduled, cloud, and workflow runs; either reject the run or explicitly stop the teacher session before automation begins.
  • Also guard record_teacher_action and teacher navigation recording while an agent owns the tab, as defense in depth.
  • Add a regression test for: start teacher mode → begin an automated run → dispatch trusted click/type/navigation → confirm no automated action is retained (or confirm the run is rejected).

Validation otherwise looks good: local test/run.js reached 1518/1519; the sole failure is the existing package.json 26.2.2 vs CHANGELOG.md 26.2.0 mismatch on main. The security corpus passed 60/60, CI passed 14 scenarios, cloud-capture passed, and GitHub's WebMCP smoke check passed.

Since #2698 has now merged, this branch also needs a rebase/conflict resolution that preserves both the workflow-manager UI and the /teach additions.

@webbrain-one

Copy link
Copy Markdown
Owner

Implemented the requested teacher/run exclusion and rebased the PR onto current main while retaining the teacher-mode work.

What changed:

  • Added one agent run-entry guard used by interactive, streaming, scheduled, cloud, and saved-workflow runs. An active teacher session now rejects the run with teacher_mode_active before agent automation starts.
  • Scheduled and cloud paths reserve the target tab and invoke the same guard before scheduled URL navigation, cloud tab activation, or capture setup.
  • Teacher startup is serialized against those run reservations, closing the start/run race.
  • record_teacher_action and teacher navigation now return agent_running without retaining the event whenever interactive, detached, scheduled, cloud, or agent execution owns the tab.
  • Added a Chrome/Firefox regression that starts teacher mode, attempts interactive/scheduled/cloud/workflow/streaming runs, verifies zero automated dispatch, then injects click/type/navigation capture while agent-owned and verifies the session still has zero actions.

Validation:

  • Node suite: 1,533 tests passed, including the new regression and scheduler/cloud tests. The only local failure is the packaging assertion for the absent untracked dist/webbrain-chrome-27.0.0.zip artifact.
  • Security corpus: 60/60 passed.
  • Rich-text toolbar guard: 33/33 passed.
  • CI unit/cloud-capture scenarios passed.

PR head is now 38beb4a and GitHub reports it mergeable; the WebMCP smoke check is queued.

@webbrain-one

Copy link
Copy Markdown
Owner

Follow-up: the WebMCP smoke check completed successfully (22s). The only remaining failing status is Vercel's external authorization-required check.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds value-free Teacher mode for capturing user demonstrations and compiling them into reusable saved workflows.

Changes:

  • Adds /teach session capture, workflow compilation, and guarded replay integration.
  • Adds Chrome/Firefox parity, localization, and automated tests.
  • Documents Teacher mode, architecture, and privacy behavior.

Reviewed changes

Copilot reviewed 70 out of 72 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
test/run.js Adds Teacher-mode compilation, storage, interlock, and wiring tests.
src/firefox/src/ui/sidepanel.js Adds Firefox /teach commands and UI handling.
src/firefox/src/ui/locales/zh.js Adds Chinese Teacher-mode strings.
src/firefox/src/ui/locales/vi.js Adds Vietnamese Teacher-mode strings.
src/firefox/src/ui/locales/uk.js Adds Ukrainian Teacher-mode strings.
src/firefox/src/ui/locales/tr.js Adds Turkish Teacher-mode strings.
src/firefox/src/ui/locales/tl.js Adds Tagalog Teacher-mode strings.
src/firefox/src/ui/locales/th.js Adds Thai Teacher-mode strings.
src/firefox/src/ui/locales/ru.js Adds Russian Teacher-mode strings.
src/firefox/src/ui/locales/pt.js Adds Portuguese Teacher-mode strings.
src/firefox/src/ui/locales/pl.js Adds Polish Teacher-mode strings.
src/firefox/src/ui/locales/nl.js Adds Dutch Teacher-mode strings.
src/firefox/src/ui/locales/ms.js Adds Malay Teacher-mode strings.
src/firefox/src/ui/locales/ko.js Adds Korean Teacher-mode strings.
src/firefox/src/ui/locales/ja.js Adds Japanese Teacher-mode strings.
src/firefox/src/ui/locales/id.js Adds Indonesian Teacher-mode strings.
src/firefox/src/ui/locales/hi.js Adds Hindi Teacher-mode strings.
src/firefox/src/ui/locales/he.js Adds Hebrew Teacher-mode strings.
src/firefox/src/ui/locales/fr.js Adds French Teacher-mode strings.
src/firefox/src/ui/locales/fa.js Adds Persian Teacher-mode strings.
src/firefox/src/ui/locales/es.js Adds Spanish Teacher-mode strings.
src/firefox/src/ui/locales/en.js Adds English Teacher-mode strings.
src/firefox/src/ui/locales/de.js Adds German Teacher-mode strings.
src/firefox/src/ui/locales/bn.js Adds Bengali Teacher-mode strings.
src/firefox/src/ui/locales/ar.js Adds Arabic Teacher-mode strings.
src/firefox/src/content/teacher-capture.js Captures trusted, value-free Firefox demonstrations.
src/firefox/src/content/accessibility-tree.js Exposes semantic role lookup.
src/firefox/src/background.js Coordinates Firefox Teacher sessions and workflow saving.
src/firefox/src/agent/workflows.js Compiles demonstrations into saved workflows.
src/firefox/src/agent/teacher-mode.js Implements session storage and run interlocking.
src/firefox/src/agent/scheduler.js Reserves scheduled-run tabs against Teacher mode.
src/firefox/src/agent/agent.js Adds guarded run claiming.
src/firefox/manifest.json Registers the Teacher capture script.
src/chrome/src/ui/sidepanel.js Adds Chrome /teach commands and UI handling.
src/chrome/src/ui/locales/zh.js Adds Chinese Teacher-mode strings.
src/chrome/src/ui/locales/vi.js Adds Vietnamese Teacher-mode strings.
src/chrome/src/ui/locales/uk.js Adds Ukrainian Teacher-mode strings.
src/chrome/src/ui/locales/tr.js Adds Turkish Teacher-mode strings.
src/chrome/src/ui/locales/tl.js Adds Tagalog Teacher-mode strings.
src/chrome/src/ui/locales/th.js Adds Thai Teacher-mode strings.
src/chrome/src/ui/locales/ru.js Adds Russian Teacher-mode strings.
src/chrome/src/ui/locales/pt.js Adds Portuguese Teacher-mode strings.
src/chrome/src/ui/locales/pl.js Adds Polish Teacher-mode strings.
src/chrome/src/ui/locales/nl.js Adds Dutch Teacher-mode strings.
src/chrome/src/ui/locales/ms.js Adds Malay Teacher-mode strings.
src/chrome/src/ui/locales/ko.js Adds Korean Teacher-mode strings.
src/chrome/src/ui/locales/ja.js Adds Japanese Teacher-mode strings.
src/chrome/src/ui/locales/id.js Adds Indonesian Teacher-mode strings.
src/chrome/src/ui/locales/hi.js Adds Hindi Teacher-mode strings.
src/chrome/src/ui/locales/he.js Adds Hebrew Teacher-mode strings.
src/chrome/src/ui/locales/fr.js Adds French Teacher-mode strings.
src/chrome/src/ui/locales/fa.js Adds Persian Teacher-mode strings.
src/chrome/src/ui/locales/es.js Adds Spanish Teacher-mode strings.
src/chrome/src/ui/locales/en.js Adds English Teacher-mode strings.
src/chrome/src/ui/locales/de.js Adds German Teacher-mode strings.
src/chrome/src/ui/locales/bn.js Adds Bengali Teacher-mode strings.
src/chrome/src/ui/locales/ar.js Adds Arabic Teacher-mode strings.
src/chrome/src/content/teacher-capture.js Captures trusted, value-free Chrome demonstrations.
src/chrome/src/content/accessibility-tree.js Exposes semantic role lookup.
src/chrome/src/cloud-runs.js Interlocks cloud-run startup with Teacher mode.
src/chrome/src/background.js Coordinates Chrome Teacher sessions and workflow saving.
src/chrome/src/agent/workflows.js Compiles demonstrations into saved workflows.
src/chrome/src/agent/teacher-mode.js Implements session storage and run interlocking.
src/chrome/src/agent/scheduler.js Reserves scheduled-run tabs against Teacher mode.
src/chrome/src/agent/agent.js Adds guarded run claiming.
src/chrome/manifest.json Registers the Teacher capture script.
README.md Lists the new Teacher commands.
docs/zh-CN/slash-commands.md Documents Teacher commands in Chinese.
docs/slash-commands.md Documents Teacher command usage.
docs/privacy-and-data-flow.md Describes value-free capture and storage.
docs/fr/slash-commands.md Documents Teacher commands in French.
docs/architecture.md Documents Teacher-mode architecture.
Files not reviewed (2)
  • src/chrome/src/ui/locales/bn.js: Generated file
  • src/chrome/src/ui/locales/hi.js: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +194 to +201
document.addEventListener('keydown', (event) => {
if (!active || !event.isTrusted || event.key !== 'Enter' || event.repeat) return;
const element = actionableFromEvent(event);
if (!isField(element) || element.matches('textarea,[contenteditable]')) return;
dirtyFields.delete(element);
submittedFields.set(element, Date.now());
enterSubmitAt = Date.now();
sendAction({ kind: 'field', submit: true, target: semanticTarget(element, { field: true }) });
Comment on lines +194 to +201
document.addEventListener('keydown', (event) => {
if (!active || !event.isTrusted || event.key !== 'Enter' || event.repeat) return;
const element = actionableFromEvent(event);
if (!isField(element) || element.matches('textarea,[contenteditable]')) return;
dirtyFields.delete(element);
submittedFields.set(element, Date.now());
enterSubmitAt = Date.now();
sendAction({ kind: 'field', submit: true, target: semanticTarget(element, { field: true }) });
Comment thread src/chrome/src/content/teacher-capture.js
Comment thread src/firefox/src/content/teacher-capture.js
Comment thread src/chrome/src/agent/teacher-mode.js Outdated
Comment on lines +188 to +192
const recentClick = options.force !== true
&& session.lastActionKind === 'click'
&& now() - session.lastActionAt < 2000;
session.currentScope = destination;
if (sameScope || recentClick) return { changed: true, reason: '', session: await write(session) };
Comment thread src/firefox/src/agent/teacher-mode.js Outdated
Comment on lines +188 to +192
const recentClick = options.force !== true
&& session.lastActionKind === 'click'
&& now() - session.lastActionAt < 2000;
session.currentScope = destination;
if (sameScope || recentClick) return { changed: true, reason: '', session: await write(session) };
throw new Error('Saved workflow is missing or invalid.');
}
await this._hydrate(tabId);
await this._claimRunEntry(tabId, 'workflow', runOptions);
throw new Error('Saved workflow is missing or invalid.');
}
await this._hydrate(tabId);
await this._claimRunEntry(tabId, 'workflow', runOptions);
Comment thread src/chrome/src/agent/scheduler.js Outdated
Comment thread src/firefox/src/agent/scheduler.js Outdated
esokullu and others added 3 commits August 8, 2026 17:48
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@webbrain-one
webbrain-one merged commit b11ceee into webbrain-one:main Aug 8, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

teacher mode

4 participants